doc: Integrate more unsorted documents

Signed-off-by: Sol Jerome <solj@ices.utexas.edu>

git-svn-id: https://svn.mcs.anl.gov/repos/bcfg/trunk/bcfg2@5754 ce84e21b-d406-0410-9b95-82705330c041

11 files changed, 103 insertions, 546 deletions

diff --git a/doc/unsorted/altsrc.txt b/doc/unsorted/altsrc.txt
deleted file mode 100644
index 251781d28..000000000
--- a/doc/unsorted/altsrc.txt
+++ /dev/null
@@ -1,102 +0,0 @@
-.. -*- mode: rst -*-
-
-.. _unsorted-altsrc:
-
-===========================
-Fun and Profit using altsrc
-===========================
-
-Altsrc is a generic, bcfg2-server-side mechanism for performing
-configuration entry name remapping for the purpose of data binding.
-
-Use Cases
-=========
-
-* Equivalent configuration entries on different architectures with
-  different names
-
-* Mapping entries with the same name to different bind results in a
-  configuration (two packages with the same name but different types)
-
-* A single configuration entry across multiple specifications
-  (multi-plugin, or multi-repo)
-
-Examples
-========
-
-* Consider the case of /etc/hosts on linux and ``/etc/inet/hosts`` on
-  solaris. These files contain the same data in the same format,
-  and should typically be synchronized, however, exist in different
-  locations. Classically, one would need to create one entry for each in
-  Cfg or TCheetah and perform manual synchronization. Or, you could use
-  symlinks and pray. Altsrc is driven from the bundle side. For example:
-
-  .. code-block:: xml
-
-      <Bundle name='netinfo'>
-         <Group name='solaris'>
-           <Path name='/etc/inet/hosts' altsrc='/etc/hosts'/>
-         </Group>
-         <Group name='linux'>
-           <Path name='/etc/hosts'/>
-         </Group>
-      </Bundle>
-
-  In this case, when a solaris host gets the 'netinfo' bundle, it will
-  get the first Path entry, which includes an altsrc parameter. This
-  will cause the server to bind the entry as if it were a Path
-  called ``/etc/hosts``. This configuration entry is still called
-  ``/etc/inet/hosts``, and is installed as such.
-
-* On encap systems, frequently multiple packages of the same name, but
-  of different types will exist. For example, there might be an openssl
-  encap package, and an openssl rpm package. This can be dealt with
-  using a bundle like:
-
-  .. code-block:: xml
-
-      <Bundle name='openssl'>
-         <Package name='openssl' altsrc='openssl-encap'/>
-         <Package name='openssl' altsrc='openssl-rpm'/>
-      </Bundle>
-
-  This bundle will bind data for the packages "openssl-encap" and
-  "openssl-rpm", but will be delivered to the client with both packages
-  named "openssl" with different types.
-
-* Finally, consider the case where there exist complicated, but
-  completely independent specifications for the same configuration entry
-  but different groups of clients. The following bundle will allow the use
-  of two different TCheetah templates ``/etc/firewall-rules-external``
-  and ``/etc/firewall-rules-internal`` for different clients based on
-  their group membership.
-
-  .. code-block:: xml
-
-      <Bundle name='firewall'>
-         ...
-         <Group name='conduit'>
-           <Path name='/etc/firewall-rules' altsrc='/etc/firewall-rules-external'/>
-         </Group>
-         <Group name='internal'>
-           <Path name='/etc/firewall-rules' altsrc='/etc/firewall-rules-internal'/>
-         </Group>
-      </Bundle>
-
-* Consider the case where a variety of files can be constructed by a
-  single template (TCheetah or TGenshi). It would be possible to copy
-  this template into the proper location for each file, but that requires
-  proper synchronization upon modification and knowing up front what
-  the files will all be called. Instead, the following bundle allows
-  the use of a single template for all proper config file instances.
-
-  .. code-block:: xml
-
-      <Bundle name='netconfig'>
-        <Path name='/etc/sysconfig/network-scripts/ifcfg-eth0' altsrc='/etc/ifcfg-template'/>
-        <Path name='/etc/sysconfig/network-scripts/ifcfg-eth1' altsrc='/etc/ifcfg-template'/>
-        <Path name='/etc/sysconfig/network-scripts/ifcfg-eth2' altsrc='/etc/ifcfg-template'/>
-      </Bundle>
-
-  altsrc can be used as a parameter for any entry type, and can be used
-  in any structure, including Bundler and Base.
diff --git a/doc/unsorted/annotated_examples.txt b/doc/unsorted/annotated_examples.txt
deleted file mode 100644
index b5b7e6b4e..000000000
--- a/doc/unsorted/annotated_examples.txt
+++ /dev/null
@@ -1,191 +0,0 @@
-.. -*- mode: rst -*-
-
-.. _unsorted-annotated_examples:
-
-==================
-Annotated Examples
-==================
-
-ntp example
-===========
-
-Author: Jason Pepas 
-
-Here is a series of example configurations for bcfg2, each introducing
-another layer of functionality.
-
-* After each change, run 'bcfg-repo-validate -v'.  
-* Run the server with 'bcfg2-server -v'.  
-* Update the client with 'bcfg2 -v -d -n'. (will not actually make client changes)
-
-package only
-------------
-
-Our example starts with the bare minimum configuration setup.  We have
-a client, a profile group, a list of packages, and a base configuration.
-
-::
-
-    # cat Metadata/clients.xml 
-    <Clients version='3.0'>
-    <Client profile='fedora' pingable='N' pingtime='0' name='foo.bar.com'/>
-    </Clients>
-
-    # cat Metadata/groups.xml 
-    <Groups version='3.0'>
-    <Group profile='true' name='fedora' toolset='rh'/>
-    </Groups>
-
-    # cat Base/base.xml 
-    <Base>
-    <Group name='fedora'>
-    <Package name='ntp'/>
-    </Group>
-    </Base>
-
-    # cat Pkgmgr/packages.xml
-    <PackageList type='rpm' priority='0'>
-    <Package name='ntp' version='4.2.0.a.20050816-11.FC5'/>
-    </PackageList>
-
-add service
------------
-
-Configure the service, and add it to the base.::
-
-    # cat Svcmgr/services.xml 
-    <Services priority='0'>
-    <Service name='ntpd' status='on'/>
-    </Services>
-
-    # cat Base/base.xml 
-    <Base>
-    <Group name='fedora'>
-    <Package name='ntp'/>
-    <Service name='ntpd'/>
-    </Group>
-    </Base>
-
-add config file
----------------
-
-Setup an etc directory structure, and add it to the base.::
-
-    # cat Cfg/etc/ntp.conf/ntp.conf 
-    server ntp1.utexas.edu
-
-    # cat Base/base.xml 
-    <Base>
-    <Group name='fedora'>
-    <Package name='ntp'/>
-    <Service name='ntpd'/>
-    <Path name='/etc/ntp.conf'/>
-    </Group>
-    </Base>
-
-create a bundle
----------------
-
-The above configuration layout works fine for a single service, but
-that method of organization would quickly become a nightmare as you
-approach the number of packages, services, and config files required to
-represent a fully configured host.  Bundles allow the grouping of related
-configuration entries that are used to provide a single service. This
-is done for several reasons:
-
-* Grouping related things in one place makes it easier to add those
-  entries for a multiple groups of clients
-* Grouping entries into bundles makes their validation occur
-  collectively. This means that config files can override the
-  contents of packages. Also, config files are rechecked after
-  packages are upgraded, so that they can be repaired if the package
-  install clobbered them. 
-* Services associated with a bundle get restarted whenever any
-  entity in that bundle is modified. This ensures that new
-  configuration files and software are used after installation.
-
-The config file, package, and service are really all related components
-describing the idea of an ntp client, so they should be logically
-grouped together.  We use a bundle to accomplish this.::
-
-    # cat Bundler/ntp.xml
-    <Bundle name='ntp' version='2.0'>
-    <Package name='ntp'/>
-    <Service name='ntpd'/>
-    <Path name='/etc/ntp.conf'/>
-    </Bundle>
-
-After this bundle is created, it must be associated with a group (or
-groups). Add a bundle child element to the group(s) which should install
-this bundle.::
-
-
-    # cat Metadata/groups.xml
-    <Groups>
-    ...
-    <Group name='fedora'>
-      <Bundle name='ntp'/>
-    </Group>
-    ...
-    </Groups>
-
-Once this bundle is created, a client reconfigure will install these
-entries. If any are modified, then the ntpd service will be
-restarted. If you only want ntp configurations to be updated (and
-nothing else), the bcfg2 client can be run with a -b <bundle name>
-option that will only update entries in the specified bundle.
-
-mysql example
-=============
-
-Author: Patrick Ruckstuhl
-
-I had some time ago to continue with putting my configuration into
-bcfg2 and maybe this helps someone else.
-
-I added a new bundle:
-
-.. code-block: xml
-
-    <Bundle name="mysql-server" version="3.0">
-       <Path name="/root/bcfg2-install/mysql/users.sh"/>
-       <Path name="/root/bcfg2-install/mysql/users.sql"/>
-       <PostInstall name="/root/bcfg2-install/mysql/users.sh"/>
-       <Package name="mysql-server-4.1"/>
-       <Service name="mysql"/>
-    </Bundle>
-
-The `users.sh` script looks like this:
-
-.. code-block: sh
-
-    #!/bin/sh
-
-    mysql --defaults-extra-file=/etc/mysql/debian.cnf mysql \
-        < /root/bcfg2-install/mysql/users.sql
-
-On debian there is a user account in ``/etc/mysql/debian.cnf``
-automatically created, but you could also (manually) create a user in
-the database that has enough permissions and add the login information
-in a file yourself. This file looks like this::
-
-    [client]
-    host     = localhost
-    user     = debian-sys-maint
-    password = XXXXXXXXXX
-
-The ``users.sql`` looks like this::
-
-    DELETE FROM db;
-    INSERT INTO db VALUES ('localhost', 'phpmyadmin', 'pma', 'Y', 'Y',
-    'Y', 'Y', 'N', 'N', 'N', 'N', 'N', 'N', 'N', 'N');
-
-    DELETE FROM user WHERE User <> 'debian-sys-maint';
-    INSERT INTO user VALUES ('localhost', 'root', 'XXXXXXXXXXX', 'Y', 'Y',
-    'Y', 'Y', 'Y', 'Y', 'Y', 'Y', 'Y', 'Y', 'Y', 'Y', 'Y', 'Y', 'Y', 'Y',
-    'Y', 'Y', 'Y', 'Y', 'Y', '', '', '', '', 0, 0, 0);
-    INSERT INTO user VALUES ('localhost', 'pma', '', 'N', 'N', 'N', 'N',
-    'N', 'N', 'N', 'N', 'N', 'N', 'N', 'N', 'N', 'N', 'N', 'N', 'N', 'N',
-    'N', 'N', 'N', '', '', '', '', 0, 0, 0);
-
-    FLUSH PRIVILEGES;
diff --git a/doc/unsorted/boundentry.txt b/doc/unsorted/boundentry.txt
deleted file mode 100644
index 6dfa4b9bd..000000000
--- a/doc/unsorted/boundentry.txt
+++ /dev/null
@@ -1,15 +0,0 @@
-.. -*- mode: rst -*-
-
-.. _unsorted-boundentry:
-
-=============
-Bound Entries
-=============
-
-This feature is a mechanism to specify a full entry at once from a bundle. Traditionally, entries are defined in two stages. First, an abstract entry is defined in a bundle. This entry includes a type (the XML tag) and a name attribute. Then this entry is bound for a client, providing the appropriate instance of that entry for the client. Specifying a bound entry short-circuits this process; the only second stage processing on Bound entries is to remove the "Bound" prefix from the element tag. The use of a bound entry allows the single stage definition of a complete entry. Bound entries can be used for any type.
-
-.. code-block:: xml
-
-    <Bundle name='ntp'>
-       <BoundPackage name='ntp' type='deb' version='1:4.2.4p4+dfsg-3ubuntu2.1'/>
-    </Bundle>
diff --git a/doc/unsorted/contributors.txt b/doc/unsorted/contributors.txt
deleted file mode 100644
index b9dd7cd77..000000000
--- a/doc/unsorted/contributors.txt
+++ /dev/null
@@ -1,32 +0,0 @@
-.. -*- mode: rst -*-
-
-.. _unsorted-contributors:
-
-============
-Contributors
-============
-
-A large number of people have contributed to Bcfg2.
-
-* Narayan Desai - wrote and designed much of the bcfg1/2
-* Andrew Lusk - early development and hardening
-* Brian Pellin - early development and hardening
-* Rick Bradshaw
-* Cory Lueninghoener - Bugfixes and change management code
-* Scott Matott
-* Ti Leggett - ebuild packaging and bugfixes, RPM packaging
-* Joey Hagedorn - Mac OS X support in bcfg1, launchd support in Bcfg2, and both versions of the reporting system
-* Zach Lowry - Solaris support and general hardening
-* Jos Catnook - Bugfixes
-* Jason Pepas - Package manager source generation scripts
-* Patrick Ruckstuhl - Templating bugfixes
-* Chris Vuletich - statistics annotations and client features
-* Ken Raffenetti - Hostbase
-* Sami Haahtinen - Debian Packaging
-* Danny Clark - Encap Packaging
-* Michael Jinks - Gentoo support
-* David Dahl - Hostbase
-* Pedro Flores - Reporting system design help
-* David Strauss - CentOS, RHEL, Yum, and Bazaar VCS support
-
-The entire MCS systems team has provided invaluable help in the design process and refinement of the user interface. In particular, Gene Rackow and Sandra Bittner have provided great assistance throughout this project. Philip Steinbachs provided detailed feedback as an early external user.
diff --git a/doc/unsorted/doc_architecture.txt b/doc/unsorted/doc_architecture.txt
deleted file mode 100644
index 0aca0ef10..000000000
--- a/doc/unsorted/doc_architecture.txt
+++ /dev/null
@@ -1,128 +0,0 @@
-.. -*- mode: rst -*-
-
-.. _unsorted-doc_architecture:
-
-==================
-Bcfg2 Architecture
-==================
-
-Bcfg2 is based on a client-server architecture.  The client is responsible for interpreting (but not processing) the configuration served by the server.  This configuration is literal, so no local process is required.  After completion of the configuration process, the client uploads a set of statistics to the server.  This section will describe the goals and then the architecture motivated by it.
-
-Goals
-=====
-
-* '''Model configurations using declarative semantics.'''[[BR]]
-  Declarative semantics maximize the utility of configuration management tools; they provide the most flexibility for the tool to determine the right course of action in any given situation.  This means that users can focus on the task of describing the desired configuration, while leaving the task of transitioning clients states to the tool.
-* '''Configuration descriptions should be comprehensive.'''[[BR]]
-  This means that configurations served to the client should be sufficient to reproduce all desired functionality.  This assumption allows the use of heuristics to detect extra configuration, aiding in reliable, comprehensive configuration definitions.
-* '''Provide a flexible approach to user interactions.'''[[BR]]
-  Most configuration management systems take a rigid approach to user interactions; that is, either the client system is always correct, or the central system is.  This means that users are forced into an overly proscribed model where the system asserts where correct data is.  Configuration data modification is frequently undertaken on both the configuration server and clients.  Hence, the existence of a single canonical data location can easily pose a problem during normal tool use.  Bcfg2 takes a different approach.
-
-The default assumption is that data on the server is correct, however, the client has option to run in another mode where local changes are catalogued for server-side integration. If the Bcfg2 client is run in dry run mode, it can help to reconcile differences between current client state and the configuration described on the server.
-The Bcfg2 client also searches for extra configuration; that is, configuration that is not specified by the configuration description.  When extra configuration is found, either configuration has been removed from the configuration description on the server, or manual configuration has occurred on the client. Options related to two-way verification and removal are useful for configuration reconciliation when interactive access is used.
-
-* Plugins and administrative applications.
-* Incremental operations.
-
-The Bcfg2 Client
-================
-
-The Bcfg2 client performs all client configuration or reconfiguration operations. It renders a declarative configuration specification, provided by the Bcfg2 server, into a set of configuration operations which will, if executed, attempt to change the client's state into that described by the configuration specification. Conceptually, the Bcfg2 client serves to isolate the Bcfg2 server and specification from the imperative operations required to implement configuration changes.
-
-This isolation allows declarative specifications to be manipulated symbolically on the server, without needing to understand the properties of the underlying system tools. In this way, the  Bcfg2 client acts as a sort of expert system that "knows" how to implement declarative configuration changes.
-
-The operation of the Bcfg2 client is intended to be as simple as possible. The normal configuration process consists of four main steps:
-
-* Probe Execution
-  During the probe execution stage, the client connects to the server and downloads a series of probes to execute. These probes reveal local facts to the Bcfg2 server. For example, a probe could discover the type of video card in a system. The Bcfg2 client returns this data to the server, where it can influence the client configuration generation process.
-* Configuration Download and Inventory
-  The Bcfg2 client now downloads a configuration specification from the Bcfg2 server.  The configuration describes the complete target state of the machine. That is, all aspects of client configuration should be represented in this specification. For example, all software packages and services should be represented in the configuration specification. The client now performs a local system inventory.  This process consists of verifying each entry present in the configuration specification.  After this check is completed, heuristic checks for configuration not included in the configuration specification.  We refer to this inventory process as 2-way validation, as first we verify that the client contains all configuration that is included in the specification, then we check if the client has any extra configuration that isn't present.  This provides a fairly rigorous notion of client configuration congruence. Once the 2-way verification process has been performed, the client has built a list of all configuration entries that are out of spec.  This list has two parts: specified configuration that is incorrect (or missing) and unspecified configuration that should be removed.
-* Configuration Update
-  The client now attempts to update its configuration to match the specification. Depending on options, changes may not (or only partially) be performed.  First, if extra configuration correction is enabled, extra configuration can be removed. Then the remaining changes are processed. The Bcfg2 client loops while progress is made in the correction of these incorrect configuration entries. This loop results in the client being able to accomplish all it will be able to during one execution. Once all entries are fixed, or no progress is being made, the loop terminates.Once all configuration changes that can be performed have been, bundle dependencies are handled. Bundle groupings result in two different behaviors. Contained entries are assumed to be inter-dependant.  To address this, the client re-verifies each entry in any bundle containing an updates configuration entry. Also, services contained in modified bundles are restarted.
-* Statistics Upload
-  Once the reconfiguration process has concluded, the client reports information back to the server about the actions it performed during the reconfiguration process.  Statistics function as a detailed return code from the client. The server stores statistics information. Information included in this statistics update includes (but is not limited to):
-
- * Overall client status (clean/dirty)
- * List of modified configuration entries
- * List of uncorrectable configuration entries
-
-Architecture Abstraction
-------------------------
-
-The Bcfg2 client internally supports the administrative tools available on different architectures.  For example, rpm and apt-get are both supported, allowing operation of Debian, Redhat, SUSE, and Mandriva systems. The client toolset is specified in the configuration specification.  The client merely includes a series of libraries which describe how to interact with the system tools on a particular platform.
-
-Three of the libraries exist.  There is a base set of functions, which contain definitions describing how to perform POSIX operations. Support for configuration files, directories, and symlinks are included here. Two other libraries subclass this one, providing support for Debian and rpm-based systems.
-
-The Debian toolset includes support for apt-get and update-rc.d. These tools provide the ability to install and remove packages, and to install and remove services.
-
-The Redhat toolset includes support for rpm and chkconfig. Any other platform that uses these tools can also use this toolset. Hence, all of the other familiar rpm-based distributions can use this toolset without issue.
-
-Other platforms can easily use the POSIX toolset, ignoring support for packages or services.  Alternatively, adding support for new toolsets isn't difficult.  Each toolset consists of about 125 lines of python code.
-
-
-The Bcfg2 Server
-================
-
-The Bcfg2 server is responsible for taking a network description and turning it into a series of configuration specifications for particular clients.  It also manages probed data and tracks statistics for clients.
-
-The Bcfg2 server takes information from two sources when generating client configuration specifications.  The first is a pool of metadata that describes clients as members of an aspect-based classing system. That is, clients are defined in terms of aspects of their behavior. The other is a file system repository that contains mappings from metadata to literal configuration.  These are combined to form the literal configuration specifications for clients.
-
-
-The Configuration Specification Construction Process
-----------------------------------------------------
-
-As we described in the previous section, the client connects to the server to request a configuration specification. The server uses the client's metadata and the file system repository to build a specification that is tailored for the client. This process consists of the following steps:
-
-* Metadata Lookup
-  The server uses the client's IP address to initiate the metadata lookup. This initial metadata consists of a (profile, image) tuple. If the client already has metadata registered, then it is used. If not, then default values are used and stored for future use. This metadata tuple is expanded using some profile and class definitions also included in the metadata. The end result of this process is metadata consisting of hostname, profile, image, a list of classes, a list of attributes and a list of bundles.
-* Abstract Configuration Construction
-  Once the server has the client metadata, it is used to create an abstract configuration.  An abstract configuration contains all of the configuration elements that will exist in the final specification without any specifics. All entries will be typed (ie the tagname will be one of Package, !ConfigurationFile, Service, !Symlink, or Directory) and will include a name.  These configuration entries are grouped into bundles, which document installation time interdependencies.
-* Configuration Binding
-  The abstract configuration determines the structure of the client configuration, however, it doesn't contain literal configuration information. After the abstract configuration is created, each configuration entry must be bound to a client-specific value. The Bcfg2 server uses plugins to provide these client-specific bindings. The Bcfg2 server core contains a dispatch table that describes which plugins can handle requests of a particular type. The responsible plugin is located for each entry. It is called, passing in the configuration entry and the client's metadata. The behavior of plugins is explicitly undefined, so as to allow maximum flexibility. The behaviours of the stock plugins are documented elsewhere in this manual. Once this binding process is completed, the server has a literal, client-specific configuration specification. This specification is complete and comprehensive; the client doesn't need to process it at all in order to use it. It also represents the totality of the configuration specified for the client.
-
-The Literal Configuration Specification
-=======================================
-
-Literal configuration specifications are served to clients by the Bcfg2 server. This is a differentiating factor for Bcfg2; all other major configuration management systems use a non-literal configuration specification. That is, the clients receive a symbolic configuration that they process to implement target states. We took the literal approach for a few reasons:
-
-* A small list of configuration element types can be defined, each of which can have a set of defined semantics.  This allows the server to have a well-formed model of client-side operations.  Without a static lexicon with defined semantics, this isn't possible. This allows the server, for example, to record the update of a package as a coherent event.
-* Literal configurations do not require client-side processing. Removing client-side processing reduces the critical footprint of the tool.  That is, the Bcfg2 client (and the tools it calls) need to be functional, but the rest of the system can be in any state.  Yet, the client will receive a correct configuration.
-* Having static, defined element semantics also requires that all operations be defined and implemented in advance. The implementation can maximize reliability and robustness. In more ad-hoc setups, these operations aren't necessarily safely implemented.
-
-The Structure of Specifications
--------------------------------
-
-Configuration specifications contain some number of clauses.  Two types of clauses exist.  Bundles are groups of inter-dependent configuration entities.  The purpose of bundles is to encode installation-time dependencies such that all new configuration is properly activated during reconfiguration operations. T hat is, if a daemon configuration file is changed, its daemon should be restarted. Another example of bundle usage is the reconfiguration of a software package. If a package contains a default configuration file, but it gets overwritten by an environment-specific one, then that updated configuration file should survive package upgrade.  The purpose of bundles is to describe services, or reconfigured software packages.  Independent clauses contains groups of configuration entities that aren't related in any way.  This provides a convenient mechanism that can be used for bulk installations of software.
-
-Each of these clauses contains some number of configuration entities.  Five types of configuration entities exist: !ConfigurationFile, Package, !SymLink, Directory, and Service. Each of these correspond to the obvious system item. Configuration specifications can get quite large; many systems have specifications that top one megabyte in size. An example of one is included in an appendix.  These configurations can be written by hand, or generated by the server.  The easiest way to start using Bcfg2 is to write small static configurations for clients.  Once configurations get larger, this process gets unwieldy; at this point, using the server makes more sense.
-
-Design Considerations
-=====================
-
-This section will discuss several aspects of the design of bcfg2, and the particular use cases that motivated them. Initially, this will consist of a discussion of the system metadata, and the intended usage model for package indices as well.
-
-System Metadata
----------------
-
-Bcfg2 system metadata describes the underlying patterns in system configurations. It describes commonalities and differences between these specifications in a rigorous way. The groups used by bcfg2's metadata are responsible for differentiating clients from one another, and building collections of allocatable configuration.
-
-The Bcfg2 metadata system has been designed with several high-level goals in mind. Flexibility and precision are paramount concerns; no configuration should be undescribable using the constructs present in the bcfg2 repository. We have found (generally the hard way) that any assumptions about the inherent simplicity of configuration patterns tend to be wrong, so obscenely complex configurations must be representable, even if these requirements seem illogical during the implementation.
-
-In particular, we wanted to streamline several operations that commonly occurred in our environment.
-
-* Copying one node's profile to another node.
-  In many environments, many nodes are instances of a common configuration specification. They all have similar roles and software. In our environment, desktop machines were the best example of this. Other than strictly per-host configuration like SSH keys, all desktop machines use a common configuration specification. This trivializes the process of creating a new desktop machine.
-* Creating a specialized version of a currently existing profile.
-  In environments with highly varied configurations, departmental infrastructure being a good example, "another machine like X but with extra software" is a common requirement.  For this reason, it must be trivially possible to inherit most of a configuration specification from  some more generic source, while being able to describe overriding aspects in a convenient fashion.
-* 3. Compose several pre-existing configuration aspects to create a new profile.
-  The ability to compose configuration aspects allows the easy creation of new profiles based on a series of predefined set of configuration specification fragments. The end result is more agility in environments where change is the norm.
-
-  In order for a classing system to be comprehensive, it must be usable in complex ways. The Bcfg2 metadata system has constructs that map cleanly to first-order logic. This implies that any complex configuration pattern can be represented (at all) by the metadata, as first-order logic is provably comprehensive.  (There is a discussion later in the document describing the metadata system in detail, and showing how it corresponds to first-order logic)
-
-These use cases motivate several of the design decisions that we made:
-* There must be a many to one correspondence between clients and groups. Membership in a given profile group must imbue a client with all of its configuration properties.
-
-Package Management
-------------------
-
-The interface provided in the bcfg2 repository for package specification was designed with automation in mind.  The goal was to support an append only interface to the repository, so that users do not need to continuously re-write already existing bits of specification.
diff --git a/doc/unsorted/gentoo.txt b/doc/unsorted/gentoo.txt
index 2375fcb17..015a19687 100644
--- a/doc/unsorted/gentoo.txt
+++ b/doc/unsorted/gentoo.txt
@@ -6,46 +6,61 @@
 Gentoo
 ======
 
-''This document tries to lay out anything Gentoo-specific that you need to know in order to use bcfg2. Mostly that has to do with getting it to cooperate with the various pieces of Portage. Services, all things POSIX, and just about anything else that bcfg2 does will work the same on Gentoo as on any other distribution.  bcfg2 is new on Gentoo; please let the list know if you find errors or omissions.''
+This document tries to lay out anything Gentoo-specific that you need
+to know in order to use bcfg2. Mostly that has to do with getting it
+to cooperate with the various pieces of Portage. Services, all things
+POSIX, and just about anything else that bcfg2 does will work the same
+on Gentoo as on any other distribution.  bcfg2 is new on Gentoo; please
+let the list know if you find errors or omissions.
 
 Installing bcfg2
 ================
 
-Early in July 2008, bcfg2 was added to the Gentoo portage tree.  So far it's only keyworded for ~x86, but we hope to see it soon in the amd64 and x64-solaris ports.  If you're using Gentoo on some other architecture, it should still work provided that you have a reasonably up to date Python; try adding `app-admin/bcfg2 ~*` to your `/etc/portage/package.keywords` file.
+Early in July 2008, bcfg2 was added to the Gentoo portage tree.  So far
+it's only keyworded for ~x86, but we hope to see it soon in the amd64 and
+x64-solaris ports.  If you're using Gentoo on some other architecture, it
+should still work provided that you have a reasonably up to date Python;
+try adding `app-admin/bcfg2 ~*` to your `/etc/portage/package.keywords`
+file.
 
-If you don’t use emerge to install bcfg2, you’ll want to make sure you have all the prerequisites installed first. For a server, you’ll need:
+If you don’t use portage to install bcfg2, you’ll want to make sure you
+have all the prerequisites installed first. For a server, you’ll need:
 
-* `app-admin/gamin` or `app-admin/fam`;
-* `dev-python/pyopenssl`;
-* `dev-python/lxml`
+* ``app-admin/gamin`` or ``app-admin/fam``
+* ``dev-python/lxml``
 
 Clients will need at least:
 
-* `app-portage/gentoolkit`;
-* `dev-python/lxml` or `dev-python/elementtree` (if you’re running python-2.4 or below);
+* ``app-portage/gentoolkit``
 
 Package Repository
 ==================
 
 You’ll need (to make) at least one archive of binary packages. The
-Portage driver calls `emerge` with the `–getbinpkgonly` option. See
-:manpage:`make.conf(5)` and :manpage:`emerge(1)` manpages, specifically the `PORTAGE_BINHOST` environment variable.
+Portage driver calls ``emerge`` with the ``–getbinpkgonly`` option. See
+:manpage:`make.conf(5)` and :manpage:`emerge(1)` manpages, specifically
+the *PORTAGE_BINHOST* environment variable.
 
 Time Saver: quickpkg
 --------------------
 
-If you have a standing Gentoo machine that you want to preserve or propagate, you can generate a complete package archive based on the present state of the system by using the quickpkg utility. For example:
+If you have a standing Gentoo machine that you want to preserve or
+propagate, you can generate a complete package archive based on the
+present state of the system by using the quickpkg utility. For example:
 
 .. code-block:: sh
 
     for pkg in `equery -q l` ; do quickpkg "=$pkg" ; done
 
-…will leave you with a complete archive of all the packages on your system in `/usr/portage/packages/All`, which you can then move to your ftp server.
+…will leave you with a complete archive of all the packages on your
+system in ``/usr/portage/packages/All``, which you can then move to your
+ftp server.
 
 Cataloging Packages In Your Repository
 --------------------------------------
 
-Once you have a set of packages, you’ll need to create a catalog for them in /var/lib/bcfg2/Pkgmgr. Here’s a template:
+Once you have a set of packages, you will need to create a catalog for
+them in ``/var/lib/bcfg2/Pkgmgr``. Here’s a template:
 
 .. code-block:: xml
 
@@ -67,11 +82,17 @@ Once you have a set of packages, you’ll need to create a catalog for them in /
       </Group>
     </PackageList>
 
-The `<Group>` name (in our example, “gentoo-200701-vmware”) should be included by any host which will draw its packages from this list. Our collection of packages for this class of machines is at the listed URI, and we only have one collection of packages for this batch of machines so in our case the `priority` doesn’t really matter, we’ve set it to `0`.
+The `<Group>` name (in our example, “gentoo-200701-vmware”) should
+be included by any host which will draw its packages from this list. Our
+collection of packages for this class of machines is at the listed URI,
+and we only have one collection of packages for this batch of machines so
+in our case the `priority` doesn’t really matter, we’ve set it to `0`.
 
 Notice that package name fields are in `CAT/TITLE` format.
 
-Here’s a hack which will generate a list of Package lines from a system’s database of installed packages, especially useful in conjunction with the `quickpkg` example above:
+Here is a hack which will generate a list of Package lines from
+a system’s database of installed packages, especially useful in
+conjunction with the `quickpkg` example above:
 
 .. code-block:: sh
 
@@ -85,9 +106,16 @@ Here’s a hack which will generate a list of Package lines from a system’s da
 Configuring Client Machines
 ===========================
 
-Set up `/etc/bcfg2.conf` the way you would for any other bcfg2 client.
+Set up ``/etc/bcfg2.conf`` the way you would for any other bcfg2 client.
 
-In `make.conf`, set `PORTAGE_BINHOST` to point to the URI of your package repository.  You may want to create versions of `make.conf` for each package repository you maintain, with appropriate `PORTAGE_BINHOST` URI's in each, and associated with that package archive's group under `Cfg` -- for example, we have `Cfg/etc/make.conf/make.conf.G99_gentoo-200701-vmware`.  If a client host switches groups, and the new group needs a different set of packages, everything should just fall into place.
+In ``make.conf``, set *PORTAGE_BINHOST* to point to the URI of
+your package repository.  You may want to create versions of
+``make.conf`` for each package repository you maintain, with
+appropriate *PORTAGE_BINHOST* URI's in each, and associated with
+that package archive's group under ``Cfg`` -- for example, we have
+``Cfg/etc/make.conf/make.conf.G99_gentoo-200701-vmware``.  If a client
+host switches groups, and the new group needs a different set of packages,
+everything should just fall into place.
 
 Pitfalls
 ========
@@ -116,13 +144,34 @@ leading to noise like this::
     !!! /etc/ssh/ssh_config has incorrect md5sum
      * 62 out of 66 files good
 
-''We can ignore the lines for `ssh_config` and `sshd_config`; those will be caught by bcfg2 as registered config files and handled appropriately.''
-
-Because bcfg2 relies on the client system’s native package reporting tool to judge the state of installed packages, complaints like these about trivial or intractable verification failures can trigger unnecessary bundle reinstalls when the bcfg2 client runs. bcfg2 will catch on after a pass or two that the situation isn’t getting any better with repeated package installs, stop trying, and list those packages as “bad” in the client system’s statistics.
-
-Aside from filing bugs with the Gentoo package maintainers, your narrator has been unable to come up with a good approach to this. Maybe write a series of `Rules` definitions according to what the package database thinks it should find, and/or stage copies of affected files under `Cfg`, and associate those rules and files with the affected package in a bundle? Annoying but possibly necessary if you want your stats file to look good.
+We can ignore the lines for ``ssh_config`` and ``sshd_config``; those will
+be caught by bcfg2 as registered config files and handled appropriately.
+
+Because bcfg2 relies on the client system’s native package reporting
+tool to judge the state of installed packages, complaints like these
+about trivial or intractable verification failures can trigger unnecessary
+bundle reinstalls when the bcfg2 client runs. bcfg2 will catch on after a
+pass or two that the situation isn’t getting any better with repeated
+package installs, stop trying, and list those packages as “bad” in
+the client system’s statistics.
+
+Aside from filing bugs with the Gentoo package maintainers, your narrator
+has been unable to come up with a good approach to this. Maybe write a
+series of ``Rules`` definitions according to what the package database
+thinks it should find, and/or stage copies of affected files under
+``Cfg``, and associate those rules and files with the affected package in
+a bundle? Annoying but possibly necessary if you want your stats file
+to look good.
 
 /boot
 -----
 
-Gentoo as well as some other distros recommend leaving `/boot` unmounted during normal runtime. This can lead to trouble during verification and package installation, for example when `/boot/grub/grub.conf` turns up missing. The simplest way around this might just be to ensure that `/boot` is mounted whenever you run bcfg2, possibly wrapping bcfg2 in a script for the purpose. I’ve also thought about adding `Action` clauses to bundles for grub and our kernel packages, which would mount `/boot` before the bundle installs and unmount it afterward, but this doesn’t get around the problem of those packages flunking verification.
+Gentoo as well as some other distros recommend leaving ``/boot`` unmounted
+during normal runtime. This can lead to trouble during verification and
+package installation, for example when ``/boot/grub/grub.conf`` turns
+up missing. The simplest way around this might just be to ensure that
+``/boot`` is mounted whenever you run bcfg2, possibly wrapping bcfg2
+in a script for the purpose. I’ve also thought about adding *Action*
+clauses to bundles for grub and our kernel packages, which would mount
+``/boot`` before the bundle installs and unmount it afterward, but this
+doesn’t get around the problem of those packages flunking verification.
diff --git a/doc/unsorted/groupnames.txt b/doc/unsorted/groupnames.txt
deleted file mode 100644
index 7df223387..000000000
--- a/doc/unsorted/groupnames.txt
+++ /dev/null
@@ -1,17 +0,0 @@
-.. -*- mode: rst -*-
-
-.. _unsorted-groupnames:
-
-===========
-Group Names
-===========
-
-This pages describes a set of common group names. The goal is to build a small set of common group names that can be used across installations. With any luck, this will increase portability of repository contents. Use of these names is, and will continue to be entirely optional. None of these groups are profile groups. This will result in a highly functional example repository, hosted in [source:/trunk/repository svn]
-
-|| '''Group Name''' || '''Category''' || '''Semantic Content''' ||
-|| linux || os || Clients running the Linux kernel ||
-|| solaris || os || Clients running the Solaris kernel ||
-
-The following image describes the base compatibility OS groups for bcfg2. This file is available from [source:/trunk/repository/Metadata/bcfg2-os.xml].
-
-.. image:: viz.png
diff --git a/doc/unsorted/mailinglist.txt b/doc/unsorted/mailinglist.txt
index d7cf8a8ca..8f49b2706 100644
--- a/doc/unsorted/mailinglist.txt
+++ b/doc/unsorted/mailinglist.txt
@@ -6,9 +6,14 @@
 Mailing List
 ============
 
-To subscribe to the mailing list for bcfg2 please visit: [https://lists.mcs.anl.gov/mailman/listinfo/bcfg-dev bcfg-dev mailman page]
+To subscribe to the mailing list for bcfg2 please visit the `bcfg-dev
+mailman page`_
 
-[http://dir.gmane.org/gmane.comp.sysutils.bcfg2.devel Searchable archives] are available from Gmane. You can also read the mailing list from any NNTP client via Gmane.
+`Searchable archives`_ are available from Gmane. You can also read the
+mailing list from any NNTP client via Gmane.
+
+.. _bcfg-dev mailman page: https://lists.mcs.anl.gov/mailman/listinfo/bcfg-dev
+.. _Searchable archives: http://dir.gmane.org/gmane.comp.sysutils.bcfg2.devel
 
 .. raw:: html
 
diff --git a/doc/unsorted/nisauth.txt b/doc/unsorted/nisauth.txt
deleted file mode 100644
index 3dfad1d26..000000000
--- a/doc/unsorted/nisauth.txt
+++ /dev/null
@@ -1,19 +0,0 @@
-.. -*- mode: rst -*-
-
-.. _unsorted-nisauth:
-
-==================
-NIS Authentication
-==================
-
-'''Edit Django settings'''
-
-Django allows for custom authentication backends to its login procedure.  Hostbase has an NIS authentication backend that verifies a user to be in the unix group allowed to modify Hostbase.
-
-To enable this feature:
-
-* first edit your {{{Hostbase/settings.py}}} file and uncomment the line {{{'Hostbase.backends.NISBackend',}}} in the list of {{{AUTHENTICATION_BACKENDS}}}
-* enter the name of the unix group you want to give access to Hostbase in the {{{AUTHORIZED_GROUP}}} variable
-* in your {{{Hostbase/hostbase/views.py}}} file at the very bottom, uncomment the block(s) of lines that give you the desired level of access
-
-Hostbase will now direct the user to a login page if he or she is not authorized to view a certain page.  Users should log in with their regular Unix username and password.
diff --git a/doc/unsorted/ssl.txt b/doc/unsorted/ssl.txt
index 6189fdd3c..919f7ea71 100644
--- a/doc/unsorted/ssl.txt
+++ b/doc/unsorted/ssl.txt
@@ -8,9 +8,13 @@ Python SSL
 
 The ssl module can be found `here <http://pypi.python.org/pypi/ssl>`_.
 
-With this change, SSL certificate based client authentication is supported. In order to use this, based CA-type capabilities are required. A central CA needs to be created, with each server and all clients getting a signed cert. See [wiki:Authentication] for details.
+With this change, SSL certificate based client authentication is
+supported. In order to use this, based CA-type capabilities are
+required. A central CA needs to be created, with each server and all
+clients getting a signed cert. See [wiki:Authentication] for details.
 
-Setting up keys is accomplished with three settings, each in the "`[communication]`" section of bcfg2.conf::
+Setting up keys is accomplished with three settings, each in the
+"`[communication]`" section of bcfg2.conf::
 
     key = /path/to/ssl private key
     certificate = /path/to/signed cert for that key
@@ -20,14 +24,23 @@ Setting up keys is accomplished with three settings, each in the "`[communicatio
 Python SSL Backport Packaging
 =============================
 
-Both the Bcfg2 server and client are able to use the in-tree ssl module included with python 2.6. The client is also able to still use M2Crypto. A python ssl backport exists for 2.3, 2.4, and 2.5. With this, M2Crypto is not needed, and tlslite is no longer included with bcfg2 sources. See [wiki:Authentication] for details.
+Both the Bcfg2 server and client are able to use the in-tree ssl module
+included with python 2.6. The client is also able to still use M2Crypto. A
+python ssl backport exists for 2.3, 2.4, and 2.5. With this, M2Crypto
+is not needed, and tlslite is no longer included with bcfg2 sources. See
+[wiki:Authentication] for details.
 
-To build a package of the ssl backport for .deb based distributions that don't ship with python 2.6, you can follow these instructions, which use [http://github.com/astraw/stdeb/tree/master stdeb]. Alternatively if you happen to have .deb packaging skills, it would be great to get policy-complaint .debs into the major deb-based distributions.
+To build a package of the ssl backport for .deb based distributions
+that don't ship with python 2.6, you can follow these instructions,
+which use `stdeb`_. Alternatively if you happen to have .deb packaging
+skills, it would be great to get policy-complaint .debs into the major
+deb-based distributions.
+
+.. _stdeb: http://github.com/astraw/stdeb/tree/master
 
 The following commands were used to generate :download:`this
-<python-ssl_1.14-1_amd64.deb>` debian package ('''NOTE:''' Version
-numbers for the SSL module have changed). The `easy_install` command can
-be found in the `python-setuptools` package.::
+<python-ssl_1.14-1_amd64.deb>` debian package The ``easy_install`` command
+can be found in the `python-setuptools` package.::
 
     sudo aptitude install python-all-dev fakeroot
     sudo easy_install stdeb
@@ -39,6 +52,8 @@ be found in the `python-setuptools` package.::
     dpkg-buildpackage -rfakeroot -uc -us
     sudo dpkg -i ../python-ssl_1.14-1_amd64.deb
 
+.. note:: Version numbers for the SSL module have changed.
+
 For complete bcfg2 goodness, you'll also want to package stdeb using stdeb.
 The completed debian package can be grabbed from :download:`here
 <python-stdeb_0.3-1_all.deb>`, which was generated using the following::
diff --git a/doc/unsorted/writing_specification.txt b/doc/unsorted/writing_specification.txt
index 9fe847298..ce7630fde 100644
--- a/doc/unsorted/writing_specification.txt
+++ b/doc/unsorted/writing_specification.txt
@@ -13,7 +13,7 @@ Bcfg2 specifications are logically divided in to three areas:
 * Literal
 
 The metadata portion of the configuration assigns a client to its profile
-group and to its non profle groups. The profile group is assigned
+group and to its non-profile groups. The profile group is assigned
 in Metadata/clients.xml and the non profile group assignments are in
 Metadata/groups.xml.
 
@@ -22,7 +22,7 @@ an abstract configuration for the client. An abstract configuration for
 a client identifies the configuration entities (packages, configuration
 files, service, etc) that a client requires, but it does not identify
 them explicitly. For instance an abstract configuration may identify
-that a client needs the bcfg2 package with
+that a client needs the Bcfg2 package with
 
 .. code-block:: xml
 
@@ -353,11 +353,3 @@ Each of the generators is covered on their own page.
 
 || '' '''Plugin''' '' || '' '''Description''' '' ||
 || [wiki:Plugins/Actions Actions] || Action entries are commands that are executed either before bundle installation, after bundle installation or both. ||
-|| [wiki:Plugins/Cfg Cfg] || The Cfg plugin provides a repository to describe configuration file contents for clients. In its simplest form, the Cfg repository is just a directory tree modeled off of the directory tree on your client machines. ||
-|| [wiki:Plugins/Deps Deps] || The deps plugin allows inter-entry dependencies (like package prerequisites) to be mapped independent of client roles. ||
-|| [wiki:Plugins/Hostbase Hostbase] || Hostbase is an IP management system built on top of Bcfg2.  ||
-|| [wiki:Plugins/Pkgmgr Pkgmgr] || The Pkgmgr plugin define what packages are available for a host or image and where to find those packages. ||
-|| [wiki:Plugins/Rules Rules] || The Rules plugin is responsible for defining the details about the Directory, Permissions, !SymLink and Service elements.  ||
-|| [wiki:Plugins/SSHbase SSHbase] || SSHbase is a purpose build bcfg2 plugin for managing ssh host keys. It is responsible for making ssh keys persist beyond a client rebuild and building a proper ssh_known_hosts file, including a correct localhost record for the current system. ||
-|| [wiki:Plugins/TCheetah TCheetah] || The TCheetah plugin allows you to use the [http://www.cheetahtemplate.org/ cheetah templating system] to create files, instead of the various diff-based methods offered by the Cfg plugin. It also allows you to include the results of probes executed on the client in the created files. ||
-|| [wiki:Plugins/TGenshi TGenshi] || The TGenshi plugin uses the [http://genshi.edgewall.org/ genshi toolkit] for generating files and functions much like the TCheetah plugin ||