last step of repo switches

git-svn-id: https://svn.mcs.anl.gov/repos/bcfg/trunk/bcfg2@1716 ce84e21b-d406-0410-9b95-82705330c041

13 files changed, 971 insertions, 1077 deletions

diff --git a/doc/Makefile b/doc/Makefile
index dcbb6f8ea..3f3c06e5b 100644
--- a/doc/Makefile
+++ b/doc/Makefile
@@ -1,4 +1,5 @@
 SOURCES = *.xml
 
 manual.pdf: $(SOURCES)
-	docbook2pdf --dcl /usr/share/sgml/declaration/xml.dcl manual.xml
\ No newline at end of file
+	xsltproc --stringparam body.font.family Helvetica --stringparam fop.extensions 1 /usr/share/xml/docbook/stylesheet/nwalsh/fo/docbook.xsl manual.xml > manual.fo
+	fop -fo manual.fo -pdf manual.pdf
diff --git a/doc/arch.txt b/doc/arch.txt
deleted file mode 100644
index 73da91b53..000000000
--- a/doc/arch.txt
+++ /dev/null
@@ -1,25 +0,0 @@
-The BCFG2 Architecture consists of four main parts:
-
-The core is the component containing shared data:
- - file alteration monitor
- - metadata store
- - generators
- - structuring agents
- - change notification mechanism
-
-Generators: configuration atom construction mechanism
- - configuration repository
- - package management
- - service management
- - ssh key management
- - hostbase
-
-Structuring Agents - code that forms the configuration into
-      independant and dependant clauses
- - Bundler
- - Translator
- - Literal
- - Image data
-
-Client - state transition engine that runs on clients. 
- 
\ No newline at end of file
diff --git a/doc/architecture.xml b/doc/architecture.xml
index abb90c640..7c54fca45 100644
--- a/doc/architecture.xml
+++ b/doc/architecture.xml
@@ -72,7 +72,7 @@
       </listitem>
       <listitem>
 	<para>
-	  Generators and administrative applications.
+	  Plugins and administrative applications.
 	</para>
       </listitem>
       <listitem>
@@ -104,6 +104,12 @@
       simple as possible. The normal configuration process consists of
       four main steps:</para>
 
+<!--     <procedure> -->
+<!--       <title>Bcfg2 Client Execution Process</title> -->
+
+<!--       <step>foo</step> -->
+<!--     </procedure> -->
+
     <variablelist>
       <varlistentry>
 	<term>Probe Execution</term>
@@ -316,18 +322,18 @@
 	      literal configuration information. After the abstract
 	      configuration is created, each configuration entry must
 	      be bound to a client-specific value. The Bcfg2 server
-	      uses generators to provide these client-specific bindings.
+	      uses plugins to provide these client-specific bindings.
 	    </para>
 
 	    <para>
 	      The Bcfg2 server core contains a dispatch table that
-	      describes which generator can handle requests of a
-	      particular type. The responsible generator is located
+	      describes which plugins can handle requests of a
+	      particular type. The responsible plugin is located
 	      for each entry. It is called, passing in the
 	      configuration entry and the client's metadata. The
-	      behavior of generators is explicitly undefined, so as to
-	      allow maximum flexibility. The behavior of the stock
-	      generators is documented elsewhere in this manual.
+	      behavior of plugins is explicitly undefined, so as to
+	      allow maximum flexibility. The behaviors of the stock
+	      plugins are documented elsewhere in this manual.
 	    </para>
 
 	    <para>
diff --git a/doc/concepts.xml b/doc/concepts.xml
deleted file mode 100644
index 1d9c3c498..000000000
--- a/doc/concepts.xml
+++ /dev/null
@@ -1,111 +0,0 @@
-<chapter>
-  <title>Design Goals & Concepts</title>
-  
-  <para>
-    Bcfg2 was designed with several goals in mind. This section will
-    describe those goals, and how they were manifested in the
-    design. This section will also define important concepts used in
-    Bcfg2.
-  </para>
-
-  <section>
-    <title>Goals</title>
-
-    <itemizedlist>
-      <listitem>
-	<para>
-	  Model configurations using declarative
-	  semantics. Declarative semantics maximize the utility of
-	  configuration management tools; they provide the most
-	  flexibility for the tool to determine the right course of
-	  action in any given situation. This means that users can
-	  focus on the task of describing the desired configuration,
-	  while leaving the task of transitioning clients states to
-	  the tool. 
-	</para>
-      </listitem>
-      <listitem>
-	<para>
-	  Configuration descriptions should be comprehensive. This
-	  means that configurations served to the client should be
-	  sufficent to reproduce all desired functionality. This
-	  assumption allows the use of heuristics to detect extra
-	  configuration, aiding in reliable, comprehensive
-	  configuration definitions.
-	</para>
-      </listitem>
-      <listitem>
-	<para>
-	  Provide a flexible approach to user interactions. Most
-	  configuration management systems take a rigid approach to
-	  user interactions; that is, either the client system is
-	  always correct, or the central system is. This means that
-	  users are forced into an overly proscribed model where the
-	  system asserts where correct data is. Configuration data
-	  modification is frequently undertaken on both the
-	  configuration server and clients. Hence, the existance of a
-	  single canonical data location can easily pose a problem
-	  during normal tool use.
-	</para>
-	<para>
-	  Bcfg2 takes a different approach. The default assumption is
-	  that data on the server is correct, however, the client has
-	  option to run in another mode where local changes are
-	  catalogued for server-side integration. If the Bcfg2 client is
-	  run in dry run mode, it can help to reconcile differences
-	  between current client state and the configuration described
-	  on the server.
-	</para>
-	<para>
-	  The Bcfg2 client also searches for extra configuration; that
-	  is, configuration that is not specified by the configuration
-	  description. When extra configuration is found, either
-	  configuration has been removed from the configuration
-	  description on the server, or manual configuration has
-	  occurred on the client. Options related to two-way
-	  verification and removal are useful for configuration
-	  reconciliation when interactive access is used.
-	</para>
-      </listitem>
-      <listitem>
-	<para>
-	  Generators, and administrative applications.
-	</para>
-      </listitem>
-      <listitem>
-	<para>
-	  Imcremental operations.
-	</para>
-      </listitem>
-    </itemizedlist>
-  </section>
-
-  <section>
-    <title>Important Concepts</title>
-    <variablelist>
-      <varlistentry>
-	<term>Bundles</term>
-	<listitem>
-	  <para>
-	    Bundles are groups of interdependent configuration
-	    elements. Service configurations including software,
-	    configuration files, and service activations are a good
-	    example of bundles. When any of these components are
-	    modified, all should be re-checked, and any associated
-	    services should be restarted. We refer to this process as
-	    coherent reconfiguration; this guarentees that all
-	    configuration changes are active before reconfiguration
-	    has completed.
-	  </para>
-	</listitem>
-      </varlistentry>
-      <varlistentry>
-	<term>Metadata</term>
-	<listitem>
-	  <para/>
-	</listitem>
-      </varlistentry>
-    </variablelist>
-  </section>
-
-</chapter>
\ No newline at end of file
diff --git a/doc/deployment.xml b/doc/deployment.xml
index d687ea187..1114589d3 100644
--- a/doc/deployment.xml
+++ b/doc/deployment.xml
@@ -52,58 +52,6 @@
       ports.
     </para>
 
-    <variablelist>
-      <varlistentry>
-	<term>Package</term>
-	<listitem>
-	  <para>
-	    A software package. This entity includes a package name
-	    and version number. It may optionally include installation
-	    information (such as a package source URL) if one is
-	    needed.
-	  </para>
-	</listitem>
-      </varlistentry>
-      <varlistentry>
-	<term>ConfigFile</term>
-	<listitem>
-	  <para>
-	    A configuration file. This entity includes a file path,
-	    owner, group, permissions, and file contents. 
-	  </para>
-	</listitem>
-      </varlistentry>
-      <varlistentry>
-	<term>SymLink</term>
-	<listitem>
-	  <para>
-	    A symbolic link. This entity includes a source and
-	    destination. 
-	  </para>
-	</listitem>
-      </varlistentry>
-      <varlistentry>
-	<term>Service</term>
-	<listitem>
-	  <para>
-	    A service (de)activation. This controls services, a la
-	    chkconfig or update-rc.d. Services are restarted
-	    whenever co-located configuration entities are
-	    modified. This ensures that any configuration changes
-	    are flushed out to all active processes.
-	  </para>
-	</listitem>
-      </varlistentry>
-      <varlistentry>
-	<term>Directory</term>
-	<listitem>
-	  <para>
-	    A filesystem directory. This entity includes an owner,
-	    group and permissions.
-	  </para>
-	</listitem>
-      </varlistentry>
-    </variablelist>
   </section>
 
   <section>
@@ -129,166 +77,12 @@
   </section>
 
   <section>
-    <title>Object Oriented Configs</title>
-    <para>
-      One of the most powerful and useful parts about bcfg2 metadata
-      system is the ability to have truely Object Oriented configs. I
-      have found that this has made me understand the machines I am
-      managing in a whole new light. Instead of focusing on what is
-      installed, I now focus on how machines relate to each other, or
-      what pieces of the metadata are similar in their configs. To
-      illistrate this think about the following example machines:
-    </para>
-
-    <itemizedlist>
-      <listitem><para>A users desktop machine</para></listitem>
-      <listitem><para>A multiuser compute machine</para></listitem>
-      <listitem><para>A users home machine. (telecommuter)</para></listitem>
-    </itemizedlist>
-    <para>
-      These 3 machines have 3 distinct focuses for usage, but in reality
-      they can have a very similar metadata, depending on how the config is
-      broken up or the view that is taken of the config. If you focus first
-      on where the machines are the same it will help to build the common
-      metadata. below is what all 3 machines have in common:
-    </para>
-    <itemizedlist>
-      <listitem><para>users need to have the software they need to perform
-	  there work. </para></listitem>
-    </itemizedlist>
-    <para>
-      let just encode this into metadata by creating a Class called
-      common-software that contains all the common software between all 3
-      machines.
-    </para>
-    <programlisting>
-      <![CDATA[ 
-      <Class name='common-software'>
-      <Bundle ... />
-      ....
-    </Class>
-      ]]>
-    </programlisting>
-    <para>
-      now we need to find where they differ:
-    </para>
-    <variablelist>
-      <varlistentry>
-	<term>Desktop machines</term>
-	<listitem>
-	  <itemizedlist>
-	    <listitem>
-	      <para>need to have a GUI interface( Xwindows or what not )</para>
-	    </listitem>
-	    <listitem>
-	      <para>use NIS for authentification</para>
-	    </listitem>
-	    <listitem>
-	      <para>use Autofs to mount home directories</para>
-	    </listitem>
-	  </itemizedlist>
-	</listitem>
-      </varlistentry>
-      <varlistentry>
-	<term>Multiuser compute machines</term>
-	<listitem>
-	  <itemizedlist>
-	    <listitem>
-	      <para>only accessible by SSH, No GUI interface</para>
-	    </listitem>
-	    <listitem>
-	      <para>use NIS for authentification</para>
-	    </listitem>
-	    <listitem>
-	      <para>use Autofs to mount home directories</para>
-	    </listitem>
-	  </itemizedlist>
-	</listitem>
-      </varlistentry>
-      <varlistentry>
-	<term>Home machine</term>
-	<listitem>
-	  <itemizedlist>
-	    <listitem>
-	      <para>need to have a GUI interface( Xwindows or what not )</para>
-	    </listitem>
-	    <listitem>
-	      <para>use static password file</para>
-	    </listitem>
-	    <listitem>
-	      <para>use local disk for home directories</para>
-	    </listitem>
-	  </itemizedlist>
-	</listitem>
-      </varlistentry>
-    </variablelist> 
-    <para>
-      As you can see that there are common things pairwise in these configs
-      that can be further exploited by the object oriented system of bcfg2
-    </para>
-    <programlisting>
-      <![CDATA[ 
-      <Class name="Gui-Interface">
-      <Bundle ... />
-      ...
-    </Class>
-      <Class name="NIS-Autofs">
-      <Bundle ... />
-      ...
-    </Class>
-      ]]>
-    </programlisting>
-    <para>
-      now all that is left is to ensure that all needs are met and we find
-      we need to make one more class:
-    </para>
-    <programlisting>
-      <![CDATA[ 
-      <Class name="static-password-local-disk">
-      <Bundle ... />
-      ...
-    </Class>
-      ]]>
-    </programlisting>
-    <para>
-      Now we can mix and match these classes together to build the 3
-      profiles or even build new profiles with these descrete entities. 
-    </para>
-    <programlisting>
-      <![CDATA[ 
-      <Profile name='desktop'>
-      <Class name='common-software'/>
-      <Class name="Gui-Interface"/>
-      <Class name="NIS-Autofs">
-    </Profile>
-      <Profile name='computerserver'>
-      <Class name='common-software'/>
-      <Class name="NIS-Autofs">
-    </Profile>
-      <Profile name='home-desktop'>
-      <Class name='common-software'/>
-      <Class name="Gui-Interface"/>
-      <Class name="static-password-local-disk">
-    </Profile>
-      <Profile name='bare-system'>
-      <Class name='common-software'/>
-    </Profile>
-      ]]>
-    </programlisting>
-    <para>
-      The free form object oriented fashion in which metadata can be
-      constructed is truely a double edge sword. On one hand you can build
-      up a nice list of descrete entities that can compose even the most
-      complicated configs, but it also allows for the creation of entities
-      that could provide monolithic solutions to each machines config. It is
-      all in how one views the machines and how much they are willing to
-      harness the power of the OO based metadata system.
-    </para>
-  </section>
-  <section>
-    <title>Tips & Tricks</title>
+    <title>Bcfg2 Server Administration</title>
+    
     <para/>
+
   </section>
+
   <section>
     <title>An example application of bcfg2</title>
     <para>
diff --git a/doc/development.xml b/doc/development.xml
new file mode 100644
index 000000000..057d20786
--- /dev/null
+++ b/doc/development.xml
@@ -0,0 +1,165 @@
+<chapter>
+<title>Developing for Bcfg2</title>
+
+  <para>
+    While the Bcfg2 server provides a good interface for representing
+    general system configurations, its plugin interface offers the
+    ability to implement configuration interfaces and representation
+    tailored to problems encountered by a particular site. This
+    chapter describes what plugins are good for, what they can do, and
+    how to implement them.
+  </para>
+  
+  <section>
+    <title>Bcfg2 Plugins</title>
+
+    <para>
+      Bcfg2 plugins are loadable python modules that the Bcfg2 server
+      loads at initialization time. These plugins can contribute to
+      the functions already offered by the Bcfg2 server or can extend
+      its functionality. In general, plugins will provide some portion
+      of the configuration for clients, with a data representation
+      that is tuned for a set of common tasks. Much of the core
+      functionality of Bcfg2 is implemented by several plugins,
+      however, they are not special in any way; new plugins could
+      easily supplant one or all of them. 
+    </para>
+
+    <table>
+      <title>Bcfg2 Plugin Functions</title>
+      <tgroup cols='2'>
+	<colspec colnum='1'/>
+	<colspec colnum='2'/>
+	<thead>
+	  <row><entry>Name</entry><entry>Description</entry></row>
+	</thead>
+	<tbody>
+	  <row><entry>Probes</entry><entry>Plugins can send executable
+	  code to clients, where local contributions to configuration
+	  state can be gathered.</entry></row>
+	  <row><entry>Abstract Configuration Structures</entry>
+	    <entry>A plugin can define new groups of interdependent
+	    and independent configuration entities</entry></row>
+	  <row><entry>Literal Configuration Entities</entry>
+	    <entry>Plugins can provide literal configuration entity
+	    information.</entry></row>
+	  <row><entry>XML-RPC Functions</entry>
+	    <entry>Plugins can expose a set of functions through the
+	    Bcfg2 server's authenticated XML-RPC interface.</entry></row>
+	</tbody>
+      </tgroup>
+    </table>
+
+  </section>
+
+  <section>
+    <title>Writing Bcfg2 Plugins</title>
+
+    <para>
+      Bcfg2 plugins are python classes that subclass from
+      Bcfg2.Server.Plugin.Plugin. Several plugin-specific values must
+      be set in the new plugin. These values dictate how the new
+      plugin will behave with respect to the above four functions.
+    </para>
+
+
+    <table>
+      <title>Bcfg2 Plugin Members</title>
+      <tgroup cols='3'>
+	<colspec colnum='1' colwidth='1*'/>
+	<colspec colnum='2' colwidth='3*'/>
+	<colspec colnum='3' colwidth='2*'/>
+	<thead>
+	  <row><entry>Name</entry><entry>Description</entry><entry>Format</entry></row>
+	</thead>
+	<tbody>
+	  <row><entry>__name__</entry><entry>The name of the
+	      plugin</entry><entry>string</entry>
+	  </row>
+	  <row><entry>__version__</entry>
+	    <entry>The plugin version (generally tied to revctl
+	    keyword expansion).</entry><entry>string</entry></row>
+	  <row><entry>__author__</entry>
+	    <entry>The plugin author.</entry><entry>string</entry></row>
+	  <row><entry>__rmi__</entry>
+	    <entry>Set of functions to be exposed as XML-RPC
+	    functions</entry>
+	    <entry>List of function names (strings)</entry></row>
+	  <row><entry>Entries</entry><entry>Multidimentional
+	  dictionary of keys that point to the function used to bind
+	  literal contents for a given configuration
+	  entity.</entry><entry>Dictionary of
+	  ConfigurationEntityType, Name keys and function reference
+	  values</entry></row>
+	  <row><entry>BuildStructures</entry><entry>Function that
+	  returns a list of the structures for a given
+	  client</entry><entry>Member function</entry></row>
+	  <row><entry>GetProbes</entry><entry>Function that returns a
+	  list of probes that a given client should
+	  execute</entry><entry>Member function</entry></row>
+	  <row><entry>ReceiveData</entry><entry>Function that accepts
+	  the probe results for a given client.</entry><entry>Member
+	  function</entry></row>
+	</tbody>
+      </tgroup>
+    </table>
+    
+    <section>
+      <title>An Example Plugin</title>
+
+      <example>
+	<title>A Simple Plugin</title>
+      <programlisting>import socket, Bcfg2.Server.Plugin
+
+class Chiba(Bcfg2.Server.Plugin.Plugin):
+    '''the Chiba plugin builds the following files:
+      -> /etc/network/interfaces'''
+
+    __name__ = 'Chiba'
+    __version__ = '$Id: chiba.py 1702 2006-01-19 20:20:51Z desai '
+    __author__ = 'bcfg-dev@mcs.anl.gov'
+    Entries = {'ConfigFile':{}}
+
+    def __init__(self, core, datastore):
+        Bcfg2.Server.Plugin.Plugin.__init__(self, core, datastore)
+        self.repo = Bcfg2.Server.Plugin.DirectoryBacked(self.data, 
+	  self.core.fam)
+        self.Entries['ConfigFile']['/etc/network/interfaces'] \ 
+	  = self.build_interfaces
+
+    def build_interfaces(self, entry, metadata):
+        '''build network configs for clients'''
+        entry.attrib['owner'] = 'root'
+        entry.attrib['group'] = 'root'
+        entry.attrib['perms'] = '0644'
+        try:
+            myriaddr = socket.gethostbyname("%s-myr" % \
+	  metadata.hostname)
+        except socket.gaierror:
+            self.LogError("Failed to resolve %s-myr"% metadata.hostname)
+            raise Bcfg2.Server.Plugin.PluginExecutionError, ("%s-myr" \
+            % metadata.hostname, 'lookup')
+        entry.text = self.repo.entries['interfaces-template'].data % \
+	  myriaddr
+      </programlisting>
+    </example>
+
+    <para>
+	Bcfg2 server plugins must subclass the
+	Bcfg2.Server.Plugin.Plugin class. Plugin constructors must
+	take two arguments: an instance of a Bcfg2.Core object, and a
+	location for a datastore. __name__, __version__, __author__,
+	and Entries are used to describe what the plugin is and how it
+	works. Entries describes a set of configuration entries that
+	can be provided by the generator, and a set of handlers that
+	can bind in the proper data. build_interfaces is an example of
+	a handler. It gets client metadata and an configuration entry
+	passed in, and binds data into entry as appropriate. This
+	results in a <filename>/etc/network/interfaces</filename> file
+	that has static information derived from DNS for a given host.
+    </para>
+
+    </section>
+
+  </section>
+</chapter>
\ No newline at end of file
diff --git a/doc/generators.txt b/doc/generators.txt
deleted file mode 100644
index 4ad6dc74e..000000000
--- a/doc/generators.txt
+++ /dev/null
@@ -1,74 +0,0 @@
-A generator is a module, loaded into bcfg2's address space, that can
-be called to produce configuration data for clients. The code contains
-can do anything, so any configuration pattern can be modeled. These
-are useful for a variety of tasks; we currently use this interface to
-define all of our sources of configuration logic. While this construct
-was available in bcfg1, it suffered from a variety of issues, mainly
-related to performance. The new API provides for high enough speed
-execution that the generator API is the sole provider of configuration
-data to the bcfg2 server. 
-
-Several generators have been implemented in bcfg2:
-Cfg: A basic configuration file repository
-SSHBase: A ssh key management system
-Pkgmgr: Provides information about available software
-Svcmgr: Provides information about available services, and activations
-Debconf: Generator debconf settings based on probed client data
-
-Any of these generators can be enabled in bcfg2.conf, and custom
-generators can similarly be added. 
-
-Generators are during the client configuration process at two
-times. First, generators provide client-side probes. These can be used
-to determine local client state where appropriate. Generators can
-provide a list of probes for a client to execute. The data generated
-by this client execution is routed back to the source generator, which
-can then use this data to generate the client configuration. 
-
-The second function is to provide explicit values for the elements of
-a client configuration. This function is used in the following way:
-
-When a client requests its configuration, the bcfg2 daemon queries all
-structuring agents for pertinent configurations. In general, this
-means that all bundles and base configuration data for the appropriate
-image are constructed. At this point, all of these are abstract, that
-is, all entities contain name and type attributes, but don't contain
-versions, or explicit file contents or status. Generators are queried
-for individual configuration elements, of the form, which version of
-package zsh should be installed on client X. 
-
-Finding the generator responsible for a configuration entity has two
-steps. First, each generator has a dispatch table (called
-generator.__provides__) This dispatch table has a key for each type of
-configuration element it handles, which in turn has keys for
-particular instances of those types. For example, the generator that
-handled configuration file /etc/ssh/sshd_config would have the
-following dispatch table entry:
-self.__provides__['ConfigFile']['/etc/ssh/sshd_config']
-The value of this key is a function that can be called to bind in
-configuration elements. (like version, owner, permissions, or file
-data) 
-
-Generators can be activated in other ways, but i would just as soon
-not admit it in public. 
-
-Probes can be created in generators to discover information about
-clients. These xml elements look like:
-<Probe source='my-generator' name='test1' interpreter='/bin/sh'>
-hostname</Probe>
-
-A generator can create as many of these as needed. Bcfg2 will query
-for these by calling a generator's GetProbes method. By default, this
-function returns an empty list, but can return an arbitrary number of
-probe elements. 
-
-These probes are forwarded to the client, where they are
-executed. Output is collected and sent back to the bcfg2 server. the
-source attribute is used to determine which data belongs to which
-generator, and the AcceptProbeData method is called for each probe
-result belonging to the generator. This data may then be used in
-constructing configuration elements for the client. 
-
-There is a generator base class that provides skeleton methods, which
-custom generators should subclass. It is named
-Bcfg2.Server.Generator.Generator. 
\ No newline at end of file
diff --git a/doc/generators.xml b/doc/generators.xml
deleted file mode 100644
index 053c754e3..000000000
--- a/doc/generators.xml
+++ /dev/null
@@ -1,266 +0,0 @@
-<chapter>
-    <title>Generators</title>
-    
-    <para> 
-    Generators are modules which are loaded by the Bcfg2 server,
-    based on directives in <filename>/etc/bcfg2.conf</filename>. They
-    provide concrete, fully-specified configuration entries for
-    clients. This chapter documents the function and usage of
-    generators bundled with Bcfg2 releases. It also describes the
-    interface used to communicate with generators; modeles
-    implementing this interface can provide configuration elements for
-    clients based on any representation or requirements that may
-    exist.
-  </para>
-
-  <section>
-    <title>Bundled Generators</title>
-
-    <para>This section describes the generators that come bundled with
-    Bcfg2. As a general rule, generators requiring more than one
-    configuration file will use a generator specific directory in the
-    configuration repository. 
-    </para>
-
-    <section>
-      <title>Cfg</title>
-      <para>
-	The Cfg generator provides a configuration file repository
-	that uses literal file contents to provide client-tailored
-	configuration file entries. The Cfg generator chooses which
-	data to provide for a given client based on the aspect-based
-	metadata system used for high-level client configuration. 
-      </para>
-      <para>
-	The Cfg repository is structured much like the filesystem
-	hierarchy being configured. Each configuration file being
-	served has a corresponding directory in the configuration
-	repository. These directories have the same relative path as
-	the absolute path of the configuration file on the target
-	system. For example, if Cfg was serving data for the
-	configuration file <filename>/etc/services</filename>, then
-	its directory would be in the relative path
-	<filename>./etc/services</filename> inside of the Cfg
-	repository. 
-      </para>
-      <para>
-	Inside of this file-specific directory, three types of files
-	may exist. Base files are complete instances of configuration
-	file. Deltas are differences between a base file and the
-	target file contents. Base files and deltas are tagged with
-	metadata specifiers, which describe which groups of clients
-	the fragment pertains to. Configuration files are constructed
-	by finding the most specific base file and applying any more
-	specific deltas.
-      </para>
-      <para>
-	Specifiers are embedded in fragment filenames. For example, in
-	the fragment <filename>services.C99_webserver</filename>,
-	"C99_webserver" is the specifier. This specifier applies to
-	the class (C) webserver with a priority of 99. Other metadata
-	categories which can be used include bundle (B), profile (P),
-	hostname (H), attribute (A), and image (I). These are ordered
-	from least to most specific: image, profile, class, bundle,
-	and hostname. Global files are the least specific. Priorities
-	are used as to break ties.
-      </para>
-      <para>
-	Info files, named <filename>:info</filename> are used to
-	specify target configuration file metadata, such as owner,
-	group and permissions. If no <filename>:info</filename> is
-	provided, targets are installed with default
-	information. Default metadata is root ownership, root group
-	memberships, and 0644 file permissions.
-      </para>
-      <example>
-	<title>Cfg generator :info files</title>
-	<programlisting>
-	  owner:root
-	  group:root
-	  perms:0755
-	</programlisting>
-      </example>
-
-      <example>
-	<title>Cfg file repository example</title>
-	<programlisting>
-	  $ ls
-	  :info            passwd	  passwd.C99_chiba-login
-	  passwd.H_bio-debian  passwd.H_cvstest   passwd.H_foxtrot  
-	  passwd.H_reboot  passwd.H_rudy2    passwd.C99_netserv
-	  passwd.B99_tacacs-server.cat  passwd.H_adenine        
-	</programlisting>
-      </example>
-      
-      <para>
-	In the previous example, there exists files with each of the
-	characteristics mentioned above. All files ending in ".cat"
-	are deltas; ones with ".H_" are host specific files. There
-	exists a base file, a <filename>:info</filename> file, two
-	class-specified base files, and a bundle-specified base file.
-      </para>
-    </section>
-
-    <section>
-      <title>The Scoped XML File Group: Servicemgr</title>
-      <para>
-	The generator Servicemgr uses files formatted similarly to the
-	metadata files. It 
-	works based on a single file, which contains definitions
-	ordered by increasing specificity.
-      </para>
-
-      <example>
-	<title><filename>services.xml</filename></title>
-	<programlisting>
-	  <![CDATA[<Services>
-  <Class name='webserver'>
-    <Service status='on' name='httpd' port='80' protocol='tcp'>
-      <User address='0.0.0.0' mask='32'/>
-    </Service>
-  </Class>
-  <Host name='mailhost'>
-    <Service name='sendmail' status='on' protocol='tcp' port='80'>
-      <User address='0.0.0.0' mask='32'/>
-    </Service>
-  </Host>
-  <Host name='thai'>
-    <Service name='ssh' status='off'/>
-  </Host>
-  <Service name='ssh' status='on' protocol='tcp' port='22'/>
-  <Service name='ntp-server' status='on' />
-</Services>]]>
-	</programlisting>
-      </example>
-
-      <para>
-	This set of service definitions is intrepreted in the
-	following way. Webservers run httpd, the host mailhost runs
-	sendmail, and all machines run ssh, and the ntp-server. 
-      </para>
-    </section>
-
-  </section>
-
-  <section>
-    <title>The Generator API</title>
-    <para>
-      The Bcfg2 core has a well-formed API used to call
-      generators. This mechanism allows all stock generators to be
-      runtime selected; no stock generators are required. The
-      generator API has two main functions. The first is communication
-      to the Bcfg2 core: the list of entries a particular generator
-      can bind must be communicated to the core so that the proper
-      generator can be called. The second function is the actual
-      production of client-specific configuration element data; this
-      data is then included in client configurations.
-    </para>
-
-    <para>
-      The inventory function is provided by a python dictionary,
-      called __provides__ in each generator object. This dictionary
-      has a key for each type of configuration entry (ConfigFile,
-      Package, Directory, SymLink, Service), whose value is a
-      dictionary indexed by configuration element name. For example,
-      the data path to information about the service "sshd" could be
-      reached at __provides__['Service']['sshd']. The value of each of
-      these keys is a function that can be called to bind
-      client-specific values to a configuration entry. This function
-      is used in the next section. 
-    </para>
-
-    <para>
-      The handler function located by the __provides__ dictionary is
-      called with a static API. The function prototype for each of
-      these handlers is:
-    </para>
-    
-    <example>
-      <title>The Generator handler API</title>
-      <programlisting>
-	def Handler(self, entry, metadata):
-	    generator logic here
-      </programlisting>
-    </example>
-
-    <para>
-      The data supplied upon handler invokation includes two
-      parts. The first is the entry. This is a ElementTree.Element
-      object, which already contains the configuration element type
-      (ie Service) and name. All other data is bound into this object
-      in this function. The range of data bound depends on the data
-      type. The other data provided to handlers is client metadata,
-      information about the current client, including hostname, image,
-      profile, classes and bundles. The metadata is typically used to
-      choose entry contents.
-    </para>
-  </section>
-
-  <section>
-    <title>Writing a Generator</title>
-    <para>
-      Writing a generator is a fairly straightforward task. At a high
-      level, generators are instantiated by the Bcfg2 core, and then
-      used to provide configuration entry contents. This means that
-      the two points where control passes into a generator from Bcfg2
-      are during initial object instantiation, and every time a
-      generator-provided configuration entry is bound.
-    </para>
-
-    <para>
-      Currently, generators must be written in python. They can
-      perform arbitrary operations, hence, a generator could be
-      written that executed logic in another language, but this
-      functionality is currently not implemented. 
-    </para>
-
-    <example>
-      <title>Simple Generator</title>
-      <programlisting>
-	from socket import gethostbyname, gaierror
-	from syslog import syslog, LOG_ERR
-	from Bcfg2.Server.Generator import Generator, DirectoryBacked, SingleXMLFileBacked, GeneratorError
-
-class Chiba(Generator):
-    '''the Chiba generator builds the following files:
-      -> /etc/network/interfaces'''
-
-    __name__ = 'Chiba'
-    __version__ = '$Id: Chiba.py 1.12 05/01/15 11:05:02-06:00 desai@topaz.mcs.anl.gov $'
-    __author__ = 'bcfg-dev@mcs.anl.gov'
-    __provides__ = {'ConfigFile':{}}
-
-    def __init__(self, core, datastore):
-        Generator.__init__(self, core, datastore)
-        self.repo = DirectoryBacked(self.data, self.core.fam)
-        self.__provides__['ConfigFile']['/etc/network/interfaces'] = self.build_interfaces
-
-    def build_interfaces(self, entry, metadata):
-        '''build network configs for clients'''
-        entry.attrib['owner'] = 'root'
-        entry.attrib['group'] = 'root'
-        entry.attrib['perms'] = '0644'
-        try:
-            myriaddr = gethostbyname("%s-myr" % metadata.hostname)
-        except gaierror:
-            syslog(LOG_ERR, "Failed to resolve %s-myr"% metadata.hostname)
-            raise GeneratorError, ("%s-myr" % metadata.hostname, 'lookup')
-        entry.text = self.repo.entries['interfaces-template'].data % myriaddr
-      </programlisting>
-    </example>
-
-    <para>
-      Generators must subclass the Bcfg2.Server.Generator.Generator
-      class. Generator constructors must take two arguments: an
-      instance of a Bcfg2.Core object, and a location for a
-      datastore. __name__, __version__, __author__, and __provides__
-      are used to describe what the generator is and how it
-      works. __provides__ describes a set of configuration entries
-      that can be provided by the generator, and a set of handlers
-      that can bind in the proper data. build_interfaces is an example
-      of a handler. It gets client metadata and an configuration entry
-      passed in, and binds data into entry as appropriate. 
-    </para>
-
-  </section>
-</chapter>
\ No newline at end of file
diff --git a/doc/install.xml b/doc/install.xml
index 13cf2847f..27b88c636 100644
--- a/doc/install.xml
+++ b/doc/install.xml
@@ -5,8 +5,8 @@
     <title>Pre-requisites</title>
     <para>
       Bcfg2 is written in python using several modules not included
-      with most distributions. Element Tree, available from
-      http://www.effbot.org provides convenient XML handling.
+      with most distributions. lxml provides convenient xml
+      handling. M2crypto wraps openssl calls for https support. 
     </para>
     
     <para>
@@ -16,125 +16,95 @@
       use SSL functions.
     </para>
 
-    <para>ElementTree can be downloaded from
-    http://www.effbot.org/downloads. It can be installed by running
-    the setup script against the python installation. 
+    <para>lxml is required for xml parsing. It can be downloaded from
+    http://www.codespeak.net/lxml. It, in turn, requires libxml2,
+    libxslt, and pyrex. 
     </para>
-
-    <programlisting>$ python setup.py build
-running build
-running build_py
-creating build
-creating build/lib
-creating build/lib/elementtree
-copying elementtree/ElementInclude.py -> build/lib/elementtree
-copying elementtree/ElementPath.py -> build/lib/elementtree
-copying elementtree/ElementTree.py -> build/lib/elementtree
-copying elementtree/HTMLTreeBuilder.py -> build/lib/elementtree
-copying elementtree/SgmlopXMLTreeBuilder.py -> build/lib/elementtree
-copying elementtree/SimpleXMLTreeBuilder.py -> build/lib/elementtree
-copying elementtree/SimpleXMLWriter.py -> build/lib/elementtree
-copying elementtree/TidyHTMLTreeBuilder.py -> build/lib/elementtree
-copying elementtree/TidyTools.py -> build/lib/elementtree
-copying elementtree/XMLTreeBuilder.py -> build/lib/elementtree
-copying elementtree/__init__.py -> build/lib/elementtree
-$ python setup.py install
-...
-    </programlisting>
-
+    
     <para>
       The python fam binding can be downloaded from
       python-fam.sourceforge.net. FAM (on several linux distributions)
       has been depricated in favor of gamin. The Bcfg server will
       autodetect which modules are available, and use appropriate file
-      caching logic.
-    </para>
-  </sect1>
-  <sect1>
-    <title>Bcfg2 Installation</title>
-    <para>
+      caching logic. It can be installed by running the setup.py script.
     </para>
+
+    <table>
+      <title>Bcfg2 Software Prerequisites</title>
+      <tgroup cols='3'>
+	<colspec colnum='1' colwidth='2*'/>
+	<colspec colnum='2' colwidth='5*'/>
+	<colspec colnum='3' colwidth='8*'/>
+	<thead>
+	  <row><entry>Name</entry><entry>Description</entry><entry>URL</entry></row>
+	</thead>
+	<tbody>
+	  <row><entry>lxml</entry><entry>XML
+	  Processing</entry><entry><ulink
+	  url="http://codespeak.net/lxml"/></entry></row>
+	  <row><entry>pyrex</entry><entry>C to Python language
+	  interoperability (needed for lxml)</entry><entry><ulink
+	  url="http://www.cosc.canterbury.ac.nz/~greg/python/Pyrex"/></entry></row>
+	  <row><entry>M2Crypto</entry>
+	    <entry>OpenSSL bindings for Python</entry><entry><ulink
+	    url="http://wiki.osafoundation.org/bin/view/Projects/MeTooCrypto"/></entry></row>
+	  <row><entry>Swig</entry>
+	    <entry>C to Python language interoperability (needed for
+	    M2Crypto)</entry><entry><ulink
+	    url="http://www.swig.org"/></entry></row>
+	  <row><entry>Fam</entry><entry>File Alteration
+	  Monitor</entry><entry><ulink
+	  url="http://oss.sgi.com"/></entry></row>
+	  <row><entry>Gamin</entry><entry>Alternate File Alteration
+	      Monitor</entry><entry><ulink
+	      url="http://www.gnome.org/~veillard/gamin/"/></entry></row>
+	  <row><entry>Python-fam</entry><entry>Python bindings for fam
+	  (not needed with
+	  gamin)</entry><entry><ulink url="http://python-fam.sourceforge.net"/></entry></row>
+	</tbody>
+      </tgroup>
+    </table>
+
   </sect1>
   <sect1>
     <title>Bcfg2 Initial Setup and Testing</title>
     <para>Once the Bcfg2 software is installed, the configuration file
     and repository must be created. The example configuration file in
     <filename>bcfg2/examples/bcfg2.conf</filename> can be used, with
-    minor modifications. 
+    minor modifications.  This should be placed in
+    <filename>/etc/bcfg2.conf</filename>. If it is placed in another
+    location, each program takes a command line argument to specify
+    its alternate location. 
     </para>
+
     <example>
-      <title>bcfg2.conf</title>
+      <title>/etc/bcfg2.conf</title>
       <programlisting>[server]
-	repository = /disks/bcfg2
-	structures = Bundler,Base
-	generators = SSHbase,Cfg,Pkgmgr,Svcmgr
-	metadata = /disks/bcfg2/etc
-      </programlisting>
+repository = /disks/bcfg2
+structures = Bundler,Base
+generators = SSHbase,Cfg,Pkgmgr,Svcmgr</programlisting>
     </example>
-    <para>This configuration file sets the location of the
-      configuration repository. It also activates two structures, and
-      four generators. Structures are components that generate
-      abstract configuration fragments. These are the form of the
-      configuration. Generators provide client-specific values for
-      each configuration settings contained in all abstract
-      configuration fragments. Both of these are described in Section
-      ???.</para>
-  </sect1>
-  <sect1>
-    <title>Daemon Configuration</title> 
-    <para>Bcfg2 uses SSSlib, the
-      communication libraries from the Scalable Systems Software project
-      for communication abstraction. This library provides a unified
-      messaging interface on top of several wire protocols with
-      different authentication and encryption mechanisms. The default
-      protocol is "challenge" which is a challenge response protocol
-      with no data encryption. (SSL protection will be configured
-      later). SSSlib also includes service location functionality;
-      this allows software to locate components by name, regardless of
-      their respective network locations. This function is provided
-      with both static and dynamic implementations. Static component
-      location setup will be sufficient for most Bcfg2 deployments.
-    </para>
 
     <para>
-      Static component lookups depend on the file
-      <filename>/etc/sss.conf</filename>. This file contains
-      information about static service locations. This file must be
-      the same on the server and all clients for communication to work
-      properly. A location definition for the bcfg2 component will
-      allow all clients to find and connect to it. 
-    </para>
-    <example>
-      <title>/etc/sss.conf</title>
-      <programlisting>
-	<![CDATA[	<locations>
-	  <location component="bcfg2" host="bcfgserver"
-	  port="8052" protocol="challenge" schema_version="1.0" tier="1"/>
-	</locations>]]>
-      </programlisting>
-    </example>
-    <para>This allows SSSlib to locate the bcfg2 component on the
-    machine bcfgserver, port 8052, with the wire protocol "challenge".
+      This configuration file sets the top level location of the
+      configuration repository. It also activates two structures, and
+      four generators. Both structures and generators are instances of
+      Bcfg2 server plugins. Structures generate abstract configuration
+      fragments. These form the inventory of the
+      configuration. Generators provide client-specific literal values
+      for each configuration entity contained in the abstract
+      configuration.
     </para>
   </sect1>
-  <sect1>
-    <title>New-Style XML-RPC Deployments</title>
 
-    <para>
-      A new version of the Bcfg2 software is in testing that will
-      provide simplified and standards compliant communications
-      facilities. Instead of the use of SSSlib for communication, the
-      server and clients can use HTTPS XML-RPC instead. This has
-      required reimplementing the server and providing XML-RPC support
-      for the client, but provides drastically simplified setup for
-      new installs.
-    </para>
+  <sect1>
+    <title>Daemon Configuration</title>
 
     <para>
-      The prerequisite list now includes ElementTree, M2Crypto (for
-      SSL functions) and Python 2.2 or newer. ElementTree and M2Crypto
-      are both python modules that can be easily installed and are
-      already packaged for many Linux distributions.
+      Bcfg2 uses XML-RPC over HTTPS for all communications. 
+      All communications occur over this transport. HTTPS provides
+      data security, while an embedded username and password provide
+      authentication. 
     </para>
 
     <sect2>
@@ -146,33 +116,40 @@ $ python setup.py install
       </para>
       
       <programlisting>
-openssl req -x509 -nodes -days 1000 -newkey rsa:1024 -out server.pem -keyout server.pem
+openssl req -x509 -nodes -days 1000 -newkey rsa:1024 \
+	-out bcfg2.key -keyout bcfg2.key
       </programlisting>
 
-      <para>This command will generate an SSL key including both an
-      RSA key and a certificate. This is suitable for use with the
-      Bcfg2 XML-RPC server.</para>
+      <para>
+	This command will generate an SSL key including both an
+	RSA key and a certificate. This is suitable for use with the
+	Bcfg2 server. The path to this key should be put in the
+	bcfg2 configuration file in section communication, setting
+	key. 
+      </para>
 
     </sect2>
 
     <sect2>
-      <title>Communication Bootstrapping</title>
+      <title>Client Communication Setup</title>
 
       <para>
 	The Bcfg2 client must be able to find the server's
 	location. This is accomplished through the use of the
 	communication settings in <filename>/etc/bcfg2.conf</filename>
-	Two settings for the this section are required: protocol and
-	server url. 
+	Several settings must be included in this file: the server
+	url, a username and a password. 
       </para>
 
       <example>
-	<title>Bcfg2 XML-RPC Communication Settings</title>
-	<programlisting>
-	  [communication]
-	  protocol = xmlrpc/ssl
-	  url = https://localhost:9443
-	</programlisting>
+	<title>/etc/bcfg2.conf</title>
+	<programlisting>[communication]
+protocol = xmlrpc/ssl
+password = pwd
+user = root
+	  
+[components]
+bcfg2 = https://bcfg2server:8765</programlisting>
       </example>
     </sect2>
   </sect1>
diff --git a/doc/manual.xml b/doc/manual.xml
index 88d7e0eb3..878a104fb 100644
--- a/doc/manual.xml
+++ b/doc/manual.xml
@@ -1,40 +1,51 @@
 <!DOCTYPE book PUBLIC "-//OASIS//DTD DocBook V4.4//EN"
 "http://www.oasis-open.org/docbook/xml/4.4/docbookx.dtd"
 [
-<!ENTITY concepts SYSTEM "concepts.xml">
 <!ENTITY architecture SYSTEM "architecture.xml">
 <!ENTITY install SYSTEM "install.xml">
-<!ENTITY generators SYSTEM "generators.xml">
+<!ENTITY specification SYSTEM "specs.xml">
 <!ENTITY deployment SYSTEM "deployment.xml">
+<!ENTITY development SYSTEM "development.xml">
 <!ENTITY reports SYSTEM "reports.xml">
 ]>
 <book id="bcfg2">
   <bookinfo>
     <title>Bcfg2 Manual</title>
-    <author>
-      <firstname>Narayan</firstname>
-      <surname>Desai</surname>
-      <email>desai@mcs.anl.gov</email>
-    </author>
-    <author>
-      <firstname>Rick</firstname>
-      <surname>Bradshaw</surname>
-      <email>bradshaw@mcs.anl.gov</email>
-    </author>
-    <author>
-      <firstname>Joey</firstname>
-      <surname>Hagedorn</surname>
-      <email>hagedorn@mcs.anl.gov</email>
-    </author>
-    <affiliation>
-      <orgname>Argonne National Laboratory</orgname>
-      <orgdiv>MCS Division</orgdiv> 
-    </affiliation>
+    <authorgroup>
+      <author>
+	<firstname>Narayan</firstname>
+	<surname>Desai</surname>
+	<email>desai@mcs.anl.gov</email>
+	<affiliation>
+	  <orgname>Argonne National Laboratory</orgname>
+	  <orgdiv>MCS Division</orgdiv> 
+	</affiliation>
+      </author>
+      <author>
+	<firstname>Rick</firstname>
+	<surname>Bradshaw</surname>
+	<email>bradshaw@mcs.anl.gov</email>
+	<affiliation>
+	  <orgname>Argonne National Laboratory</orgname>
+	  <orgdiv>MCS Division</orgdiv> 
+	</affiliation>
+      </author>
+      <author>
+	<firstname>Joey</firstname>
+	<surname>Hagedorn</surname>
+	<email>hagedorn@mcs.anl.gov</email>
+	<affiliation>
+	  <orgname>Argonne National Laboratory</orgname>
+	  <orgdiv>MCS Division</orgdiv> 
+	</affiliation>
+      </author>
+    </authorgroup>
     <date>September 2, 2005</date>
+    <edition>Manual for version 0.8.0</edition>
     <releaseinfo>$Revision$</releaseinfo>
-    <pubdate>September 2005</pubdate>
+    <pubdate>January 2006</pubdate>
     <copyright>
-      <year>2005</year>
+      <year>2005-2006</year>
       <holder>Argonne National Laboratory</holder>
     </copyright> 
     
@@ -56,18 +67,17 @@
 
     <revhistory>
       <revision>
-	<revnumber>0.7.1pre6</revnumber>
+	<revnumber>0.8.0</revnumber>
 	<date>$Date$</date>
 	<revremark>$Id$</revremark>
       </revision>
     </revhistory>
   </bookinfo>
 
-  <toc/>
-
 &architecture;
 &install;
-&generators;
+&specification;
 &deployment;
+&development;
 &reports;
 </book>
\ No newline at end of file
diff --git a/doc/repo-quickstart.txt b/doc/repo-quickstart.txt
deleted file mode 100644
index d52fc08a2..000000000
--- a/doc/repo-quickstart.txt
+++ /dev/null
@@ -1,61 +0,0 @@
-The Bcfg2 repository has a bunch of parts, each with different
-functions.
-
-Cfg/ -> Configuration file repository
-Pkgmgr/ -> Package indices for different images
-SSHbase/ -> SSH Key data files
-etc/ -> Single files for generators or system wide settings
-
-Adding to the repository:
-
-1. If the addition is a single package or configuration file (without
-   any associated service) it can be added to etc/base.xml. This file
-   contains single items which need to be installed on
-   clients. Entries can be nested, depending on where they should be
-   installed. Global entries go at the top level, as do containers
-   corresponding to particular Images. Items which should be on all
-   instances of an image should directly inside of the Image tag, and
-   containers for classes can also be added at this level. 
-
-<Base>
- <ConfigFile name="/etc/passwd" />
- <ConfigFile name="/etc/group" />
- <Image name="debian-sarge">
-   <ConfigFile name="/etc/motd" />
-   <Class name='workstation'>
-     <Package name='gdm'/>
-   </Class>
-  </Image>
-</Base>
-
-    In the above example, /etc/passwd and group belong on all
-    machines, while /etc/motd belongs on all debian-sarge
-    machines. the package gdm belongs only on debian-sarge machines in
-    class workstation. 
-
-2. If a package is associated with a service, then it should be
-   installed as a part of a bundle. This means that the installation
-   mechanism will ensure that all components are installed properly
-   and that services are restarted when any underlying components
-   change. The bundle making process consist of the following steps:
-
-   - Create the bundle file. (look at <REPO>/Bundler/ssh.xml for an
-     example) This file contains a list of all interrelated
-     configuration items. Abstraction based on system (a grouping of
-     images) is possible in this file. Note that explicit
-     configuration data, like file contents or package versions aren't
-     specified in this file.
-
-   - Add all new configuration items into the generator
-     sources. The package list automatically contains all packages
-     available, so it shouldn't need to be changed. Config files must
-     be added to the repository in <REPO>/Cfg. Services must be
-     activated in <REPO>/etc/services.xml. 
-
-   - Activate the bundle for a class in <REPO>/etc/metadata.xml
-
-In all cases, run /usr/sbin/ValidateBcfg2Repo <repo> after changes are
-made. This wil check that the files still parse (ie typos, or quote
-errors)  and check their values against xml schemas, which will ensure
-that all element names are correct and that required attributes are
-included. 
\ No newline at end of file
diff --git a/doc/reports.xml b/doc/reports.xml
index 76f609fb8..e02e49cf4 100644
--- a/doc/reports.xml
+++ b/doc/reports.xml
@@ -1,109 +1,58 @@
 <chapter>
-  <title>The BCFG2 Reporting System</title>
-
-  <section>
-    <title>Reporting Quick Start</title>
-
-  <example>
-  <title><filename>report-configuration.xml</filename></title>
-  <programlisting><![CDATA[<Reports>
-      <Report name='core_stats' good='Y' modified='Y'>
-          <Delivery mechanism='mail' type='nodes-digest'>
-              <Destination address='user@domain.tld'/>
-              <Destination address='user@otherdomain.tld'/>           
-          </Delivery>
-          <Delivery mechanism='www' type='nodes-digest'>
-              <Destination address='/var/www/core_stats.html'/>
-          </Delivery>
-          <Machine name='.*'/>
-      </Report>
-
-      <Report name='stats_for_a_machines' good='N' modified='Y'>
-          <Delivery mechanism='mail' type='nodes-digest'>
-              <Destination address='user@domain.tld'/>
-          </Delivery>
-          <Delivery mechanism='mail' type='overview-stats'>
-              <Destination address='user@otherdomain.tld'/>
-          </Delivery>
-          <Machine name='a.*'/>
-          <Machine name='x-aim'/>
-      </Report>
-    </Reports>]]>
-  </programlisting>
-  </example>
-
-  <para>This configuration will generate two separate reports and deliver
-    them a number of different ways. For more information on exactly what 
-    each section does, please refer to the Configuration section below</para>
-
-  <para>In order to run reports, you need to run the following command
-    regularly via <command>Cron</command>:
-    <command>GenerateHostInfo</command></para>
-  <para>Once configured correctly, just wait for the e-mail or check
-    look at the output html files with a web browser. You can run
-    <command>GenerateHostInfo</command> by hand if you would like
-    in order to try it out immediately. If you are trying to view a
-    HTML file and find that it doesn't work as expected, make sure
-    that the required source directory is in the same directory. They 
-    are static, but might need to be moved from <![CDATA[prefix]]>/share/bcfg2/
-    in to the directory from which you are viewing your reports.</para>
-</section>
-
-  <section>
-    <title>Concepts; Why to report</title>
-
-    <para>Reports play an important role in effectively managing systems
-      with BCFG. There are two primary functions they fulfill; providing 
-      otherwise unobtainable information, and presenting additional
-      helpful information to allow for easier admistration. Reports
-      can contain information including system statistics, discrepancies
-      between specified and actual configuration, invalid configuration,
-      and auditing information among other things.</para>
-
-    <para>The flexible XML configuration file allows reports to be configured
-      to deliver only the information that is important. Additional reports 
-      can easily be created, providing site-specific capability to manage
-      at record effiency. The capability to harvest information regarding
-      statistics, configuration, and problems in a single location should
-      prove to be powerful.</para>
-  </section>
+  <title>BCFG2 Reports</title>
+
+  <para>
+    Reports play an important role in effectively managing systems
+    with BCFG. There are two primary functions they fulfill; providing
+    otherwise unobtainable information, and presenting additional
+    helpful information to allow for easier admistration. Reports
+    can contain information including system statistics, discrepancies
+    between specified and actual configuration, invalid configuration,
+    and auditing information among other things.
+  </para>
+
+  <para>
+    The flexible XML configuration file allows reports to be configured
+    to deliver only the information that is important. Additional reports 
+    can easily be created, providing site-specific capability to manage
+    at record effiency. The capability to harvest information regarding
+    statistics, configuration, and problems in a single location should
+    prove to be powerful.
+  </para>
 
   <section>
     <title>How it all works</title>
 
-    <para>The BCFG2 Reporting System consists of a number of
-      elements. The core is the <command>StatReports</command>
-      Executable. <command>StatReports</command> reads
-      a default configuration file (or a config file specified on the
-      command line) then prepares and delivers the reports specified in
-      the configuration file. It is expected that this executable will be
+    <para>
+      The BCFG2 Reporting System consists of a number of elements. The
+      core is the <command>StatReports</command>
+      Executable. <command>StatReports</command> reads a default
+      configuration file (or a config file specified on the command
+      line) then prepares and delivers the reports specified in the
+      configuration file. It is expected that this executable will be
       run by the adminstrator periodically via <command>cron</command>
-      or similar
-      facilty. The executable can also be run manually on demand for a
-      special sort of report that needs to be generated immediately.
+      or similar facilty. The executable can also be run manually on
+      demand for a special sort of report that needs to be generated
+      immediately.
     </para>
     
     <para>
-      <command>StatReports</command> gets the data it reports from a number of
-      sources. The <filename>hostinfo.xml</filename> 
-      file is specific to the reporting system
-      and generated by the executable python script <command>GenerateHostInfo
-      </command>. 
-      <filename>Hostinfo.xml</filename> contains information about if a
-      host is currently
-      pingable or not, and a mapping of short hostnames to
-      FQDNs. <command>GenerateHostInfo</command> will be run automatically
-      by <command>StatReports</command> if the <filename>hostinfo.xml</filename>
-      file is older than 23.5 hours. This number is
-      chosen, because it is likely an administrator will recieve reports
-      daily about the status of hosts and if they had run BCFG the
-      previous night. It is possible to execute
-      <command>GenerateHostInfo</command> to update the
-      <filename>hostinfo.xml</filename> file at any interval via cron,
-      but it does take
-      some time to complete, so be sure to give it a few minutes. This
-      will only likely be of use if your BCFG clients are set to update
-      more often than nightly and you would like reports after each run.
+      <command>StatReports</command> gets the data it reports from a
+      number of sources. <filename>Metadata/clients.xml</filename>
+      contains information about if a host is currently pingable or
+      not, and a mapping of short hostnames to
+      FQDNs. <command>GenerateHostInfo</command> will be run
+      automatically by <command>StatReports</command> if the
+      <filename>Metadata/clients.xml</filename> file is older than
+      23.5 hours. This number is chosen, because it is likely an
+      administrator will recieve reports daily about the status of
+      hosts and if they had run BCFG the previous night. It is
+      possible to execute <command>GenerateHostInfo</command> to
+      update the <filename>Metadata/clients.xml</filename> file at any
+      interval via cron, but it does take some time to complete, so be
+      sure to give it a few minutes. This will only likely be of use
+      if your BCFG clients are set to update more often than nightly
+      and you would like reports after each run.
     </para>
     
     <para>
@@ -115,11 +64,9 @@
     </para>
     
     <para>
-      Finally <command>StatReports</command> is able to get any information
-      out of the <filename>metadata.xml</filename> file.
-      This will likely be most useful in the future
-      where the range of reports will include auditing style reports,
-      that must include configuration information.
+      Finally <command>StatReports</command> is able to get information
+      out of the <filename>Metadata/groups.xml</filename> file as well.
+      This allows reports to describe the configured profile for each client.
     </para>	
   </section>
 
@@ -134,91 +81,73 @@
       options. For each of those reports, each can be delivered by
       Mail, WWW, or via RSS (or any combination of the three.) In the
       future, additional report types will be added, and if necessary,
-      additional types of deliveries will be created.
+      additional types of deliveries will be created. Tables
+      describing report types and report delivery mechanisms follow.
     </para>
 
-    <para>Here is a list of the report types currently defined.</para>
-
-    <variablelist>
-      <varlistentry>
-	<term>Overview-Stats</term>
-	<listitem>
-	  <para>
-	    This report provides informatoin about a large number of
-	    machines and their states. It is often found to be useful
-	    when the group of machines it is connected with is simply
-	    All Nodes, which gives an overall outlook on your
-	    network's health. It makes sense to get this report via
-	    any mechanism
-	  </para>
-	</listitem>
-      </varlistentry>
-      <varlistentry>
-	<term>Nodes-Digest</term>
-	<listitem>
-	  <para>
-	    This is a report that includes details about each node,
-	    specifically what packages, files, etc are broken, and
-	    other node specific info. It makes sense to recieve this
-	    via any mechanism
-	    </para>
-	</listitem>
-      </varlistentry>
-      <varlistentry>
-	<term>Nodes-Individual</term>
-	<listitem>
-	  <para>
-	    This report includes details about each node, but
-	    information is separated in to separate sections (such as
-	    separate e-mails or RSS articles) before sending. This
-	    works well with e-mail filters and error
-	    detection. Currently WWW is not a supported delivery
-	    mechanism for this type of report, because it is not
-	    completely clear how such a report could be used.
-	  </para>
-	</listitem>
-      </varlistentry>
-    </variablelist>
+    <table>
+      <title>Bcfg2 Report Types</title>
+      <tgroup cols='2'>
+	<colspec colnum='1' colwidth='1*'/>
+	<colspec colnum='2' colwidth='4*'/>
+	<thead>
+	  <row><entry>Report Type</entry><entry>Description</entry></row>
+	</thead>
+	<tbody>
+	  <row><entry>Overview-Stats</entry>
+	    <entry>
+	      <para>
+		This report provides informatoin about a large number of
+		machines and their states. It is often found to be useful
+		when the group of machines it is connected with is simply
+		All Nodes, which gives an overall outlook on your
+		network's health. It makes sense to get this report via
+		any mechanism
+	      </para>
+	    </entry></row>
+	  <row><entry>Nodes-Digest</entry>
+	    <entry>
+	      <para>
+		This report includes details about each node,
+		specifically what packages, files, etc are broken, and
+		other node specific info. It makes sense to recieve
+		this via any mechanism
+	      </para>
+	    </entry></row>
+	  <row><entry>Nodes-Individual</entry>
+	    <entry>
+	      <para>
+		This report includes details about each node, but
+		information is separated in to separate sections (such
+		as separate e-mails or RSS articles) before
+		sending. This works well with e-mail filters and error
+		detection. Currently WWW is not a supported delivery
+		mechanism for this type of report, because it is not
+		completely clear how such a report could be used.
+	      </para>
+	    </entry></row>
+	</tbody>
+      </tgroup>
+    </table>
+
+    <table>
+      <title>Bcfg2 Report Delivery Mechanisms</title>
+      <tgroup cols='2'>
+	<colspec colnum='1' colwidth='1*'/>
+	<colspec colnum='2' colwidth='3*'/>
+	<thead>
+	  <row><entry>Name</entry><entry>Description</entry></row>
+	</thead>
+	<tbody>
+	  <row><entry>www</entry><entry>XHTML file</entry></row>
+	  <row><entry>rss</entry><entry>An RSS file <comment>(links do
+	  not point at real web links, since they may not exist)</comment></entry></row>
+	  <row><entry>mail</entry><entry>A plaintext email
+	  message</entry></row>
+	</tbody>
+      </tgroup>
+    </table>
 
-    <para>A list of delivery mechanisms follows:</para>
-
-    <variablelist>
-      <varlistentry>
-	<term>www</term>
-	<listitem>
-	  <para>
-	    This delivery produces HTML reports which can be delivered
-	    via the WWW. it is important that you copy the directory:
-	    <![CDATA[<prefix>]]>/share/bcfg2/reports/web-rprt-srcs in to the
-	    same directory as you will be serving web pages from. It
-	    includes CSS and JavaScript files necessary to properly
-	    view the WWW files.
-	  </para>
-	</listitem>
-      </varlistentry>
-      <varlistentry>
-	<term>rss</term>
-	<listitem>
-	  <para>
-	    This delivery type also should be written to some area
-	    that is likely web accessable. It is plain RSS-- One
-	    Caveat-- the "Article Link" does not actually point to any
-	    web information, because it is not clear that a
-	    corresponding web page exists for any given RSS article.
-	  </para>
-	</listitem>
-      </varlistentry>
-      <varlistentry>
-	<term>mail</term>
-	<listitem>
-	  <para>
-	    Mail is simple plaintext e-mail that is sent to the
-	    specified recipients directly from the machine on which
-	    BCFG is running by opening a pipe to sendmail.
-	  </para>
-	</listitem>
-      </varlistentry>
-    </variablelist>
   </section>
 
   <section>
@@ -244,7 +173,7 @@
       around a group of machines. <![CDATA[<Machine/>]]> tags may individually
       reference a machine by hostname (not FQDN), or also by a Python Regular
       Expression. More information can be found about such Regexes at:
-      http://docs.python.org/lib/re-syntax.html.</para>
+      <ulink url="http://docs.python.org/lib/re-syntax.html"/>.</para>
 
     <para>Any number of <![CDATA[<Delivery/>]]> elements can be made for a 
       given
@@ -261,4 +190,52 @@
       a complete e-mail address.</para>
 
   </section>
+
+  <section>
+    <title>Reporting Quick Start</title>
+
+    <para>
+      This configuration will generate two separate reports and
+      deliver them a number of different ways. For more information on
+      exactly what each section does, please refer to the
+      Configuration section above.
+    </para>
+
+  <example>
+  <title><filename>etc/report-configuration.xml</filename></title>
+  <programlisting><![CDATA[<Reports>
+      <Report name='core_stats' good='Y' modified='Y'>
+          <Delivery mechanism='mail' type='nodes-digest'>
+              <Destination address='user@domain.tld'/>
+              <Destination address='user@otherdomain.tld'/>           
+          </Delivery>
+          <Delivery mechanism='www' type='nodes-digest'>
+              <Destination address='/var/www/core_stats.html'/>
+          </Delivery>
+          <Machine name='.*'/>
+      </Report>
+
+      <Report name='stats_for_a_machines' good='N' modified='Y'>
+          <Delivery mechanism='mail' type='nodes-digest'>
+              <Destination address='user@domain.tld'/>
+          </Delivery>
+          <Delivery mechanism='mail' type='overview-stats'>
+              <Destination address='user@otherdomain.tld'/>
+          </Delivery>
+          <Machine name='a.*'/>
+          <Machine name='x-aim'/>
+      </Report>
+    </Reports>]]>
+  </programlisting>
+  </example>
+
+  <para>
+      Once configured correctly, just wait for the e-mail or check
+      look at the output html files with a web browser. You can run
+      <command>StatReports</command> by hand if you would like
+      in order to try it out immediately. 
+    </para>
+
+</section>
+
 </chapter>
\ No newline at end of file
diff --git a/doc/specs.xml b/doc/specs.xml
new file mode 100644
index 000000000..9b51a3309
--- /dev/null
+++ b/doc/specs.xml
@@ -0,0 +1,501 @@
+<chapter>
+  <title>Writing Bcfg2 Specifications</title>
+  
+  <para>
+    The Bcfg2 specification is a set of directives that describe how
+    hosts should be configured. This information is used to generate
+    client configurations. This section describes the steps taken in 
+    during the composition of bcfg2 specifications. 
+  </para>
+
+  <orderedlist>
+    <listitem>
+      <para>
+	All parts of the specification will correspond to some
+	subset of the clients that bcfg2 has record of. Find or create a
+	group corresponding to the target of this specification.
+      </para>
+    </listitem>
+    <listitem>
+      <para>
+	Add each new configuration entry to the "abstract
+	configuration" for the target group. This can be done in one
+	of two ways. If the new configuration has to do with a
+	service, or some other piece of inter-dependent configuration,
+	write a bundle. If not, add the extra configuration entries to
+	the base. 
+      </para>
+    </listitem>
+    <listitem>
+      <para>
+	Add configuration data for the above entries that apply only
+	to the target group.
+      </para>
+    </listitem>
+    <listitem>
+      <para>
+	Verify that clients not in the target group remain unaffected
+	by these changes. 
+      </para>
+    </listitem>
+    <listitem>
+      <para>
+	Re-validate the bcfg2 repository by running 
+	<command>bcfg2-repo-validate</command>.
+      </para>
+    </listitem>
+  </orderedlist>
+
+  <section>
+    <title>Interacting with Client Groups in Bcfg2</title>
+
+    <para>
+      Bcfg2 uses an aspect based classing mechanism to describe
+      configuration patterns in its specifications. The Bcfg2 metadata
+      mechanism has two types of information, client metadata and
+      group metadata. Client metadata describes what top level group a
+      client is associated. Its configuration is derived from a
+      combination of its host information and this group. Group
+      definitions describe groups in terms of what bundles they
+      include and other groups they include. Groups have a set of
+      properties that describe how they can be used. 
+    </para>
+
+    <table>
+      <title>Bcfg2 Group Parameters</title>
+      <tgroup cols='3'>
+	<colspec colnum='1' colwidth='1*'/>
+	<colspec colnum='2' colwidth='7*'/>
+	<colspec colnum='3' colwidth='2*'/>
+	<thead>
+	  <row><entry>Name</entry><entry>Description</entry>
+	    <entry>Values</entry></row>
+	</thead>
+	<tbody>
+	  <row><entry>profile</entry><entry>If a client can be
+	  directly associated with this group</entry><entry>(True|False*)</entry></row>
+	  <row><entry>public</entry><entry>If a client can freely
+	  associate itself with this group</entry><entry>(True|False*)</entry></row>
+	  <row><entry>toolset</entry>
+	    <entry>Describes which client-side logic should be used to
+	    make configuration
+	    changes</entry><entry>(rh|debian|solaris)</entry></row>
+	  <row><entry>category</entry>
+	    <entry>A group can only contain one instance of a group of
+	    any category. This provides the basis for representing
+	    groups which are conjugates of one another in a rigorous
+	    way. It also provides the basis for negation.</entry><entry>string</entry></row>
+	</tbody>
+      </tgroup>
+    </table>
+  </section>
+
+  <para>
+    When a client's configuration is generated, its metadata is the
+    fetched. This includes a list of all groups recursively
+    dereferenced, and all bundles included by those groups. This
+    collection has already been processed using the group category
+    rules, so only one instance from each group category is
+    included. This metadata is used throughout the rest of the
+    configuration generation process; it defines the client's abstract
+    configuration and specifies all literal contents of all
+    configuration entities. 
+  </para>
+
+  <section>
+    <title>Adding to the Abstract Configuration</title>
+
+    <para>
+      When writing bcfg2 specification, administrators primarily
+      perform one of two operations: addition of new configuration
+      entities or the modification of existing entries. If new
+      entities need to be added, then they must be added to the
+      abstract configuration. This is the inventory of configuration
+      entities that should be installed on a client. Two plugins
+      provide the basis for the abstract configuration, the bundler
+      and base. The bundler builds descriptions of interrelated
+      configuration entities. These are typically used for the
+      representation of services, or other complex groups of
+      entities. Base provides a laundry list of configuration entities
+      that need to be installed on hosts. These entities are
+      independent from one another, and can be installed individually
+      without worrying about the impact on other entities.
+    </para>
+
+    <para>
+      Entities in the abstract configuration (and correspondingly in
+      the literal configuration) can have one of several types. In the
+      abstract configuration, each of these entities only has a tag
+      and the name attribute set. 
+    </para>
+
+    <table>
+      <title>Bcfg2 Configuration Entity Types</title>
+      <tgroup cols='2'>
+	<colspec colnum='1'/>
+	<colspec colnum='2'/>
+	<thead>
+	  <row><entry>Name</entry><entry>Description</entry></row>
+	</thead>
+	<tbody>
+	  <row><entry>Package</entry><entry>Software Package</entry></row>
+	  <row><entry>ConfigFile</entry><entry>Configuration File</entry></row>
+	  <row><entry>Service</entry>
+	    <entry>Persistent system services and daemons</entry></row>
+	  <row><entry>Directory</entry><entry>Filesystem Directories</entry></row>
+	  <row><entry>SymLink</entry><entry>Symbolic links</entry></row>
+	  <row><entry>Permissions</entry>
+	    <entry>The permissions (not contents) of a POSIX path</entry></row>
+	</tbody>
+      </tgroup>
+    </table>
+
+    <section>
+      <title>Writing Bundles</title>
+    
+      <para>
+	Bundles consist of a set of configuration entities. These
+	entities are grouped together due to a configuration-time
+	interdependency. Basic services tend to be the simplest
+	example of these: they consist of some software package(s)
+	some configuration files and an indication that some service
+	should be activated. If any of these pieces are installed or
+	updated, all should be re-checked and any associated services
+	should be restarted. 
+      </para>
+
+      <para>
+	Bundles can also contain conditonal entries that are only used
+	for hosts in some particular groups. This is useful when a
+	service name varies from platform to platform. A group is
+	defined for each platform, hence a different service can be
+	associated with the bundle for clients in the different
+	groups. Conditional additions can also add extra functionality
+	to services as needed. This has proven useful in the case of
+	configuring webservers; php and ssl can be configured as extra
+	features that some webservers have and others do not. At the
+	same time, all configuration-time interdependencies are
+	maintained. 
+      </para>
+    </section>
+
+    <section>
+      <title>Using Base</title>
+
+      <para>
+	The Base plugin provides a mechanism to add independent
+	configuration entities to a client's abstract
+	configuration. All files in the Base/ subdirectory of the
+	respository are processed, and all entries that fall within
+	the scope of the client metadata are included in its abstract
+	configuration.  These files are similar to those used by the
+	Bundler, Svcmgr, and Pkgmgr, without the need for
+	prioritization used by the later two.
+      </para>
+      </section>
+  </section>
+
+  <section>
+    <title>Adding to the Literal Configuration</title>
+    
+    <para>
+      During the construction of the literal configuration, first the
+      abstract configuration is built, and then explicit data is bound
+      in to each entity. The previous section describes how the first
+      stage works, and this section describes the second stage. Each
+      entity will be served by one plugin. This plugin will decide
+      what explicit data should be bound in to a particular entity for
+      a given client. Each of these plugins has a specific area of the
+      configuration repository, corresponding to its name. This
+      section describes how several of the basic plugins works.
+    </para>
+
+    <section>
+      <title>Cfg</title>
+      <para>
+	The Cfg plugin provides a configuration file repository that
+	uses literal file contents to provide client-tailored
+	configuration file entries. It chooses which data to provide
+	for a given client based on the aspect-based metadata system
+	used for high-level client configuration.
+      </para>
+      <para>
+	The Cfg repository is structured much like the filesystem
+	hierarchy being configured. Each configuration file being
+	served has a corresponding directory in the configuration
+	repository. These directories have the same relative path as
+	the absolute path of the configuration file on the target
+	system. For example, if Cfg was serving data for the
+	configuration file <filename>/etc/services</filename>, then
+	its directory would be in the relative path
+	<filename>./etc/services</filename> inside of the Cfg
+	repository. 
+      </para>
+      <para>
+	Inside of this file-specific directory, three types of files
+	may exist. Base files are complete instances of configuration
+	file. Deltas are differences between a base file and the
+	target file contents. Base files and deltas are tagged with
+	metadata specifiers, which describe which groups of clients
+	the fragment pertains to. Configuration files are constructed
+	by finding the most specific base file and applying any more
+	specific deltas.
+      </para>
+      <para>
+	Specifiers are embedded in fragment filenames. For example, in
+	the fragment <filename>services.G99_webserver</filename>,
+	"G99_webserver" is the specifier. This specifier applies to
+	the group (G) webserver with a priority of 99. Files can also
+	be tagged with a host-specific (H) specifier.Global files are
+	the least specific. Priorities are used as to break ties.
+      </para>
+      <para>
+	Info files, named <filename>:info</filename> are used to
+	specify target configuration file metadata, such as owner,
+	group and permissions. If no <filename>:info</filename> is
+	provided, targets are installed with default
+	information. Default metadata is root ownership, root group
+	memberships, and 0644 file permissions.
+      </para>
+      <example>
+	<title>Cfg/filepath/:info</title>
+	<programlisting>	  owner:root
+	  group:root
+	  perms:0755</programlisting>
+      </example>
+
+      <example>
+	<title>Cfg/etc/passwd/</title>
+	<programlisting>	  $ ls
+	  :info            passwd	  passwd.G99_chiba-login
+	  passwd.H_bio-debian  passwd.H_cvstest   passwd.H_foxtrot  
+	  passwd.H_reboot  passwd.H_rudy2    passwd.G98_netserv
+	  passwd.G99_tacacs-server.cat  passwd.H_adenine</programlisting>
+      </example>
+      
+      <para>
+	In the previous example, there exists files with each of the
+	characteristics mentioned above. All files ending in ".cat"
+	are deltas; ones with ".H_" are host specific files. There
+	exists a base file, a <filename>:info</filename> file, two
+	class-specified base files, and a bundle-specified base file.
+      </para>
+    </section>
+
+    <section>
+      <title>Pkgmgr</title>
+
+      <para>
+	The Pkgmgr plugin is responsible for providing package version
+	and installation information. In the case of each "Package"
+	entity in the configuration, it binds in information needed to
+	detect, verify and install the package. It has a similar
+	format to the files used by Base and Bundler, but with a few
+	differences. First, each file has a priority. This allows the
+	same entity to be served by multiple files. The priorities can
+	be used to disambiguate in the case that multiple files serve
+	data for the same package. The other difference is that
+	automatic deriviation of package information from the file
+	attribute. The Pkgmgr has a set of regexes that can split
+	package names for several formats. The filenames are used to
+	construct installation URLs, and set several important fields
+	like package name and version. 
+      </para>
+    </section>
+    <section>
+      <title>Svcmgr</title>
+
+      <para>
+	The Svcmgr plugin describes where services should be active
+	and inactive. Its files have a similar form to those used by
+	the Pkgmgr. Several files in the Svcmgr repository can contain
+	overlapping definitions, and a per-file priority is used to
+	determine precedence, the highest priority file serving data
+	for a particular service wins, on a service by service basis.
+      </para>
+
+      <para>
+	These files also have a similar set of semantics to those used
+	by the Pkgmgr. Entries in the top level element (Services) are
+	global definitions. Group elements describe additional
+	conditions that must be matched for that definition to
+	supercede less specific ones. Deeply nested definitions must
+	have their parent condition matched, plus all parent
+	conditions as well. For example, the following declaration
+	turns ssh on by default, disables it if the client is a part
+	of group a, and reenables it if the client is a part of both
+	groups a and b.
+      </para>
+
+            <example>
+	<title>Svcmgr/ssh.xml</title>
+	<programlisting><![CDATA[<Services priority='0'>
+  <Service name='ssh' status='on'/>
+  <Group name='a'>
+    <Service name='ssh' status='off'/>
+    <Group name='b'>
+      <Service name='ssh' status='on'/>
+    </Group>
+  </Group>
+</Services>]]></programlisting>
+      </example>
+
+      <para>
+	The files used by this plugin can be structured in a number of
+	ways. The most common method is to use one large file, but
+	this can be inconvenience due to the large size of the
+	file. The data can also be split up using any convenient
+	mechanism: per-service, per-administrator, etc.
+      </para>
+    </section>
+    <section>
+      <title>SSHbase</title>
+
+      <para>
+	The SSHbase plugin implements ssh public and private key
+	management functionality. This means that a central record of
+	ssh host keys is maintained. Also, a correct ssh_known_hosts
+	file is maintained. This means that the keys for new hosts are
+	added to this configuration, and also that a correct line for
+	localhost is created. SSHbase will generate a new key for any
+	hosts that doesn't already have a key stored in the
+	repository, so it should be pre-seeded with the keys (public
+	and private) of pre-existing clients.
+      </para>
+    </section>
+  </section>
+
+  <section>
+    <title>Checking Group-External Clients for Unintended
+    Changes</title>
+
+    <para>
+      Any configuration change will apply to some set of clients.
+      Often, repository changes can have unintended consequences to
+      clients not included in the target group. To address this issue,
+      consider the changes performed, and if they can affect clients
+      in unexpected ways.
+    </para>
+
+  </section>
+
+  <section>
+    <title>Validating the Bcfg2 Repository</title>
+    
+    <para>
+      Bcfg2 includes a repository validation tool that will check all
+      XML files in the repository against included XML schemas. It is
+      critical to run this command,
+      <command>bcfg2-repo-valdate</command> after any modifications to
+      XML files. If all files validate properly, then no output will
+      be returned. It takes a "-v" option that prints out  a line for
+      each file that is validated. This can be used to ensure that all
+      files are checked. 
+    </para>
+  </section>
+
+  <section>
+    <title>Annotated Configuration Examples</title>
+
+    <para>
+      In addition to the description of the abstract process
+      above, we present several examples of the thought process and
+      actions taken to achieve a particular configuration goal. These
+      will start simple, but become more complex.
+    </para>
+
+    <section>
+      <title>Configuring /etc/motd on all hosts</title>
+
+      <para>The goal for this example is to install a uniform copy of
+      a specified <filename>/etc/motd</filename> on all hosts.</para>
+
+      <orderedlist>
+	<listitem><para>
+	    In this case, the target group is global, since we want
+	    this version of <filename>/etc/motd</filename>. As
+	    mentioned earlier, the global group is handled specially,
+	    so that all new clients, even newly created ones, are in it.
+	  </para></listitem>
+	<listitem><para>
+	    Since <filename>/etc/motd</filename> is not interdependent
+	    with any other configuration entities, it can be installed
+	    using Base instead of using Bundler. The ConfigFile
+	    entity should be placed in the globally scoped section of
+	    a base file. This adds the configuration file to the
+	    abstract configuration.
+	  </para></listitem>
+	<listitem><para>
+	    A file with the correct contents for
+	    <filename>/etc/motd</filename> should be installed in the
+	    Cfg repository area as a global file. This will provide
+	    the right literal configuration specification for each
+	    client.
+	  </para></listitem>
+	<listitem><para>
+	    Since this change is globally scoped, there are not any
+	    clients that shoud not be affected.
+	</para></listitem>
+	<listitem><para>
+	    Finally, <command>bcfg2-repo-validate</command> should be
+	    run to catch typos. 
+	  </para></listitem>
+      </orderedlist>
+    </section>
+
+    <section>
+      <title>Configuring NTP for a network</title>
+      
+      <para>
+	The goal for this example is to configure NTP for an entire
+	network. This means several things. All clients should run NTP
+	as clients. One client should run NTP as a server, and
+	ntp clients should use it instead of an external server.
+      </para>
+
+      <orderedlist>
+	<listitem><para>
+	    Two discrete groups are used for the different parts of
+	    the desired configuration state. The first is the global
+	    group, handling the "all clients run ntp" part of the
+	    configuration specification. The other part corresponds to
+	    a new group "ntp-server", which contains only clients that
+	    should function as an ntp server.
+	  </para></listitem>
+	<listitem><para>
+	    This configuration specification describes configuration
+	    entities that have interdependencies, so a bundle should
+	    be used. This bundle should contain the ntp software, the
+	    ntp configuration file, and the ntpd service.
+	  </para></listitem>
+	<listitem><para>
+	    The literal portions of three different configuration
+	    entities need to be represented. First, the Pkgmgr needs
+	    to be configured to bind a package entity named ntp. Next,
+	    the Svcmgr needs to have a global declaration that the
+	    service ntpd should be on. Finally, two different
+	    configuration files should be added to Cfg. The global
+	    version of <filename>/etc/ntp.conf</filename> should have
+	    an ntp configuration that points hosts at the local ntp
+	    server. The group "ntp-server" specific version of this
+	    file should have the proper configuration for local ntp
+	    servers. 
+	  </para></listitem>
+	<listitem><para>
+	    A quick inspection of these configuration changes show
+	    only minor possibilities for bad interactions. All
+	    machines should run ntp, and the only scope where bad
+	    interactions can occur is on ntp servers. 
+	    </para></listitem>
+	<listitem><para>
+	    Finally, run <command>bcfg2-repo-validate</command>. It
+	    will validate the new bundle that has been added, and
+	    changes to the Svcmgr and Pkgmgr indices. 
+	  </para></listitem>
+      </orderedlist>
+    </section>
+
+    <!-- <para>Need more examples here</para> -->
+
+  </section>
+</chapter>
\ No newline at end of file