doc: Add server plugin docs

Signed-off-by: Sol Jerome <solj@ices.utexas.edu>


git-svn-id: https://svn.mcs.anl.gov/repos/bcfg/trunk/bcfg2@5635 ce84e21b-d406-0410-9b95-82705330c041

28 files changed, 344 insertions, 108 deletions

diff --git a/doc/index.txt b/doc/index.txt
index f50cac4bf..2761c3270 100644
--- a/doc/index.txt
+++ b/doc/index.txt
@@ -10,6 +10,7 @@ Welcome to Bcfg2's documentation!
    quickstart/index
    authentication
    getting_started/index
+   plugins/index
    testimonials
    sites
 
diff --git a/doc/plugin-roles b/doc/plugin-roles
deleted file mode 100644
index c8c47aa93..000000000
--- a/doc/plugin-roles
+++ /dev/null
@@ -1,90 +0,0 @@
-This documents available plugin roles. 
-
-1) list of plugin roles
-
-| Role          | Class              | Status |
-|---------------+--------------------+--------|
-| Metadata      | Metadata           | done   |
-| Connector     | Connector          | done   |
-| Probing       | Probing            | done   |
-| Structure     | Structure          | done   |
-| Structure Val | StructureValidator | done   |
-| Generator     | Generator          | done   |
-| Goals Val     | GoalValidator      | done   |
-| Statistics    | Statistics         | done   |
-| Pull Source   | PullSource         | done   |
-| Pull Target   | PullTarget         | done   |
-| Version       | Version            | started|
-| Decision      | Decision           | done   |
-| Remote        | Remote             | none   |
-| Syncing       | Syncing            | none   |
-|---------------+--------------------+--------|
-
-2) Plugin Capabilities
-
-* Metadata
-** Initial metadata construction
-** Connector data accumulation
-** ClientMetadata instance delivery
-** Introspection interface (for bcfg2-info & co)
-* Connector
-** Provide additional data for ClientMetadata instances 
-* Probing
-** send executable probes to clients and receive data responses
-* Structure
-** Produce a list of configuration entries that should be included in
-   client configurations
-** Each structure plugin is produces a list of structures
-** Core verifies that each bundle listed has been constructed
-* Structure Validation
-** Validate a client entry list's internal consistency, modifying if needed
-* Generator
-* Goals Validation
-** Validate client goals, modifying if needed
-* Pull Source
-** Plugin can provide entry information about clients
-* Pull Target
-** Plugin can accept entry data and merge it into the specification
-* Version
-** Plugin can read revision information from VCS of choice
-** Will provide an interface for producing commits made by the
-   bcfg2-server
-* Decision
-
-3) Configuration of plugins
-
-Plugin configuration will be simplified substantially. Now, a single
-list of plugins (including plugins of all capabilities) is specified
-upon startup (either via bcfg2.conf or equivalent). This mechanism
-replaces the current split configuration mechanism where generators,
-structures, and other plugins are listed independently. Instead, all
-plugins included in the startup list will be initialized, and each
-will be enabled in all roles that it supports. This will remove a
-current source of confusion and potential configuration errors,
-wherein a plugin is enabled for an improper set of goals. (ie Cfg
-enabled as a structure, etc) This does remove the possibility of
-partially enabling a plugin for one of its roles without activating it
-across the board, but I think this is a corner case, which will be
-poorly supported by plugin implementers. If needed, this use case can
-be explicitly supported by the plugin author, through use of a config
-file directive. 
-
-4) User Visible Changes
-
-Connector data is added to ClientMetadata instances using the name of
-the connector plugin. This means that the dictionary of key/val probe
-pairs included with metadata is now available as metadata.Probes
-(instead of metadata.probes). Once properties are available the same
-way, they will likewise change names to metadata.Properties from their
-current name. 
-
-Plugin configuration will change. A single field "plugins" in
-bcfg2.conf will supercede the combination of the "generators" and 
-"structures" fields. 
-
-Default loading of needed plugins is now explicit; this means that
-Statistics (if used) should be listed in the plugins line of
-bcfg2.conf. 
-
-5) Notes
-* Need to ensure bundle accumulation occurs with connector groups
diff --git a/doc/plugins/account.txt b/doc/plugins/account.txt
new file mode 100644
index 000000000..311d91b24
--- /dev/null
+++ b/doc/plugins/account.txt
@@ -0,0 +1,5 @@
+.. -*- mode: rst -*-
+
+=======
+Account
+=======
diff --git a/doc/plugins/base.txt b/doc/plugins/base.txt
new file mode 100644
index 000000000..ead775c10
--- /dev/null
+++ b/doc/plugins/base.txt
@@ -0,0 +1,5 @@
+.. -*- mode: rst -*-
+
+====
+Base
+====
diff --git a/doc/plugins/bb.txt b/doc/plugins/bb.txt
new file mode 100644
index 000000000..806f180e8
--- /dev/null
+++ b/doc/plugins/bb.txt
@@ -0,0 +1,5 @@
+.. -*- mode: rst -*-
+
+==
+BB
+==
diff --git a/doc/plugins/bundler.txt b/doc/plugins/bundler.txt
new file mode 100644
index 000000000..1efbca878
--- /dev/null
+++ b/doc/plugins/bundler.txt
@@ -0,0 +1,5 @@
+.. -*- mode: rst -*-
+
+=======
+Bundler
+=======
diff --git a/doc/plugins/bzr.txt b/doc/plugins/bzr.txt
new file mode 100644
index 000000000..83bea745e
--- /dev/null
+++ b/doc/plugins/bzr.txt
@@ -0,0 +1,5 @@
+.. -*- mode: rst -*-
+
+===
+Bzr
+===
diff --git a/doc/plugins/cfg.txt b/doc/plugins/cfg.txt
new file mode 100644
index 000000000..5701f2327
--- /dev/null
+++ b/doc/plugins/cfg.txt
@@ -0,0 +1,5 @@
+.. -*- mode: rst -*-
+
+===
+Cfg
+===
diff --git a/doc/plugins/dbstats.txt b/doc/plugins/dbstats.txt
new file mode 100644
index 000000000..4afca4afe
--- /dev/null
+++ b/doc/plugins/dbstats.txt
@@ -0,0 +1,5 @@
+.. -*- mode: rst -*-
+
+=======
+DBStats
+=======
diff --git a/doc/plugins/decisions.txt b/doc/plugins/decisions.txt
new file mode 100644
index 000000000..3e32bd803
--- /dev/null
+++ b/doc/plugins/decisions.txt
@@ -0,0 +1,5 @@
+.. -*- mode: rst -*-
+
+=========
+Decisions
+=========
diff --git a/doc/plugins/deps.txt b/doc/plugins/deps.txt
new file mode 100644
index 000000000..86b3a272e
--- /dev/null
+++ b/doc/plugins/deps.txt
@@ -0,0 +1,5 @@
+.. -*- mode: rst -*-
+
+====
+Deps
+====
diff --git a/doc/plugins/fossil.txt b/doc/plugins/fossil.txt
new file mode 100644
index 000000000..893ab266e
--- /dev/null
+++ b/doc/plugins/fossil.txt
@@ -0,0 +1,5 @@
+.. -*- mode: rst -*-
+
+======
+Fossil
+======
diff --git a/doc/plugins/git.txt b/doc/plugins/git.txt
new file mode 100644
index 000000000..621c64127
--- /dev/null
+++ b/doc/plugins/git.txt
@@ -0,0 +1,5 @@
+.. -*- mode: rst -*-
+
+===
+Git
+===
diff --git a/doc/plugins/hostbase.txt b/doc/plugins/hostbase.txt
new file mode 100644
index 000000000..38580598a
--- /dev/null
+++ b/doc/plugins/hostbase.txt
@@ -0,0 +1,5 @@
+.. -*- mode: rst -*-
+
+========
+Hostbase
+========
diff --git a/doc/plugins/index.txt b/doc/plugins/index.txt
new file mode 100644
index 000000000..0d46fc102
--- /dev/null
+++ b/doc/plugins/index.txt
@@ -0,0 +1,83 @@
+.. -*- mode: rst -*-
+
+=======
+Plugins
+=======
+
+Plugins are the source of all logic used in building a config. They can perform one of several tasks:
+
+#. Generating configuration inventory lists for clients
+#. Generating configuration entry contents for clients
+#. Probing client-side state (like hardware inventory, etc) -- the generic client probing mechanism is described at :doc:`probes`.
+#. Automating administrative tasks (e.g. :doc:`sshbase` which automates ssh key management)
+#. Generating client per-entry installation decision-lists
+
+Enabling Plugins
+================
+
+In order for the bcfg2 server to use a plugin, it needs to be listed on the *plugins* line in bcfg2.conf.
+
+Default Plugins
+===============
+
+The `Bcfg2 repository`_ has the default plugin list currently distributed with Bcfg2: http://trac.mcs.anl.gov/projects/bcfg2/browser/trunk/bcfg2/src/lib/Server/Plugins.
+
+.. _Bcfg2 repository: http://trac.mcs.anl.gov/projects/bcfg2/browser/trunk/bcfg2/src/lib/Server/Plugins.
+
+Metadata (Grouping)
+-------------------
+
+* :doc:`bb`
+* :doc:`metadata`
+
+Each of these plugins has a corresponding subdirectory with the same name in the Bcfg2 repository.
+
+Abstract Configuration (Structures)
+-----------------------------------
+
+* :doc:`base`
+* :doc:`bundler`
+
+Each of these plugins has a corresponding subdirectory with the same name in the Bcfg2 repository.
+
+Literal Configuration (Generators)
+----------------------------------
+
+* :doc:`account`
+* :doc:`actions`
+* :doc:`cfg`
+* :doc:`decisions`
+* :doc:`deps`
+* :doc:`hostbase`
+* :doc:`nagiosgen`
+* :doc:`packages`
+* :doc:`pkgmgr`
+* :doc:`rules`
+* :doc:`sshbase`
+* :doc:`tcheetah`
+* :doc:`tgenshi`
+
+Each of these plugins has a corresponding subdirectory with the same name in the Bcfg2 repository.
+
+Statistics Plugins
+------------------
+
+* :doc:`dbstats`
+* :doc:`statistics`
+
+DBStats can be enabled by adding it to the plugins line in /etc/bcfg2.conf.
+
+Version Plugins
+---------------
+
+* :doc:`bzr`
+* :doc:`fossil`
+* :doc:`git`
+* :doc:`svn`
+
+Plugin Roles (in 1.0)
+=====================
+
+In version 1.0, plugins have been refactored into a series of roles. This are fine-grained plugin capabilities that govern how the server core interacts with plugins.
+
+More details can be found in :doc:`plugin-roles`
diff --git a/doc/plugins/metadata.txt b/doc/plugins/metadata.txt
new file mode 100644
index 000000000..2c169318f
--- /dev/null
+++ b/doc/plugins/metadata.txt
@@ -0,0 +1,5 @@
+.. -*- mode: rst -*-
+
+========
+Metadata
+========
diff --git a/doc/plugins/nagiosgen.txt b/doc/plugins/nagiosgen.txt
new file mode 100644
index 000000000..f157cd82b
--- /dev/null
+++ b/doc/plugins/nagiosgen.txt
@@ -0,0 +1,5 @@
+.. -*- mode: rst -*-
+
+=========
+NagiosGen
+=========
diff --git a/doc/plugins/packages.txt b/doc/plugins/packages.txt
new file mode 100644
index 000000000..b2dbfd1ab
--- /dev/null
+++ b/doc/plugins/packages.txt
@@ -0,0 +1,5 @@
+.. -*- mode: rst -*-
+
+========
+Packages
+========
diff --git a/doc/plugins/pkgmgr.txt b/doc/plugins/pkgmgr.txt
new file mode 100644
index 000000000..63cb7de1c
--- /dev/null
+++ b/doc/plugins/pkgmgr.txt
@@ -0,0 +1,5 @@
+.. -*- mode: rst -*-
+
+======
+Pkgmgr
+======
diff --git a/doc/plugins/plugin-roles.txt b/doc/plugins/plugin-roles.txt
new file mode 100644
index 000000000..f26c79cee
--- /dev/null
+++ b/doc/plugins/plugin-roles.txt
@@ -0,0 +1,127 @@
+.. -*- mode: rst -*-
+
+============
+Plugin Roles
+============
+
+This documents available plugin roles.
+
+1. list of plugin roles
+
+   +---------------+--------------------+--------+
+   | Role          | Class              | Status |
+   +===============+====================+========+
+   | Metadata      | Metadata           | done   |
+   +---------------+--------------------+--------+
+   | Connector     | Connector          | done   |
+   +---------------+--------------------+--------+
+   | Probing       | Probing            | done   |
+   +---------------+--------------------+--------+
+   | Structure     | Structure          | done   |
+   +---------------+--------------------+--------+
+   | Structure Val | StructureValidator | done   |
+   +---------------+--------------------+--------+
+   | Generator     | Generator          | done   |
+   +---------------+--------------------+--------+
+   | Goals Val     | GoalValidator      | done   |
+   +---------------+--------------------+--------+
+   | Statistics    | Statistics         | done   |
+   +---------------+--------------------+--------+
+   | Pull Source   | PullSource         | done   |
+   +---------------+--------------------+--------+
+   | Pull Target   | PullTarget         | done   |
+   +---------------+--------------------+--------+
+   | Version       | Version            | done   |
+   +---------------+--------------------+--------+
+   | Decision      | Decision           | done   |
+   +---------------+--------------------+--------+
+   | Remote        | Remote             | none   |
+   +---------------+--------------------+--------+
+   | Syncing       | Syncing            | none   |
+   +---------------+--------------------+--------+
+
+2. Plugin Capabilities
+
+   * Metadata
+
+     * Initial metadata construction
+     * Connector data accumulation
+     * ClientMetadata instance delivery
+     * Introspection interface (for bcfg2-info & co)
+
+   * Connector
+
+     * Provide additional data for ClientMetadata instances
+
+   * Probing
+
+     * send executable probes to clients and receive data responses
+
+   * Structure
+
+     * Produce a list of configuration entries that should be included in client configurations
+     * Each structure plugin is produces a list of structures
+     * Core verifies that each bundle listed has been constructed
+
+   * Structure Validation
+
+     * Validate a client entry list's internal consistency, modifying if needed
+
+   * Generator
+   * Goals Validation
+
+     * Validate client goals, modifying if needed
+
+   * Pull Source
+
+     * Plugin can provide entry information about clients
+
+   * Pull Target
+
+     * Plugin can accept entry data and merge it into the specification
+
+   * Version
+
+     * Plugin can read revision information from VCS of choice
+     * Will provide an interface for producing commits made by the bcfg2-server
+
+   * Decision
+
+3. Configuration of plugins
+
+   Plugin configuration will be simplified substantially. Now, a single
+   list of plugins (including plugins of all capabilities) is specified
+   upon startup (either via bcfg2.conf or equivalent). This mechanism
+   replaces the current split configuration mechanism where generators,
+   structures, and other plugins are listed independently. Instead, all
+   plugins included in the startup list will be initialized, and each
+   will be enabled in all roles that it supports. This will remove a
+   current source of confusion and potential configuration errors,
+   wherein a plugin is enabled for an improper set of goals. (ie Cfg
+   enabled as a structure, etc) This does remove the possibility of
+   partially enabling a plugin for one of its roles without activating it
+   across the board, but I think this is a corner case, which will be
+   poorly supported by plugin implementers. If needed, this use case can
+   be explicitly supported by the plugin author, through use of a config
+   file directive.
+
+4. User Visible Changes
+
+   Connector data is added to ClientMetadata instances using the name of
+   the connector plugin. This means that the dictionary of key/val probe
+   pairs included with metadata is now available as metadata.Probes
+   (instead of metadata.probes). Once properties are available the same
+   way, they will likewise change names to metadata.Properties from their
+   current name.
+
+   Plugin configuration will change. A single field "plugins" in
+   bcfg2.conf will supercede the combination of the "generators" and
+   "structures" fields.
+
+   Default loading of needed plugins is now explicit; this means that
+   Statistics (if used) should be listed in the plugins line of
+   bcfg2.conf.
+
+5. Notes
+
+   * Need to ensure bundle accumulation occurs with connector groups
diff --git a/doc/plugins/probes.txt b/doc/plugins/probes.txt
new file mode 100644
index 000000000..62dab61e7
--- /dev/null
+++ b/doc/plugins/probes.txt
@@ -0,0 +1,5 @@
+.. -*- mode: rst -*-
+
+======
+Probes
+======
diff --git a/doc/plugins/rules.txt b/doc/plugins/rules.txt
new file mode 100644
index 000000000..8d32df856
--- /dev/null
+++ b/doc/plugins/rules.txt
@@ -0,0 +1,5 @@
+.. -*- mode: rst -*-
+
+=====
+Rules
+=====
diff --git a/doc/plugins/sshbase.txt b/doc/plugins/sshbase.txt
new file mode 100644
index 000000000..28b541732
--- /dev/null
+++ b/doc/plugins/sshbase.txt
@@ -0,0 +1,5 @@
+.. -*- mode: rst -*-
+
+=======
+SSHbase
+=======
diff --git a/doc/plugins/statistics.txt b/doc/plugins/statistics.txt
new file mode 100644
index 000000000..4aae9c0f2
--- /dev/null
+++ b/doc/plugins/statistics.txt
@@ -0,0 +1,5 @@
+.. -*- mode: rst -*-
+
+==========
+Statistics
+==========
diff --git a/doc/plugins/svn.txt b/doc/plugins/svn.txt
new file mode 100644
index 000000000..cfe44c707
--- /dev/null
+++ b/doc/plugins/svn.txt
@@ -0,0 +1,5 @@
+.. -*- mode: rst -*-
+
+===
+Svn
+===
diff --git a/doc/plugins/tcheetah.txt b/doc/plugins/tcheetah.txt
new file mode 100644
index 000000000..1b0b4907d
--- /dev/null
+++ b/doc/plugins/tcheetah.txt
@@ -0,0 +1,5 @@
+.. -*- mode: rst -*-
+
+========
+TCheetah
+========
diff --git a/doc/plugins/tgenshi.txt b/doc/plugins/tgenshi.txt
new file mode 100644
index 000000000..f8acb9933
--- /dev/null
+++ b/doc/plugins/tgenshi.txt
@@ -0,0 +1,5 @@
+.. -*- mode: rst -*-
+
+=======
+TGenshi
+=======
diff --git a/doc/quickstart/ubuntu.txt b/doc/quickstart/ubuntu.txt
index 7ca873c48..933d46401 100644
--- a/doc/quickstart/ubuntu.txt
+++ b/doc/quickstart/ubuntu.txt
@@ -78,7 +78,7 @@ Run bcfg2 to be sure you are able to communicate with the server ::
 
     root@lucid:~# bcfg2 -vqn
     Loaded tool drivers:
-     APT      Action   DebInit  POSIX   
+     APT      Action   DebInit  POSIX
 
     Phase: initial
     Correct entries:        0
@@ -190,7 +190,7 @@ The last thing we need is for the client to have the proper arch group membershi
     root@lucid:~# grep plugins /etc/bcfg2.conf
     plugins = Base,Bundler,Cfg,Metadata,Packages,Probes,Rules,SSHbase
     root@lucid:~# mkdir /var/lib/bcfg2/Probes
-    root@lucid:~# cat /var/lib/bcfg2/Probes/groups 
+    root@lucid:~# cat /var/lib/bcfg2/Probes/groups
     #!/bin/sh
 
     ARCH=`uname -m`
@@ -229,7 +229,7 @@ Add a base-packages bundle. Let's see what happens when we just populate it with
 
 .. code-block:: xml
 
-    root@lucid:~# cat /var/lib/bcfg2/Bundler/base-packages.xml 
+    root@lucid:~# cat /var/lib/bcfg2/Bundler/base-packages.xml
     <Bundle name='base-packages'>
             <Package name='ubuntu-standard'/>
     </Bundle>
@@ -250,23 +250,23 @@ Now if we run the client in debug mode (-d), we can see what this has done for u
     Probe groups has result:
     amd64
     Loaded tool drivers:
-     APT      Action   DebInit  POSIX   
+     APT      Action   DebInit  POSIX
     The following packages are specified in bcfg2:
-     ubuntu-standard 
+     ubuntu-standard
     The following packages are prereqs added by Packages:
-     adduser             debconf             hdparm              libdevmapper1.02.1  libk5crypto3        libparted1.8-12     libxml2             passwd              upstart            
-     apt                 debianutils         info                libdns53            libkeyutils1        libpci3             logrotate           pciutils            usbutils           
-     aptitude            dmidecode           install-info        libelf1             libkrb5-3           libpopt0            lsb-base            perl-base           wget               
-     at                  dnsutils            iptables            libept0             libkrb5support0     libreadline5        lshw                popularity-contest  zlib1g             
-     base-files          dosfstools          libacl1             libgcc1             liblwres50          libreadline6        lsof                psmisc             
-     base-passwd         dpkg                libattr1            libgdbm3            libmagic1           libselinux1         ltrace              readline-common    
-     bsdmainutils        ed                  libbind9-50         libgeoip1           libmpfr1ldbl        libsigc++-2.0-0c2a  man-db              rsync              
-     bsdutils            file                libc-bin            libgmp3c2           libncurses5         libssl0.9.8         memtest86+          sed                
-     cpio                findutils           libc6               libgssapi-krb5-2    libncursesw5        libstdc++6          mime-support        sensible-utils     
-     cpp                 ftp                 libcap2             libisc50            libpam-modules      libusb-0.1-4        ncurses-bin         strace             
-     cpp-4.4             gcc-4.4-base        libcomerr2          libisccc50          libpam-runtime      libuuid1            netbase             time               
-     cron                groff-base          libcwidget3         libisccfg50         libpam0g            libxapian15         parted              tzdata             
- 
+     adduser             debconf             hdparm              libdevmapper1.02.1  libk5crypto3        libparted1.8-12     libxml2             passwd              upstart
+     apt                 debianutils         info                libdns53            libkeyutils1        libpci3             logrotate           pciutils            usbutils
+     aptitude            dmidecode           install-info        libelf1             libkrb5-3           libpopt0            lsb-base            perl-base           wget
+     at                  dnsutils            iptables            libept0             libkrb5support0     libreadline5        lshw                popularity-contest  zlib1g
+     base-files          dosfstools          libacl1             libgcc1             liblwres50          libreadline6        lsof                psmisc
+     base-passwd         dpkg                libattr1            libgdbm3            libmagic1           libselinux1         ltrace              readline-common
+     bsdmainutils        ed                  libbind9-50         libgeoip1           libmpfr1ldbl        libsigc++-2.0-0c2a  man-db              rsync
+     bsdutils            file                libc-bin            libgmp3c2           libncurses5         libssl0.9.8         memtest86+          sed
+     cpio                findutils           libc6               libgssapi-krb5-2    libncursesw5        libstdc++6          mime-support        sensible-utils
+     cpp                 ftp                 libcap2             libisc50            libpam-modules      libusb-0.1-4        ncurses-bin         strace
+     cpp-4.4             gcc-4.4-base        libcomerr2          libisccc50          libpam-runtime      libuuid1            netbase             time
+     cron                groff-base          libcwidget3         libisccfg50         libpam0g            libxapian15         parted              tzdata
+
     Phase: initial
     Correct entries:        101
     Incorrect entries:      0