documented base plugin interfaces

3 files changed, 134 insertions, 233 deletions

diff --git a/doc/conf.py b/doc/conf.py
index 97a094a59..4497b78d7 100644
--- a/doc/conf.py
+++ b/doc/conf.py
@@ -211,3 +211,5 @@ else:
 
 # If false, no module index is generated.
 latex_use_modindex = False
+
+autodoc_default_flags = ['members', 'show-inheritance']
diff --git a/doc/development/plugins.txt b/doc/development/plugins.txt
index 96469602d..2609595a7 100644
--- a/doc/development/plugins.txt
+++ b/doc/development/plugins.txt
@@ -28,296 +28,195 @@ Server Plugin Types
 
 A plugin must implement at least one of the interfaces described
 below.  Each interface is available as a class in
-``Bcfg2.Server.Plugin``.  In most cases, a plugin must also inherit
-from ``Bcfg2.Server.Plugin.Plugin``, which is the base Plugin object
-(described below).  Some of the interfaces listed below are themselves
-Plugin objects, so your custom plugin would only need to inherit from
-the plugin type.
+:mod:``Bcfg2.Server.Plugin``.  In most cases, a plugin must also
+inherit from :class:`Bcfg2.Server.Plugin.Plugin`, which is the base
+Plugin object (described below).  Some of the interfaces listed below
+are themselves Plugin objects, so your custom plugin would only need
+to inherit from the plugin type.
+
+Plugin
+^^^^^^
+
+.. autoclass:: Bcfg2.Server.Plugin.Plugin
+   :inherited-members:
+   :show-inheritance:
+
+   .. automethod:: Bcfg2.Server.Plugin.Plugin.__init__
+
+With the exceptions of :class:`Bcfg2.Server.Plugin.Statistics` and
+:class:`Bcfg2.Server.Plugin.ThreadedStatistics`, the plugin interfaces
+listed below do **not** inherit from Plugin; they simply provide
+interfaces that a given plugin may or must implement.
 
 Generator
 ^^^^^^^^^
 
-Generator plugins contribute to literal client configurations.  That
-is, they generate entry contents.  Examples are
-:ref:`server-plugins-generators-cfg` and
-:ref:`server-plugins-generators-sshbase`.
+.. autoclass:: Bcfg2.Server.Plugin.Generator
 
-An entry is generated in one of two ways:
-
-#.  First, the Bcfg2 core looks in the ``Entries`` dict attribute of
- the plugin object.  ``Entries`` is expected to be a dict whose keys
- are entry tags (e.g., ``"Path"``, ``"Service"``, etc.) and whose
- values are dicts; those dicts should map the ``name`` attribute of an
- entry to a callable that will be called to generate the content.  The
- callable will receive two arguments: the abstract entry (as an
- lxml.etree._Element object), and the client metadata object the entry
- is being generated for.
-#.  Second, if the entry is not listed in ``Entries``, the Bcfg2 core
- calls ``HandlesEntry(entry, metadata)``; if that returns True, then
- it calls ``HandleEntry(entry, metadata)``.
-
-The Generator plugin should provide one or both methods to bind
-entries, but does not have to provide both.
-
-Both ``HandleEntry()`` and the callable objects in the ``Entries``
-dict should return an lxml.etree._Element object representing the
-fully-bound entry.  They should raise
-``Bcfg2.Server.Plugin.PluginExecutionError`` with a sensible error
-message on failure.
+Examples are :ref:`server-plugins-generators-cfg` and
+:ref:`server-plugins-generators-sshbase`.
 
 Structure
 ^^^^^^^^^
 
-Structure Plugins contribute to abstract client configurations.  That
-is, they produce lists of entries that will be generated for a client.
-:ref:`server-plugins-structure-bundler-index` is a Structure plugin.
+.. autoclass:: Bcfg2.Server.Plugin.Structure
 
-Structure plugins must implement one method: ``BuildStructures()``,
-which is called with a single argument, a client metadata object
-structures should be built for.  It should return a list of
-lxml.etree._Element objects that will be added to the top-level
-``<Configuration>`` tag of the client configuration.  Consequently,
-each object in the list must consist of a container tag (e.g.,
-``<Bundle>`` or ``<Independent>``) which contains the entry tags.  It
-must not return a list of entry tags.
+:ref:`server-plugins-structures-bundler-index` is a Structure plugin.
 
 Metadata
 ^^^^^^^^
 
-Metadata plugins provide client metadata.
-:ref:`server-plugins-grouping-metadata` is a Metadata plugin.
+.. autoclass:: Bcfg2.Server.Plugin.Metadata
 
-Metadata plugins **must** implement the following methods:
-
-* ``AuthenticateConnection(cert, user, password, addresspair)``:
-  Authenticate the given client.  Arguments are:
-  * ``cert``: an x509 certificate dict;
-  * ``user``: The username of the user trying to authenticate;
-  * ``password``: The password supplied by the client;
-  * ``addresspair``: A tuple of ``(<ip address>, <hostname>)``
-  ``AuthenticateConnection()`` should return True if the authenticate
-  succeeds, False otherwise.  Failures should be logged at the error
-  level.
-* ``get_initial_metadata(client_name)``: Return a ClientMetadata
-  object that fully describes everything the Metadata plugin knows
-  about the named client.  See :file:``Metadata.py`` for a reference
-  implementation of the ClientMetadata object.
-* ``merge_additional_data(metadata, source, data)``: Add data from the
-  Connector plugin named by ``<source>`` (a string giving the name of
-  the Connector plugin) to the given metadata object.
-  ``merge_additional_data()`` should modify the ``metadata`` object in
-  place; it doesn't need to return anything.
-* ``merge_additional_groups(metadata, groups)``: Add groups from an
-  anonymous Connector plugin to the given metadata object.
-  ``merge_additional_groups()`` should modify the ``metadata`` object in
-  place; it doesn't need to return anything.
-
-Metadata plugins **may** implement the following methods:
-
-* ``viz(hosts, bundles, key, only_client, colors)``: Return a string
-  containing a graphviz document that maps out the Metadata.  The
-  first three options are boolean, and describe whether or not the
-  named item(s) should be included in the graphviz document.
-  ``only_client`` is the name of a client which, if included, will be
-  the only client whose metadata will be included on the map.
-  ``colors`` is a list of graphviz color names to use.  If
-  unimplemented, the empty string will be returned.
-* ``set_version(client, version)``: Set the version for the named
-  client to the specified version string.  If unimplemented, setting
-  the version of a client will fail silently.
-* ``set_profile(client, profile, addresspair)``: Set the profile for
-  the named client to the named profile group.  If unimplemented,
-  setting a client profile will fail silently.
-* ``resolve_client(addresspair, cleanup_cache=False)``: Given a tuple
-  of ``(<ip address>, <hostname>)``, resolve the canonical name of
-  this client.  If this method is not implemented, the hostname
-  claimed by the client is used.  (This may be a security risk; it's
-  highly recommended that you implement ``resolve_client`` if you are
-  writing a Metadata plugin.)
+:ref:`server-plugins-grouping-metadata` is a Metadata plugin.
 
 Connector
 ^^^^^^^^^
 
-Connector plugins augment client metadata instances with additional
-data, additional groups, or both.  Connector plugins include
+.. autoclass:: Bcfg2.Server.Plugin.Connector
+
+Connector plugins include
 :ref:`server-plugins-grouping-grouppatterns`,
 :ref:`server-plugins-connectors-properties`, and
 :ref:`server-plugins-probes-index`.
 
-Connector plugins should implement one or all of the following
-methods:
-
-* ``get_additional_groups(metadata)``: Return a list of additional
-  groups for the given ClientMetadata object.  If unimplemented, the
-  empty list is returned.
-* ``get_additional_data(metadata)``: Return arbitrary additional data
-  for the given ClientMetadata object.  By convention this is usually
-  a dict object, but doesn't need to be.  If unimplemented, the empty
-  dict is returned.
-
 Probing
 ^^^^^^^
 
-Probing plugins can collect data from clients and process it.
+.. autoclass:: Bcfg2.Server.Plugin.Probing
+
 Examples include :ref:`server-plugins-probes-index` and
 :ref:`server-plugins-probes-fileprobes`.
 
-Probing plugins must implement the following methods:
+Statistics
+^^^^^^^^^^
 
-* ``GetProbes(metadata)``: Return a list of probes for the given
-  ClientMetadata object.  Each probe should be an XML document,
-  described below.
-* ``ReceiveData(metadata, datalist)``: Process data returned from the
-  probes for the given ClientMetadata object.  ``datalist`` is a list
-  of lxml.etree._Element objects, each of which is a single tag; the
-  ``name`` attribute holds the unique name of the probe that was run,
-  and the text contents of the tag hold the results of the probe.
+.. autoclass:: Bcfg2.Server.Plugin.Statistics
 
-``GetProbes()`` returns a list of probes, each of which is an
-``lxml.etree._Element`` object that adheres to the following
-specification.  Each probe must the following attributes:
+The Statistics object is itself a :class:`Bcfg2.Server.Plugin.Plugin`
+object, so objects that inherit from Statistics do not have to also
+inherit from Plugin.
 
-* ``name``: The unique name of the probe.
-* ``source``: The origin of the probe; probably the name of the plugin
-  that supplies the probe.
-* ``interpreter``: The command that will be run on the client to
-  interpret the probe script.  Compiled (i.e., non-interpreted) probes
-  are not supported.
+ThreadedStatistics
+^^^^^^^^^^^^^^^^^^
 
-The text of the XML tag should be the contents of the probe, i.e., the
-code that will be run on the client.
+.. autoclass:: Bcfg2.Server.Plugin.ThreadedStatistics
 
-Statistics
+:ref:`server-plugins-statistics-dbstats` is an example of a
+ThreadedStatistics plugin.
+
+The ThreadedStatistics object is itself a
+:class:`Bcfg2.Server.Plugin.Plugin` object, so objects that inherit
+from ThreadedStatistics do not have to also inherit from Plugin.
+
+PullSource
+^^^^^^^^^^
+
+.. autoclass:: Bcfg2.Server.Plugin.PullSource
+
+:ref:`server-plugins-statistics-dbstats` is an example of a plugin
+that implements the PullSource interface
+
+PullTarget
 ^^^^^^^^^^
 
-Signal statistics handling capability
+.. autoclass:: Bcfg2.Server.Plugin.PullTarget
+
+:ref:`server-plugins-generators-sshbase` is an example of a plugin
+that implements the PullTarget interface
 
 Decision
 ^^^^^^^^
 
-Signal decision handling capability
+.. autoclass:: Bcfg2.Server.Plugin.Decision
 
-Version
-^^^^^^^
+:ref:`server-plugins-generators-decisions` is an example of a Decision
+plugin, and has much more information about how decisions are used.
 
-Interact with various version control systems
-
-Writing Bcfg2 Server Plugins
-----------------------------
-
-Bcfg2 plugins are python classes that subclass from
-Bcfg2.Server.Plugin.Plugin. Several plugin-specific values must be set
-in the new plugin. These values dictate how the new plugin will behave
-with respect to the above four functions.  The following table describes
-all important member fields.
-
-+-----------------+-----------------------------------+--------------------------+
-| Name            | Description                       | Format                   |
-+=================+===================================+==========================+
-| __name__        | The name of the plugin            | string                   |
-+-----------------+-----------------------------------+--------------------------+
-| __version__     | The plugin version (generally     | string                   |
-|                 | tied to revctl keyword expansion) |                          |
-+-----------------+-----------------------------------+--------------------------+
-| __author__      | The plugin author                 | string                   |
-+-----------------+-----------------------------------+--------------------------+
-| __rmi__         | Set of functions to be exposed as | List of function names   |
-|                 | XML-RPC functions                 | (strings)                |
-+-----------------+-----------------------------------+--------------------------+
-| Entries         | Multidimentional dictionary of    | Dictionary of            |
-|                 | keys that point to the function   | ConfigurationEntityType, |
-|                 | used to bind literal contents for | Name keys, and function  |
-|                 | a given configuration entity      | reference values         |
-+-----------------+-----------------------------------+--------------------------+
-| BuildStructures | Function that returns a list of   | Member function          |
-|                 | the structures for a given client |                          |
-+-----------------+-----------------------------------+--------------------------+
-| GetProbes       | Function that returns a list of   | Member function          |
-|                 | probes that a given client should |                          |
-|                 | execute                           |                          |
-+-----------------+-----------------------------------+--------------------------+
-| ReceiveData     | Function that accepts the probe   | Member function          |
-|                 | results for a given client        |                          |
-+-----------------+-----------------------------------+--------------------------+
-
-Example Plugin
-^^^^^^^^^^^^^^
+StructureValidator
+^^^^^^^^^^^^^^^^^^
 
-.. code-block:: python
+.. autoclass:: Bcfg2.Server.Plugin.StructureValidator
 
-    import Bcfg2.Server.Plugin
-    class MyPlugin(Bcfg2.Server.Plugin.Plugin):
-       '''An example plugin'''
-       # All plugins need to subclass Bcfg2.Server.Plugin.Plugin
-       __name__ = 'MyPlugin'
-       __version__ = '1'
-       __author__ = 'me@me.com'
-       __rmi__ = ['myfunction']
-       # myfunction is now available remotely as MyPlugin.myfunction
-
-       def __init__(self, core, datastore):
-           Bcfg2.Server.Plugin.Plugin.__init__(self, core, datastore)
-           self.Entries = {'Path':{'/etc/foo.conf': self.buildFoo}}
-
-       def myfunction(self):
-           '''function for xmlrpc rmi call'''
-           #do something
-           return True
-
-       def buildFoo(self, entry, metadata):
-           '''Bind per-client information into entry based on metadata'''
-           entry.attrib.update({'type':'file', 'owner':'root', 'group':'root', 'perms':'644'})
-           entry.text = '''contents of foo.conf'''
-
-Example Connector
-^^^^^^^^^^^^^^^^^
+Examples are :ref:`server-plugins-structures-defaults` and
+:ref:`server-plugins-structures-deps`.
 
-.. code-block:: python
+GoalValidator
+^^^^^^^^^^^^^
 
-    import Bcfg2.Server.Plugin
+.. autoclass:: Bcfg2.Server.Plugin.GoalValidator
 
-    class Foo(Bcfg2.Server.Plugin.Plugin,
-             Bcfg2.Server.Plugin.Connector):
-        '''The Foo plugin is here to illustrate a barebones connector'''
-        name = 'Foo'
-        experimental = True
+An example of a GoalValidator plugin would be the ServiceCompat plugin
+that is used to provide old-style Service tag attributes to older
+clients from a Bcfg2 1.3.0 server.  As a final stage of configuration
+generation, it translates the new "restart" and "install" attributes
+into the older "mode" attribute.
 
-        def __init__(self, core, datastore):
-            Bcfg2.Server.Plugin.Plugin.__init__(self, core, datastore)
-            Bcfg2.Server.Plugin.Connector.__init__(self)
-            self.store = XMLFileBacked(self.data, core.fam)
+Version
+^^^^^^^
+
+.. autoclass:: Bcfg2.Server.Plugin.Version
 
-        def get_additional_data(self, metadata):
+Examples include :ref:`server-plugins-version-git` and
+:ref:`server-plugins-version-svn2`.
+
+ClientRunHooks
+^^^^^^^^^^^^^^
 
-            mydata = {}
-            for data in self.store.entries['foo.xml'].xdata.get("foo", []):
+.. autoclass:: Bcfg2.Server.Plugin.ClientRunHooks
 
-                mydata[data] = "bar"
+Examples are :ref:`server-plugins-misc-trigger` and
+:ref:`server-plugins-connectors-puppetenc`.
 
-            return  dict([('mydata', mydata)])
+Exposing XML-RPC Functions
+--------------------------
+
+Plugins can expose XML-RPC functions that can then be called with
+:ref:`bcfg2-admin xcmd <server-admin-xcmd>`.  Note that there is
+absolutely no access control beyond the initial authentication, so
+take care to not expose any data or behavior via XML-RPC that you
+would not want all of your clients to be able to see or use.
+
+To expose a function, simply add its name to the ``__rmi__`` class
+attribute.  (RMI stands for "Remote Method Invocation.")  Consider
+this example from the :ref:`server-plugins-generators-packages`
+plugin:
+
+.. code-block:: python
 
-        def get_additional_groups(self, meta):
-            return self.cgroups.get(meta.hostname, list())
+    class Packages(Bcfg2.Server.Plugin.Plugin,
+                   Bcfg2.Server.Plugin.StructureValidator,
+                   Bcfg2.Server.Plugin.Generator,
+                   Bcfg2.Server.Plugin.Connector,
+                   Bcfg2.Server.Plugin.ClientRunHooks):
+        name = 'Packages'
+        conflicts = ['Pkgmgr']
+        __rmi__ = Bcfg2.Server.Plugin.Plugin.__rmi__ + ['Refresh', 'Reload']
 
-Example Metadata plugin
-^^^^^^^^^^^^^^^^^^^^^^^
+    def Refresh(self):
+        self._load_config(force_update=True)
+        return True
 
-If you would like to define your own Metadata plugin (to extend/change
-functionality of the existing Metadata plugin), here are the steps to
-do so. We will call our new plugin `MyMetadata`.
+    def Reload(self):
+        self._load_config()
+        return True
 
-#. Add MyMetadata.py
+This exposes two functions, ``Refresh`` and ``Reload``, in addition to
+any default methods that are already exposed.  To call one of these
+functions, you could run::
 
-   .. code-block:: python
+    bcfg2-admin xcmd Packages.Refresh
 
-       import Bcfg2.Server.Plugins.Metadata
+Plugin Helper Classes
+---------------------
 
-       class MyMetadata(Bcfg2.Server.Plugins.Metadata.Metadata):
-           '''This class contains data for bcfg2 server metadata'''
-           __author__ = 'bcfg-dev@mcs.anl.gov'
+.. autoclass:: Bcfg2.Server.Plugin.Debuggable
 
-           def __init__(self, core, datastore, watch_clients=True):
-               Bcfg2.Server.Plugins.Metadata.Metadata.__init__(self, core, datastore, watch_clients)
+Plugin Exceptions
+-----------------
 
-#. Add MyMetadata to ``src/lib/Server/Plugins/__init__.py``
-#. Replace Metadata with MyMetadata in the plugins line of bcfg2.conf
+.. autoexception:: Bcfg2.Server.Plugin.ValidationError
+.. autoexception:: Bcfg2.Server.Plugin.PluginInitError
+.. autoexception:: Bcfg2.Server.Plugin.PluginExecutionError
+.. autoexception:: Bcfg2.Server.Plugin.MetadataConsistencyError
+.. autoexception:: Bcfg2.Server.Plugin.MetadataRuntimeError
diff --git a/doc/server/plugins/structures/deps.txt b/doc/server/plugins/structures/deps.txt
index 7c5861d06..3c5177f4d 100644
--- a/doc/server/plugins/structures/deps.txt
+++ b/doc/server/plugins/structures/deps.txt
@@ -1,6 +1,6 @@
 .. -*- mode: rst -*-
 
-.. _server-plugins-generators-deps:
+.. _server-plugins-structures-deps:
 
 ====
 Deps