Metadata and other improvements:

* Added support for Client tag in groups.xml
* Added support for nested Group tags in groups.xml
* Added support for negated groups in groups.xml
* Added DatabaseBacked plugin mixin to easily allow plugins to connect
  to a database specified in global database settings in bcfg2.conf
* Added DBMetadata plugin that uses relational DB to store client
  records instead of writing to clients.xml

8 files changed, 229 insertions, 54 deletions

diff --git a/doc/appendix/guides/authentication.txt b/doc/appendix/guides/authentication.txt
index dab122f80..68a232f6f 100644
--- a/doc/appendix/guides/authentication.txt
+++ b/doc/appendix/guides/authentication.txt
@@ -62,7 +62,7 @@ How Authentication Works
 
 #. Next, the ip address is verified against the client record. If the
    address doesn't match, then the client must be set to
-   location=floating
+   floating='true'
 
 #. Finally, the password is verified. If the client is set to secure
    mode, the only its per-client password is accepted. If it is not set
diff --git a/doc/appendix/guides/nat_howto.txt b/doc/appendix/guides/nat_howto.txt
index 5bd3f7b13..b3492e871 100644
--- a/doc/appendix/guides/nat_howto.txt
+++ b/doc/appendix/guides/nat_howto.txt
@@ -44,7 +44,7 @@ the Client entry in clients.xml will look something like this:
 .. code-block:: xml
 
     <Client profile="desktop" name="test1"
-     uuid='9001ec29-1531-4b16-8198-a71bea093d0a' location='floating'/>
+     uuid='9001ec29-1531-4b16-8198-a71bea093d0a' floating='true'/>
 
 Alternatively, the Client entry can be setup like this:
 
diff --git a/doc/server/backends.txt b/doc/server/backends.txt
index 49bfe3b96..71ecac10b 100644
--- a/doc/server/backends.txt
+++ b/doc/server/backends.txt
@@ -2,9 +2,9 @@
 
 .. _server-backends:
 
-========
-Backends
-========
+===============
+Server Backends
+===============
 
 .. versionadded:: 1.3.0
 
diff --git a/doc/server/database.txt b/doc/server/database.txt
new file mode 100644
index 000000000..8094e9c5e
--- /dev/null
+++ b/doc/server/database.txt
@@ -0,0 +1,45 @@
+.. -*- mode: rst -*-
+
+.. _server-database:
+
+========================
+Global Database Settings
+========================
+
+.. versionadded:: 1.3.0
+
+Several Bcfg2 plugins, including
+:ref:`server-plugins-grouping-dbmetadata` and
+:ref:`server-plugins-probes-index`, can connect use a relational
+database to store data.  They use the global database settings in
+``bcfg2.conf``, described in this document, to connect.
+
+.. note::
+
+    The :ref:`server-plugins-statistics-dbstats` plugin and the
+    :ref:`reports-dynamic` do *not* currently use the global database
+    settings.  They use their own separate database configuration.
+
+Configuration Options
+=====================
+
+All of the following options should go in the ``[database]`` section
+of ``/etc/bcfg2.conf``.
+
++-------------+------------------------------------------------------------+-------------------------------+
+| Option name | Description                                                | Default                       |
++=============+============================================================+===============================+
+| engine      | The full name of the Django database backend to use. See   | "django.db.backends.sqlite3"  |
+|             | https://docs.djangoproject.com/en/dev/ref/settings/#engine |                               |
+|             | for available options                                      |                               |
++-------------+------------------------------------------------------------+-------------------------------+
+| name        | The name of the database                                   | "/var/lib/bcfg2/bcfg2.sqlite" |
++-------------+------------------------------------------------------------+-------------------------------+
+| user        | The user to connect to the database as                     | None                          |
++-------------+------------------------------------------------------------+-------------------------------+
+| password    | The password to connect to the database with               | None                          |
++-------------+------------------------------------------------------------+-------------------------------+
+| host        | The host to connect to                                     | "localhost"                   |
++-------------+------------------------------------------------------------+-------------------------------+
+| port        | The port to connect to                                     | None                          |
++-------------+------------------------------------------------------------+-------------------------------+
diff --git a/doc/server/index.txt b/doc/server/index.txt
index 6c2b7b889..1b832dbee 100644
--- a/doc/server/index.txt
+++ b/doc/server/index.txt
@@ -30,3 +30,4 @@ clients.
    bcfg2-info
    selinux
    backends
+   database
diff --git a/doc/server/plugins/generators/tgenshi/clientsxml.txt b/doc/server/plugins/generators/tgenshi/clientsxml.txt
index 7a8d1fcc4..87d6d728a 100644
--- a/doc/server/plugins/generators/tgenshi/clientsxml.txt
+++ b/doc/server/plugins/generators/tgenshi/clientsxml.txt
@@ -65,7 +65,7 @@ Possible improvements:
                name="${name}"
                uuid="${name}"
                password="${metadata.Properties['passwords.xml'].xdata.find('password').find('bcfg2-client').find(name).text}"
-               location="floating"
+               floating="true"
                secure="true"
            />\
        {% end %}\
diff --git a/doc/server/plugins/grouping/dbmetadata.txt b/doc/server/plugins/grouping/dbmetadata.txt
new file mode 100644
index 000000000..292367f0c
--- /dev/null
+++ b/doc/server/plugins/grouping/dbmetadata.txt
@@ -0,0 +1,39 @@
+.. -*- mode: rst -*-
+
+.. _server-plugins-grouping-dbmetadata:
+
+==========
+DBMetadata
+==========
+
+.. versionadded:: 1.3.0
+
+The DBMetadata plugin is an alternative to the
+:ref:`server-plugins-grouping-metadata` plugin that stores client
+records in a database rather than writing back to ``clients.xml``.
+This provides several advantages:
+
+* ``clients.xml`` will never be written by the server, removing an
+  area of contention between the user and server.
+* ``clients.xml`` can be removed entirely for many sites.
+* The Bcfg2 client list can be queried by other machines without
+  obtaining and parsing ``clients.xml``.
+* A single client list can be shared amongst multiple Bcfg2 servers.
+
+In general, DBMetadata works almost the same as Metadata.
+``groups.xml`` is parsed identically.  If ``clients.xml`` is present,
+it is parsed, but ``<Client>`` tags in ``clients.xml`` *do not* assert
+client existence; they are only used to set client options *if* the
+client exists (in the database).  That is, the two purposes of
+``clients.xml`` -- to track which clients exist, and to set client
+options -- have been separated.
+
+With the improvements in ``groups.xml`` parsing in 1.3, client groups
+can now be set directly in ``groups.xml`` with ``<Client>`` tags. (See
+:ref:`metadata-client-tag` for more details.)  As a result,
+``clients.xml`` is only necessary with DBMetadata if you need to set
+options (e.g., aliases, floating clients, per-client passwords, etc.)
+on clients.
+
+DBMetadata uses the :ref:`Global Server Database Settings
+<server-database>` to connect to its database.
diff --git a/doc/server/plugins/grouping/metadata.txt b/doc/server/plugins/grouping/metadata.txt
index 2c05e9e7e..1ab3b9c05 100644
--- a/doc/server/plugins/grouping/metadata.txt
+++ b/doc/server/plugins/grouping/metadata.txt
@@ -6,11 +6,11 @@
 Metadata
 ========
 
-The metadata mechanism has two types of information, client metadata and
-group metadata. The client metadata describes which top level group a
-client is associated with.The group metadata describes groups in terms
-of what bundles and other groups they include. Each aspect grouping
-and clients' memberships are reflected in the ``Metadata/groups.xml`` and
+The metadata mechanism has two types of information, client metadata
+and group metadata. The client metadata describes which top level
+group a client is associated with.The group metadata describes groups
+in terms of what bundles and other groups they include. Group data and
+clients' memberships are reflected in the ``Metadata/groups.xml`` and
 ``Metadata/clients.xml`` files, respectively.
 
 Usage of Groups in Metadata
@@ -85,9 +85,9 @@ Additionally, the following properties can be specified:
 | address  | Establishes an extra IP address that   | ip address     |
 |          | resolves to this client.               |                |
 +----------+----------------------------------------+----------------+
-| location | Requires requests to come from an IP   | fixed|floating |
-|          | address that matches the client        |                |
-|          | record.                                |                |
+| floating | Allows requests to come from any IP,   | true|false     |
+|          | rather than requiring requests to come |                |
+|          | from an IP associated with the client  |                |
 +----------+----------------------------------------+----------------+
 | password | Establishes a per-node password that   | String         |
 |          | can be used instead of the global      |                |
@@ -101,6 +101,9 @@ Additionally, the following properties can be specified:
 |          | resolution.                            |                |
 +----------+----------------------------------------+----------------+
 
+Floating can also be configured by setting ``location="floating"``,
+but that is deprecated.
+
 For detailed information on client authentication see
 :ref:`appendix-guides-authentication`
 
@@ -112,31 +115,88 @@ definitions. Here's a simple ``Metadata/groups.xml`` file:
 
 .. code-block:: xml
 
-    <Groups version='3.0'>
+    <Groups>
       <Group name='mail-server' profile='true'
-                                public='false'
                                 comment='Top level mail server group' >
         <Bundle name='mail-server'/>
         <Bundle name='mailman-server'/>
         <Group name='apache-server'/>
-        <Group name='rhel-as-5-x86'/>
         <Group name='nfs-client'/>
         <Group name='server'/>
+        <Group name='rhel5'>
+          <Group name='sendmail-server'/>
+        </Group>
+        <Group name='rhel6'>
+          <Group name='postfix-server'/>
+        </Group>
+      </Group>
+      <Group name='rhel'>
+        <Group name='selinux-enabled'/>
       </Group>
-      <Group name='rhel-as-5-x86'>
-         <Group name='rhel'/>
+      <Group name='oracle-server'>
+        <Group name='selinux-enabled' negate='true'/>
       </Group>
-      <Group name='apache-server'/>
-      <Group name='nfs-client'/>
-      <Group name='server'/>
-      <Group name='rhel'/>
+      <Client name='foo.eample.com'>
+        <Group name='oracle-server'/>
+        <Group name='apache-server'/>
+      </Client>
     </Groups>
 
+A Group or Client tag that does not contain any child tags is a
+declaration of membership; a Group or Client tag that does contain
+children is a conditional.  So the example above does not assign
+either the ``rhel5`` or ``rhel6`` groups to machines in the
+``mail-server`` group, but conditionally assigns the
+``sendmail-server`` or ``postfix-server`` groups depending on the OS
+of the client.  (Presumably in this example the OS groups are set by a
+probe.)
+
+Consequently, a client that is RHEL 5 and a member of the
+``mail-server`` profile group would also be a member of the
+``apache-server``, ``nfs-client``, ``server``, and ``sendmail-server``
+groups; a RHEL 6 client that is a member of the ``mail-server``
+profile group would be a member of the ``apache-server``,
+``nfs-client``, ``server``, and ``postfix-server`` groups.
+
+Client tags in ``groups.xml`` allow you to supplement the profile
+group declarations in ``clients.xml`` and/or client group assignments
+with the :ref:`server-plugins-grouping-grouppatterns` plugin.  They
+should be used sparingly.  (They are more useful with the
+:ref:`server-plugins-grouping-dbmetadata` plugin.)
+
+You can also declare that a group should be negated; this allows you
+to set defaults and override them efficiently.  Negation is applied
+after other group memberships are calculated, so it doesn't matter how
+many times a client is assigned to a group or how many times it is
+negated; a single group negation is sufficient to remove a client from
+that group.  For instance, in the following example,
+``foo.example.com`` is **not** a member of ``selinux-enabled``, even
+though it is a member of the ``foo-server`` and ``every-server``
+groups:
+
+.. code-block:: xml
+
+    <Groups>
+      <Group name="foo-server">
+        <Group name="apache-server"/>
+        <Group name="selinux-enabled"/>
+      </Group>
+      <Group name="apache-server">
+        <Group name="selinux-enabled"/>
+      </Group>
+      <Group name="every-server">
+        <Group name="selinux-enabled"/>
+      </Group>
+      <Client name="foo.example.com">
+        <Group name="selinux-enabled" negate="true"/>
+      </Client>
+
+.. note::
 
-Nested/chained groups definitions are conjunctive (logical and). For
-instance, in the above example, a client associated with the Profile
-Group ``mail-server`` is also a member of the ``apache-server``,
-``rhel-as-5-x86``, ``nfs-client``, ``server``, and ``rhel`` groups.
+    Nested Group conditionals, Client tags, and negated Group tags are
+    all new in 1.3.0.
+
+Order of ``groups.xml`` does not matter.
 
 Groups describe clients in terms for abstract, disjoint aspects. Groups
 can be combined to form complex descriptions of clients that use
@@ -165,33 +225,63 @@ Metadata Group Tag
 
 The Group Tag has the following possible attributes:
 
-+----------+------------------------------------------+--------------+
-| Name     | Description                              | Values       |
-+==========+==========================================+==============+
-| name     | Name of the group                        | String       |
-+----------+------------------------------------------+--------------+
-| profile  | If a client can be directly associated   | True|False   |
-|          | with this group                          |              |
-+----------+------------------------------------------+--------------+
-| public   | If a client can freely associate itself  | True|False   |
-|          | with this group. For use with the        |              |
-|          | *bcfg2 -p* option on the client.         |              |
-+----------+------------------------------------------+--------------+
-| category | A group can only contain one instance of | String       |
-|          | a group in any one category. This        |              |
-|          | provides the basis for representing      |              |
-|          | groups which are conjugates of one       |              |
-|          | another in a rigorous way. It also       |              |
-|          | provides the basis for negation.         |              |
-+----------+------------------------------------------+--------------+
-| default  | Set as the profile to use for clients    | True|False   |
-|          | that are not associated with a profile   |              |
-|          | in ``clients.xml``                       |              |
-+----------+------------------------------------------+--------------+
-| comment  | English text description of group        | String       |
-+----------+------------------------------------------+--------------+
-
-Groups can also contain other groups and bundles.
++----------+----------------------------------------------+--------------+
+| Name     | Description                                  | Values       |
++==========+==============================================+==============+
+| name     | Name of the group                            | String       |
++----------+----------------------------------------------+--------------+
+| profile  | If a client can be directly associated with  | True|False   |
+|          | this group                                   |              |
++----------+----------------------------------------------+--------------+
+| public   | If a client can freely associate itself with | True|False   |
+|          | this group. For use with the ``bcfg2 -p``    |              |
+|          | option on the client.                        |              |
++----------+----------------------------------------------+--------------+
+| category | A group can only contain one instance of a   | String       |
+|          | group in any one category. This provides the |              |
+|          | basis for representing groups which are      |              |
+|          | conjugates of one another in a rigorous way. |              |
+|          | way.                                         |
++----------+----------------------------------------------+--------------+
+| default  | Set as the profile to use for clients that   | True|False   |
+|          | are not associated with a profile in         |              |
+|          | ``clients.xml``                              |              |
++----------+----------------------------------------------+--------------+
+| comment  | English text description of group            | String       |
++----------+----------------------------------------------+--------------+
+| negate   | When used as a conditional, only apply the   | True|False   |
+|          | children if the named group does not match.  |              |
+|          | When used as a declaration, do not apply     |              |
+|          | the named group to matching clients.         |              |
++----------+----------------------------------------------+--------------+
+
+The ``profile``, ``public``, ``category``, ``default``, and
+``comment`` attributes are only parsed if a Group tag either a) is the
+direct child of a Groups tag (i.e., at the top level of an XML file);
+or b) has no children.  This matches legacy behavior in Bcfg2 1.2 and
+earlier.
+
+Groups can also contain other groups, clients, and bundles.
+
+.. _metadata-client-tag:
+
+Metadata Client Tag
+-------------------
+
+The Client Tag has the following possible attributes:
+
++----------+-----------------------------------------------+--------------+
+| Name     | Description                                   | Values       |
++==========+===============================================+==============+
+| name     | Name of the client                            | String       |
++----------+-----------------------------------------------+--------------+
+| negate   | Only apply the child tags if the named client | True|False   |
+|          | does not match.                               |              |
++----------+-----------------------------------------------+--------------+
+
+Clients can also contain groups, other clients (although that's likely
+nonsensical), and bundles.
+
 
 Use of XInclude
 ===============