path: root/doc/server/plugins/grouping/metadata.txt

].. -*- mode: rst -*-

.. _server-plugins-grouping-metadata:

========
Metadata
========

The metadata mechanism has two types of information, client metadata
and group metadata. The client metadata describes which top level
group a client is associated with.The group metadata describes groups
in terms of what bundles and other groups they include. Group data and
clients' memberships are reflected in the ``Metadata/groups.xml`` and
``Metadata/clients.xml`` files, respectively.

Usage of Groups in Metadata
===========================

Clients are assigned membership of groups in the Metadata descriptions.
Clients can be directly assigned to *'profile'* or *'public'* groups.
Client membership of all other groups is by those groups being
associated with the profile or public groups. This file can be indirectly
modified from clients through use of the ``-p`` flag to ``bcfg2``.

Clients are associated with profile groups in ``Metadata/clients.xml`` as
shown below.

.. _server-plugins-grouping-metadata-clients-xml:

Metadata/clients.xml
====================

The ``Metadata/clients.xml`` file contains the mappings of Profile Groups
to clients. The file is just a series of *<Client />* tags, each of which
describe one host. A sample file is below:

.. code-block:: xml

    <Clients version="3.0">
      <Client profile="backup-server" name="backup.example.com"/>
      <Client profile="console-server" name="con.example.com"/>
      <Client profile="kerberos-master" name="kdc.example.com"/>
      <Client profile="mail-server" name="mail.example.com"/>
      <Client name='foo' address='10.0.0.1'>
          <Alias name='foo-mgmt' address='10.1.0.1'/>
      </Client>
    </Clients>

Clients Tag
-----------

The Clients tag has the following possible attributes:

+---------+-----------------------+--------+
| Name    | Description           | Values |
+=========+=======================+========+
| version | Client schema version | String |
+---------+-----------------------+--------+

Client Tag
----------

Each entry in ``clients.xml`` **must** have the following properties:

+---------+---------------------------------------+--------+
| Name    | Description                           | Values |
+=========+=======================================+========+
| name    | Host name of client. This needs to be | String |
|         | the name (possibly a FQDN) returned   |        |
|         | by a reverse lookup on the connecting |        |
|         | IP address.                           |        |
+---------+---------------------------------------+--------+
| profile | Profile group name to associate this  | String |
|         | client with.                          |        |
+---------+---------------------------------------+--------+

Additionally, the following properties can be specified:

+----------+----------------------------------------+----------------+
| Name     | Description                            | Values         |
+==========+========================================+================+
| Alias    | Alternative name and address for the   | XML Element    |
|          | client.                                |                |
+----------+----------------------------------------+----------------+
| address  | Establishes an extra IP address that   | ip address     |
|          | resolves to this client.               |                |
+----------+----------------------------------------+----------------+
| floating | Allows requests to come from any IP,   | true|false     |
|          | rather than requiring requests to come |                |
|          | from an IP associated with the client  |                |
+----------+----------------------------------------+----------------+
| password | Establishes a per-node password that   | String         |
|          | can be used instead of the global      |                |
|          | password.                              |                |
+----------+----------------------------------------+----------------+
| secure   | Requires the use of the per-client     | true|false     |
|          | password for this client.              |                |
+----------+----------------------------------------+----------------+
| uuid     | Establishes a per-node name that can   | String         |
|          | be used to bypass dns-based client     |                |
|          | resolution.                            |                |
+----------+----------------------------------------+----------------+

Floating can also be configured by setting ``location="floating"``,
but that is deprecated.

For detailed information on client authentication see
:ref:`appendix-guides-authentication`

Clients Database
~~~~~~~~~~~~~~~~

.. versionadded:: 1.3.0

It is also possible to store client records in a database rather than
writing back to ``clients.xml``.  This provides several advantages:

* ``clients.xml`` will never be written by the server, removing an
  area of contention between the user and server.
* ``clients.xml`` can be removed entirely for many sites.
* The Bcfg2 client list can be queried by other machines without
  obtaining and parsing ``clients.xml``.
* A single client list can be shared amongst multiple Bcfg2 servers.

In general, storing clients in the database works almost the same as
``clients.xml``.  ``groups.xml`` is parsed identically.  If
``clients.xml`` is present, it is parsed, but ``<Client>`` tags in
``clients.xml`` *do not* assert client existence; they are only used
to set client options *if* the client exists (in the database).  That
is, the two purposes of ``clients.xml`` -- to track which clients
exist, and to set client options -- have been separated.

With the improvements in ``groups.xml`` parsing in 1.3, client groups
can now be set directly in ``groups.xml`` with ``<Client>`` tags. (See
:ref:`metadata-client-tag` for more details.)  As a result,
``clients.xml`` is only necessary if you need to set
options (e.g., aliases, floating clients, per-client passwords, etc.)
on clients.

To use the database backend instead of ``clients.xml``, set
``use_database`` in the ``[metadata]`` section of ``bcfg2.conf`` to
``true``.  You will also need to configure the :ref:`Global Server
Database Settings <server-database>`.

The ``clients.xml``-based model remains the default.

Metadata/groups.xml
===================

The ``Metadata/groups.xml`` file contains Group and Profile
definitions. Here's a simple ``Metadata/groups.xml`` file:

.. code-block:: xml

    <Groups>
      <Group name='mail-server' profile='true'
                                comment='Top level mail server group' >
        <Bundle name='mail-server'/>
        <Bundle name='mailman-server'/>
        <Group name='apache-server'/>
        <Group name='nfs-client'/>
        <Group name='server'/>
        <Group name='rhel5'>
          <Group name='sendmail-server'/>
        </Group>
        <Group name='rhel6'>
          <Group name='postfix-server'/>
        </Group>
      </Group>
      <Group name='rhel'>
        <Group name='selinux-enabled'/>
      </Group>
      <Group name='oracle-server'>
        <Group name='selinux-enabled' negate='true'/>
      </Group>
      <Client name='foo.eample.com'>
        <Group name='oracle-server'/>
        <Group name='apache-server'/>
      </Client>
    </Groups>

A Group or Client tag that does not contain any child tags is a
declaration of membership; a Group or Client tag that does contain
children is a conditional.  So the example above does not assign
either the ``rhel5`` or ``rhel6`` groups to machines in the
``mail-server`` group, but conditionally assigns the
``sendmail-server`` or ``postfix-server`` groups depending on the OS
of the client.  (Presumably in this example the OS groups are set by a
probe.)

Consequently, a client that is RHEL 5 and a member of the
``mail-server`` profile group would also be a member of the
``apache-server``, ``nfs-client``, ``server``, and ``sendmail-server``
groups; a RHEL 6 client that is a member of the ``mail-server``
profile group would be a member of the ``apache-server``,
``nfs-client``, ``server``, and ``postfix-server`` groups.

Client tags in ``groups.xml`` allow you to supplement the profile
group declarations in ``clients.xml`` and/or client group assignments
with the :ref:`server-plugins-grouping-grouppatterns` plugin.  They
should be used sparingly.  (They are more useful when you are using
the database backend for client records.)

You can also declare that a group should be negated; this allows you
to set defaults and override them efficiently.  Negation is applied
after other group memberships are calculated, so it doesn't matter how
many times a client is assigned to a group or how many times it is
negated; a single group negation is sufficient to remove a client from
that group.  For instance, in the following example,
``foo.example.com`` is **not** a member of ``selinux-enabled``, even
though it is a member of the ``foo-server`` and ``every-server``
groups:

.. code-block:: xml

    <Groups>
      <Group name="foo-server">
        <Group name="apache-server"/>
        <Group name="selinux-enabled"/>
      </Group>
      <Group name="apache-server">
        <Group name="selinux-enabled"/>
      </Group>
      <Group name="every-server">
        <Group name="selinux-enabled"/>
      </Group>
      <Client name="foo.example.com">
        <Group name="selinux-enabled" negate="true"/>
      </Client>

.. note::

    Nested Group conditionals, Client tags, and negated Group tags are
    all new in 1.3.0.

Order of ``groups.xml`` does not matter.

Groups describe clients in terms for abstract, disjoint aspects. Groups
can be combined to form complex descriptions of clients that use
configuration commonality and inheritance. Groups have several attributes,
described below:


Metadata Groups Tag
-------------------

The Groups tag has the following possible attributes:

+----------+---------------------------------+--------+
| Name     | Description                     | Values |
+==========+=================================+========+
| version  | Group schema version            | String |
+----------+---------------------------------+--------+
| origin   | URL of master version           | String |
|          | (for common repository)         |        |
+----------+---------------------------------+--------+
| revision | Master version control revision | String |
+----------+---------------------------------+--------+

Metadata Group Tag
------------------

The Group Tag has the following possible attributes:

+----------+----------------------------------------------+--------------+
| Name     | Description                                  | Values       |
+==========+==============================================+==============+
| name     | Name of the group                            | String       |
+----------+----------------------------------------------+--------------+
| profile  | If a client can be directly associated with  | True|False   |
|          | this group                                   |              |
+----------+----------------------------------------------+--------------+
| public   | If a client can freely associate itself with | True|False   |
|          | this group. For use with the ``bcfg2 -p``    |              |
|          | option on the client.                        |              |
+----------+----------------------------------------------+--------------+
| category | A group can only contain one instance of a   | String       |
|          | group in any one category. This provides the |              |
|          | basis for representing groups which are      |              |
|          | conjugates of one another in a rigorous way. |              |
+----------+----------------------------------------------+--------------+
| default  | Set as the profile to use for clients that   | True|False   |
|          | are not associated with a profile in         |              |
|          | ``clients.xml``                              |              |
+----------+----------------------------------------------+--------------+
| comment  | English text description of group            | String       |
+----------+----------------------------------------------+--------------+
| negate   | When used as a conditional, only apply the   | True|False   |
|          | children if the named group does not match.  |              |
|          | When used as a declaration, do not apply     |              |
|          | the named group to matching clients.         |              |
+----------+----------------------------------------------+--------------+

The ``profile``, ``public``, ``category``, ``default``, and
``comment`` attributes are only parsed if a Group tag either a) is the
direct child of a Groups tag (i.e., at the top level of an XML file);
or b) has no children.  This matches legacy behavior in Bcfg2 1.2 and
earlier.

Groups can also contain other groups, clients, and bundles.

.. _metadata-client-tag:

Metadata Client Tag
-------------------

The Client Tag has the following possible attributes:

+----------+-----------------------------------------------+--------------+
| Name     | Description                                   | Values       |
+==========+===============================================+==============+
| name     | Name of the client                            | String       |
+----------+-----------------------------------------------+--------------+
| negate   | Only apply the child tags if the named client | True|False   |
|          | does not match.                               |              |
+----------+-----------------------------------------------+--------------+

Clients can also contain groups, other clients (although that's likely
nonsensical), and bundles.


Use of XInclude
===============

`XInclude <http://www.w3.org/TR/xinclude/>`_ is a W3C specification
for the inclusion of external XML documents into XML source
files. Much like the use of ``#include`` in C, this allows complex
definitions to be split into smaller, more manageable pieces. As of
bcfg2-0.9.0pre1, the `Metadata`_ plugin supports the use of XInclude
specifications to split the ``clients.xml`` and ``groups.xml``
files. This mechanism allows the following specification to produce
useful results:

.. code-block:: xml

    <Groups version='3.0' xmlns:xi="http://www.w3.org/2001/XInclude">
     <xi:include href="my-groups.xml" />
     <xi:include href="their-groups.xml" />
    </Groups>

Each of the included groups files has the same format. These files are
properly validated by ``bcfg2-lint``. This mechanism is useful for
composing group definitions from multiple sources, or setting
different permissions in an svn repository.

Probes
======

The metadata plugin includes client-side probing functionality. This
is fully documented :ref:`here <server-plugins-probes-index>`.

Metadata Caching
================

.. versionadded:: 1.3.0

Client metadata can be cached in order to improve performance.  This
is particularly important if you have lots of templates that use
metadata from other clients (e.g., with the MetadataQuery interface
described below.  See :ref:`server-caching` for a full description of
the caching features available.

.. _server-plugins-grouping-metadata-clientmetadata:


ClientMetadata
==============

A special client metadata class is available to the 
:ref:`TGenshi <server-plugins-generators-tgenshi-index>` and
:ref:`TCheetah <server-plugins-generators-tcheetah>` plugins.

+----------------------+------------------------------------------------+-------------------+
| Attribute            | Description                                    | Value             |
+======================+================================================+===================+
| hostname             | Client hostname                                | String            |
+----------------------+------------------------------------------------+-------------------+
| profile              | Client profile                                 | String            |
+----------------------+------------------------------------------------+-------------------+
| aliases              | Client aliases                                 | List              |
+----------------------+------------------------------------------------+-------------------+
| addresses            | Adresses this client is known by               | List              |
+----------------------+------------------------------------------------+-------------------+
| groups               | Groups this client is a member of              | List              |
+----------------------+------------------------------------------------+-------------------+
| categories           | Categories of this clients groups              | List              |
+----------------------+------------------------------------------------+-------------------+
| uuid                 | uuid identifier for this client                | String            |
+----------------------+------------------------------------------------+-------------------+
| password             | bcfg password for this client                  | String            |
+----------------------+------------------------------------------------+-------------------+
| connectors           | connector plugins known to this client         | List              |
+----------------------+------------------------------------------------+-------------------+
| query                | MetadataQuery object                           | MetadataQuery     |
+----------------------+------------------------------------------------+-------------------+

|

+------------------------------+------------------------------------------------+-------------------+
| Method                       | Description                                    | Value             |
+==============================+================================================+===================+
| inGroup(group)               | True if this client is a memnber of 'group'    | Boolean           |
+------------------------------+------------------------------------------------+-------------------+
| group_in_category(category)  | Returns the group in 'category' if the client  | String            |
|                              | is a member of 'category', otherwise ''        |                   |
+------------------------------+------------------------------------------------+-------------------+

MetadataQuery
-------------

This class provides query methods for the metadata of all clients
known to the Bcfg2 server.  Note that ``*by_groups()`` and
``*by_profiles()`` behave differently; for a client to be included in
the return value of a ``by_groups()`` method, it must be a member of
*all* groups listed in the argument; for a client to be included in
the return value of a ``by_profiles()`` method, it must have any group
listed as its profile group.

+------------------------------+------------------------------------------------+-------------------+
| Method                       | Description                                    | Value             |
+==============================+================================================+===================+
| by_name(client)              | Get ClientMetadata object for 'client'         | ClientMetadata    |
+------------------------------+------------------------------------------------+-------------------+
| by_groups(groups)            | Get ClientMetadata object for clients in all   | List of           |
|                              | listed groups                                  | ClientMetadata    |
+------------------------------+------------------------------------------------+-------------------+
| by_profiles(client)          | Get ClientMetadata objects for clients whose   | List of           |
|                              | profile matches any listed profile group       | ClientMetadata    |
+------------------------------+------------------------------------------------+-------------------+
| names_by_groups(groups)      | Get the names of all clients in all listed     | List of strings   |
|                              | groups                                         |                   |
+------------------------------+------------------------------------------------+-------------------+
| names_by_profiles(profiles)  | Get the names of clients whose profile matches | List of strings   |
|                              | any listed profile group                       |                   |
+------------------------------+------------------------------------------------+-------------------+
| all_clients()                | All known client hostnames                     | List of strings   |
+------------------------------+------------------------------------------------+-------------------+
| all_groups()                 | All known group names                          | List of strings   |
+------------------------------+------------------------------------------------+-------------------+
| all_groups_in_category(cat)  | The names of all groups in category 'cat'      | List of strings   |
+------------------------------+------------------------------------------------+-------------------+
| all()                        | Get ClientMetadata for all clients             | List of           |
|                              |                                                | ClientMetadata    |
+------------------------------+------------------------------------------------+-------------------+