path: root/doc/server/plugins/generators/cfg.txt

.. -*- mode: rst -*-

.. _server-plugins-generators-cfg:

===
Cfg
===

The Cfg plugin provides a repository to describe configuration file
contents for clients. In its simplest form, the Cfg repository is just a
directory tree modeled off of the directory tree on your client machines.

The Cfg Repository
==================

The Cfg plugin is enabled by including **Cfg** on the **plugins** line of
the **[server]** section of your Bcfg2 server config file. The repository
itself lives in ``/var/lib/bcfg2/Cfg``, assuming you are using the default
repository location of ``/var/lib/bcfg2``. The contents of this directory
are a series of directories corresponding to the real-life locations of
the files on your clients, starting at the root level. For example::

    % ls Cfg
    bin/  boot/  etc/  opt/  root/  usr/  var/

Specific config files go in like-named directories in this
heirarchy.  For example the password file, ``/etc/passwd``, goes
in ``Cfg/etc/passwd/passwd``, while the ssh pam module config file,
``/etc/pam.d/sshd``, goes in ``Cfg/etc/pam.d/sshd/sshd``. The reason for
the like-name directory is to allow multiple versions of each file to
exist, as described below. Note that these files are exact copies of what
will appear on the client machine (except when using genshi or cheetah
templating -- see below).

Group-Specific Files
====================

It is often the case that you want one version of a config file for
all of your machines except those in a particular group. For example,
``/etc/fstab`` should look alike on all of your desktop machines, but
should be different on your file servers. Bcfg2 can handle this case
through use of group-specific files.

As mentioned above, all Cfg entries live in like-named directories
at the end of their directory tree. In the case of fstab, the file at
``Cfg/etc/fstab/fstab`` will be handed out by default to any client that
asks for a copy of ``/etc/fstab``. Group-specific files are located in
the same directory and are named with the following syntax::

    /path/to/filename/filename.GNN_groupname

**NN** is a priority number where **00** is lowest and **99**
is highest, and **groupname** is the name of a group defined in
``Metadata/groups.xml``. Back to our fstab example, we might have a
``Cfg/etc/fstab/`` directory that looks like this::

    fstab
    fstab.G50_server
    fstab.G99_fileserver

By default, clients will receive the plain fstab file when they request
``/etc/fstab``. Any machine that is in the **server** group, however, will
instead receive the ``fstab.G50_server`` file. Finally, any machine that
is in the **fileserver** group will receive the ``fstab.G99_fileserver``
file, even if they are also in the **server** group.

Host-Specific Files
===================

Similar to the case with group-specific files, there are cases where
a specific machine should have a different version of a file than all
others. This can be accomplished with host-specific files. The format
of a host-specific file name is::

    /path/to/filename/filename.H_host.example.com

Host-specific files have a higher priority than group specific
files. Again, the fstab example::

    fstab
    fstab.G50_server
    fstab.G99_fileserver
    fstab.H_host.example.com

In this case, *host.example.com* will always get the host-specific
version, even if it is part of the **server** or **fileserver** (or both)
classes.

.. note::

    If you have the ability to choose between using a group-specific and a
    host-specific file, it is almost always best to use a group-specific
    one. That way if a hostname changes or an extra copy of a particular
    client is built, it will get the same changes as the original.

Templates
=========

.. _server-plugins-generators-cfg-genshi:

Genshi Templates
----------------

Genshi templates allow you to use the `Genshi
<http://genshi.edgewall.org>`_ templating system.  This is similar to
the deprecated :ref:`server-plugins-generators-tgenshi-index` plugin.
Genshi templates should be named with a ``.genshi`` extension, e.g.::

    % ls Cfg/etc/motd 
    info.xml  motd.genshi

See the genshi `documentation
<http://genshi.edgewall.org/wiki/Documentation>`_ for examples of
Genshi syntax.

Inside Genshi Templates
~~~~~~~~~~~~~~~~~~~~~~~

Several variables are pre-defined inside templates:

* **metadata** is the client's :ref:`metadata
  <server-plugins-grouping-metadata-clientmetadata>`
* **name** is the path name specified in Bcfg2
* **path** is the path to the TGenshi template.  It starts with a
    leading slash, and is relative to the Bcfg2 specification root.
    E.g., ``/Cfg/etc/foo.conf/foo.conf.genshi`` or
    ``/TGenshi/etc/foo.conf/template.newtxt.H_foo.example.com``

Troubleshooting
~~~~~~~~~~~~~~~

When developing a template, you can see what the template would
generate on a client with :ref:`bcfg2-info <server-bcfg2-info>`::

    bcfg2-info buildfile <path> <hostname>

E.g.::

    bcfg2-info buildfile /etc/foo.conf foo.example.com

To generate a file with an altsrc attribute, you can run::

    bcfg2-info buildfile /etc/foo/foo.conf --altsrc=/etc/foo.conf \
        foo.example.com

Sometimes, it's useful to be able to do more in-depth troubleshooting
by running the template manually. To do this, run ``bcfg2-info
debug``, and, once in the Python interpreter, run::

    metadata = self.build_metadata("<hostname>")
    path = "<relative path to template (see note below)>"
    
``path`` should be set to the path to the template file with a leading
slash, relative to the Bcfg2 specification root. See `Inside Genshi
Templates`_ for examples.

Then, run::
    
    import os, Bcfg2.Options
    from genshi.template import TemplateLoader, NewTextTemplate
    name = os.path.dirname(path[path.find('/', 1):])
    setup = Bcfg2.Options.OptionParser({'repo':
                                        Bcfg2.Options.SERVER_REPOSITORY})
    setup.parse('--')
    template = TemplateLoader().load(setup['repo'] + path, cls=NewTextTemplate)
    print template.generate(metadata=metadata, path=path, name=name).render()

This gives you more fine-grained control over how your template is
rendered.

You can also use this approach to render templates that depend on
:ref:`altsrc <server-plugins-structures-altsrc>` tags by setting
``path`` to the path to the template, and setting ``name`` to the path
to the file to be generated, e.g.::

    metadata = self.build_metadata("foo.example.com")
    path = "/Cfg/etc/sysconfig/network-scripts/ifcfg-template/ifcfg-template.genshi"
    name = "/etc/sysconfig/network-scripts/ifcfg-bond0"

Error handling
~~~~~~~~~~~~~~

Situations may arise where a templated file cannot be generated due to
missing or incomplete information.  A TemplateError can be raised to
force a bind failure and prevent sending an incomplete file to the
client.  For example, this template::

    {% python
        from genshi.template import TemplateError
        grp = None
        for g in metadata.groups:
            if g.startswith('ganglia-gmond-'):
                grp = g
                break
        else:
            raise TemplateError, "Missing group"
    %}\

will fail to bind if the client is not a member of a group starting with
"ganglia-gmond-".  The syslogs on the server will contain this message::

    bcfg2-server[5957]: Genshi template error: Missing group
    bcfg2-server[5957]: Failed to bind entry: Path /etc/ganglia/gmond.conf

...indicating the bind failure and message raised with the TemplateError.

Handling Dollar Signs
~~~~~~~~~~~~~~~~~~~~~

In a Genshi template, ``$`` is a special character and must be escaped
by doubling, i.e., ``$$``.  For instance, to embed the Subversion
``$Id$`` keyword in a Genshi template, you would have to do ``$$Id$$``.

Examples
~~~~~~~~

.. toctree::
   :glob:
   :maxdepth: 1

   examples/genshi/*

.. _server-plugins-generators-cfg-cheetah:

Cheetah Templates
-----------------

Cheetah templates allow you to use the `cheetah templating system
<http://www.cheetahtemplate.org/>`_.  This is similar to
the deprecated :ref:`server-plugins-generators-tcheetah` plugin.
Cheetah templates should be named with a ``.cheetah`` extension, e.g.::

    % ls Cfg/etc/motd 
    info.xml  motd.cheetah

Inside Cheetah Templates
~~~~~~~~~~~~~~~~~~~~~~~~

Several variables are pre-defined inside templates:

* **self.metadata** is the client's :ref:`metadata
  <server-plugins-grouping-metadata-clientmetadata>`
* **self.path** is the path name specified in Bcfg2
* **self.source_path** is the path to the Genshi template on the filesystem.

Examples
~~~~~~~~

.. toctree::
   :glob:
   :maxdepth: 1

   examples/cheetah/*

Comments and Cheetah
~~~~~~~~~~~~~~~~~~~~

As Cheetah processes your templates it will consider hash "#" style
comments to be actual comments in the template and will strip them
from the final config file.  If you would like to preserve the comment
in the final config file you need to escape the hash character '\#'
which will tell Cheetah (and Python) that you do in fact want the
comment to appear in the final config file.::

    # This is a comment in my template which will be stripped when it's processed through Cheetah
    \# This comment will appear in the generated config file.


Notes on Using Templates
------------------------

Templates can be host and group specific as well.  Deltas will not be
processed for any genshi or cheetah base file.

.. note::

    If you are using templating in combination with host-specific
    or group-specific files, you will need to ensure that the ``.genshi``
    or ``.cheetah`` extension is at the **end** of the filename. Using the
    examples from above for *host.example.com* and group *server* you would
    have the following (using genshi only)::

        Cfg/etc/fstab/fstab.H_host.example.com.genshi
        Cfg/etc/fstab/fstab.G50_server.genshi

Genshi templates take precence over cheetah templates.  For example, if
two files exist named

    Cfg/etc/fstab/fstab.genshi
    Cfg/etc/fstab/fstab.cheetah

the cheetah template is ignored.  But you can mix genshi and cheetah when
using different host-specific or group-specific files.  For example:

    Cfg/etc/fstab/fstab.H_host.example.com.genshi
    Cfg/etc/fstab/fstab.G50_server.cheetah

.. _server-plugins-generators-cfg-encryption:

Encrypted Files
===============

.. versionadded:: 1.3.0

Bcfg2 allows you to encrypt files stored in ``Cfg/`` to protect the
data in them from other people who need access to the repository.  See
also :ref:`server-plugins-connectors-properties-encryption` for
information on encrypting elements in Properties files, which is often
more friendly for tracking changes in a VCS.

.. note::

    This feature is *not* intended to secure the files against a
    malicious attacker who has gained access to your Bcfg2 server, as
    the encryption passphrases are held in plaintext in
    ``bcfg2.conf``.  This is only intended to make it easier to use a
    single Bcfg2 repository with multiple admins who should not
    necessarily have access to each other's sensitive data.

See :ref:`server-encryption` for more details on encryption in Bcfg2
in general.

Encrypting Files
----------------

An encrypted file should end with ``.crypt``, e.g.::

    Cfg/etc/foo.conf
    Cfg/etc/foo.conf/foo.conf.crypt
    Cfg/etc/foo.conf/foo.conf.G10_foo.crypt

Encrypted Genshi or Cheetah templates can have the extensions in
either order, e.g.::

    Cfg/etc/foo.conf/foo.conf.crypt.genshi
    Cfg/etc/foo.conf/foo.conf.G10_foo.genshi.crypt
    Cfg/etc/foo.conf/foo.conf.H_bar.example.com.crypt.cheetah

To encrypt or decrypt a file, use :ref:`bcfg2-crypt`.

Deltas
======

.. note::

    In Bcfg2 1.3 and newer, deltas are deprecated.  It is recommended
    that you use templates instead.  The
    :ref:`TemplateHelper plugin
    <server-plugins-connectors-templatehelper>` comes with an example
    helper that can be used to include other files easily, a subset of
    cat file functionality.  ``bcfg2-lint`` checks for deltas and
    warns about them.

Bcfg2 has finer grained control over how to deliver configuration
files to a host. Let's say we have a Group named file-server. Members
of this group need the exact same ``/etc/motd`` as all other hosts except
they need one line added. We could copy motd to ``motd.G01_file-server``,
add the one line to the Group specific version and be done with it,
but we're duplicating data in both files. What happens if we need to
update the motd? We'll need to remember to update both files then. Here's
where deltas come in. A delta is a small change to the base file. There
are two types of deltas: cats and diffs. The cat delta simply adds or
removes lines from the base file. The diff delta is more powerful since
it can take a unified diff and apply it to the base configuration file
to create the specialized file. Diff deltas should be used very sparingly.

Cat Files
---------

Continuing our example for cat files, we would first create a file named
``motd.G01_file-server.cat``. The .cat suffix designates that the file is
a diff. We would then edit that file and add the following line::

    +This is a file server

The **+** at the begining of the file tells Bcfg2 that the line should be
appended to end of the file. You can also start a line with **-** to tell
Bcfg2 to remove that exact line wherever it might be in the file. How do
we know what base file Bcfg2 will choose to use to apply a delta? The
same rules apply as before: Bcfg2 will choose the highest priority,
most specific file as the base and then apply deltas in the order of
most specific and then increasing in priority. What does this mean in
real life. Let's say our machine is a web server, mail server, and file
server and we have the following configuration files::

    motd
    motd.G01_web-server
    motd.G01_mail-server.cat
    motd.G02_file-server.cat
    motd.H_foo.example.com.cat

If our machine **isn't** *foo.example.com* then here's what would happen:

Bcfg2 would choose ``motd.G01_web-server`` as the base file. It is
the most specific base file for this host. Bcfg2 would apply the
``motd.G01_mail-server.cat`` delta to the ``motd.G01_web-server``
base file. It is the least specific delta. Bcfg2 would then apply the
``motd.G02_file-server.cat`` delta to the result of the delta before
it. If our machine **is** *foo.example.com* then here's what would happen:

Bcfg2 would choose ``motd.G01_web-server`` as the base file. It
is the most specific base file for this host. Bcfg2 would apply the
``motd.H_foo.example.com.cat`` delta to the ``motd.G01_web-server`` base
file. The reason the other deltas aren't applied to *foo.example.com*
is because a **.H_** delta is more specific than a **.G##_** delta. Bcfg2
applies all the deltas at the most specific level.

.. _server-plugins-generators-cfg-validation:

Content Validation
==================

To ensure that files with invalid content are not pushed out, you can
provide a content validation script that will be run against each
file.  Create a file called ``:test`` inside the directory for the
file you want to test.  For example::

  Cfg/etc/sudoers/:test

You can also create host- and group-specific validators::

  Cfg/etc/sudoers/:test.G80_foogroup
  Cfg/etc/sudoers/:test.H_bar.example.com

A validator script has the following attributes:

* It must be executable, or specify a valid bangpath;
* The entire content of the file is passed to the validator on
  stdin;
* The validator is not called with any flags or arguments;
* The validator must return 0 on success and non-zero on failure; and
* The validator must output a sensible error message on failure.

For ``sudoers``, a very simple validator is::

  #!/bin/sh
  visudo -cf -

This uses the ``visudo`` command's built-in validation.

If you wish to disable validation, this can be done with the following
setting in ``bcfg2.conf``::

  [cfg]
  validation=no

If you have a very large number of validators, you may wish to disable
validation by default to avoid slowing down the generation of
configurations on the server, and use ``bcfg2-test`` (for instance, as
a post-commit hook or as part of a code review process) to run
validation.  You can do this by setting ``validation=no`` in
``bcfg2.conf`` as described above, and then calling ``bcfg2-test``
with the ``--cfg-validation`` flag.

File permissions
================

File permissions for entries handled by Cfg are controlled via the use
of :ref:`server-info` files. Note that you **cannot** use both a
Permissions entry and a Path entry to handle the same file.