path: root/doc/server/plugins/connectors/properties.txt

.. -*- mode: rst -*-

.. _server-plugins-connectors-properties:

==========
Properties
==========

The Properties plugin is a connector plugin that adds information from
XML, JSON, and YAML files into client metadata instances.

Enabling Properties
===================

First, ``mkdir /var/lib/bcfg2/Properties``. Each property file goes in
this directory. Each will automatically be cached by the server, and
reread/reparsed upon changes. Add **Properties** to your ``plugins``
line in ``/etc/bcfg2.conf``.

Data Structures
===============

Properties adds a new dictionary to client metadata instances that maps
property file names to PropertyFile instances.

A property file can be one of three types:

* If the filename ends with ``.xml``, it will be parsed as XML and
  handled by :class:`Bcfg2.Server.Plugins.Properties.XMLPropertyFile`.
  See `XML Property Files`_ below.
* If the filename ends with ``.json`` and JSON libraries are installed
  (either ``json`` or ``simplejson``, although ``json`` is highly
  recommended), it will be parsed as `JSON <http://www.json.org/>`_
  and handled by
  :class:`Bcfg2.Server.Plugins.Properties.JSONPropertyFile`.  See
  `JSON Property Files`_ below.
* If the filename ends with ``.yaml`` or ``.yml`` and PyYAML is
  installed, it will be parsed as `YAML <http://www.yaml.org/>`_ and
  handled by
  :class:`Bcfg2.Server.Plugins.Properties.YAMLPropertyFile`.  See
  `YAML Property Files`_ below.

The XML interface is undoubtably the most powerful, as it natively
supports schemas to check the data validity, client- and
group-specific data, and data encryption.

Usage
=====

Common Interface
----------------

Different data types have different interfaces, but there are some
usage patterns common to all properties files.

Specific property files can be referred to in templates as
``metadata.Properties[<filename>]``.

The data in property files is accessible via different attributes:

+-----------+----------------+
| Data Type | Data Attribute |
+===========+================+
| XML       | ``xdata``      |
+-----------+----------------+
| JSON      | ``json``       |
+-----------+----------------+
| YAML      | ``yaml``       |
+-----------+----------------+

For instance, in a :ref:`Genshi template
<server-plugins-generators-cfg-genshi>`, you might do::

    {% for item in metadata.Properties['foo.json'].json %}\
    ${item}
    {% end %}\

    {% for key, value in metadata.Properties['foo.yml'].yaml %}\
    ${key} = ${value}
    {% end %}\

    {% for el in metadata.Properties['foo.xml'].xdata.findall("Tag") %}\
    ${el.get("name")} = ${el.text}
    {% end %}\

The raw contents of a properties file as a string are available via
the ``data`` attribute, e.g., ``metadata.Properties['prop-file'].data``.

.. _server-plugins-connectors-properties-write-back:

Writing to Properties files
~~~~~~~~~~~~~~~~~~~~~~~~~~~

.. versionadded:: 1.2.0

If you need to make persistent changes to properties data, you can use
the ``write`` method of the
:class:`Bcfg2.Server.Plugins.Properties.PropertyFile` class::

    {% python
    import lxml.etree
    from genshi.template import TemplateError
    lxml.etree.SubElement(metadata.Properties['foo.xml'],
                          "Client",
                          name=metadata.hostname)
    if not metadata.Properties['foo.xml'].write():
        raise TemplateError("Failed to write changes back to foo.xml")

The interface is the same for YAML or JSON data.

If writing XML data, the ``write`` method checks the data in the
object against its schema before writing it; see `Data Structures`_
for details.

Note that use of the ``write`` method can cause race conditions if you
run more than one Bcfg2 server.  If you run more than one Bcfg2
server, you can disable Properties write-back by setting the following
in ``bcfg2.conf``::

    [properties]
    writes_enabled = false

XML Property Files
------------------

The data in an XML property file can be accessed with the ``xdata``
attribute, an :class:`lxml.etree._Element` object documented `here
<http://codespeak.net/lxml/tutorial.html#the-element-class>`_.

In addition to the ``xdata`` attribute that can be used to access the
raw data, the following access methods are defined:

* ``Match()`` parses the Group and Client tags in the file and returns
  a list of elements that apply to the client described by a set of
  metadata.  For instance::

    {% python
    ntp_servers = [el.text
                   for el in metadata.Properties['ntp.xml'].Match(metadata)
                   if el.tag == "Server"]
    %}
* ``XMLMatch()`` parses the Group and Client tags in the file and
  returns an XML document containing only the data that applies to the
  client described by a set of metadata.  (The Group and Client tags
  themselves are also removed, leaving only the tags and data
  contained in them.)  For instance::

    {% python
    ntp_servers = [el.text
                   for el in metadata.Properties['ntp.xml'].XMLMatch(metadata).findall("//Server")]
    %}

  ``XMLMatch()`` can be run automatically on properties files by using
  the :ref:`server-plugins-connectors-properties-automatch` feature.

You can also access the XML data that comprises a property file
directly in one of several ways:

* ``metadata.Properties['prop-file'].xdata`` is an lxml.etree._Element
  object representing the top-level element in the file.
* ``metadata.Properties['prop-file'].data`` is the raw contents of the
  property file as a string.
* ``metadata.Properties['prop-file'].entries`` is a list of
  lxml.etree._Element objects representing the direct children of the
  top-level element.  (I.e., everything directly under the
  ``<Properties>`` tag.)

The XML data in a property file is arbitrary, but a matching ``.xsd``
file can be created to assign a schema to a property file, which will
be checked when running ``bcfg2-lint``.  For instance, given::

    Properties/dns-config.xml
    Properties/dns-config.xsd

``dns-config.xml`` will be validated against ``dns-config.xsd``.

Although Properties files are technically freeform XML, the top-level
XML tag should be ``<Properties>``.


JSON Property Files
-------------------

The data in a JSON property file can be accessed with the ``json``
attribute, which is the loaded JSON data.  The JSON properties
interface does not provide any additional functionality beyond the
`Common Interface`_.

YAML Property Files
-------------------

The data in a YAML property file can be accessed with the ``yaml``
attribute, which is the loaded YAML data.  Only a single YAML document
may be included in a file.

The YAML properties interface does not provide any additional
functionality beyond the `Common Interface`_.

.. _server-plugins-connectors-properties-automatch:

Automatch
=========

.. versionadded:: 1.3.0

You can enable
:func:`Bcfg2.Server.Plugin.helpers.StructFile.XMLMatch()` for all XML
Property files by setting ``automatch`` to ``true`` in the
``[properties]`` section of ``bcfg2.conf``.  This makes
``metadata.Properties`` values :class:`lxml.etree._Element` objects
that contain only matching data.  (This makes it impossible to do
:ref:`server-plugins-connectors-properties-write-back` as a
side-effect.)

In Python terms, setting ``automatch=true`` is the same as doing the
following at the top of each template::

    {% python
    for prop in metadata.Properties.values():
        prop = prop.XMLMatch(metadata)
    %}

The example above that describes ``XMLMatch()`` would then become
simply::

    {% python
    ntp_servers = [el.text
                   for el in metadata.Properties['ntp.xml'].findall("//Server")]
    %}

You can also enable automatch for individual Property files by setting
the attribute ``automatch="true"`` on the top-level ``<Property>``
tag.  Conversely, if automatch is enabled by default in
``bcfg2.conf``, you can disable it for an individual Property file by
setting ``automatch="false"`` on the top-level ``<Property>`` tag.

If you want to see what ``XMLMatch()``/automatch would produce for a
given client on a given Properties file, you can use :ref:`bcfg2-info
<server-bcfg2-info>`::

    bcfg2-info automatch props.xml foo.example.com

If automatch is not enabled, you can force ``bcfg2-info`` to perform
it anyway with ``-f``::

    bcfg2-info automatch -f props.xml foo.example.com

.. note::

    Be sure to notice that enabling automatch changes the type of the
    data in ``metadata.Properties``; with automatch disabled, the
    values of the ``metadata.Properties`` dict are
    :class:`Bcfg2.Server.Plugins.Properties.PropertyFile` objects.
    With automatch enabled, they are :class:`lxml.etree._Element`
    objects.

.. _server-plugins-connectors-properties-encryption:

Encrypted Properties data
=========================

.. versionadded:: 1.3.0

You can encrypt selected data in XML Properties files to protect that
data from other people who need access to the repository.  See
:ref:`server-encryption-configuration` for details on configuring
encryption passphrases.  The data is decrypted transparently
on-the-fly by the server; you never need to decrypt the data in your
templates.  Encryption is only supported on XML properties files.

.. note::

    This feature is *not* intended to secure the files against a
    malicious attacker who has gained access to your Bcfg2 server, as
    the encryption passphrases are held in plaintext in
    ``bcfg2.conf``.  This is only intended to make it easier to use a
    single Bcfg2 repository with multiple admins who should not
    necessarily have access to each other's sensitive data.

Properties files are encrypted on a per-element basis; that is, rather
than encrypting the whole file, only the character content of
individual elements is encrypted.  This makes it easier to track
changes to the file in a VCS, and also lets unprivileged users work
with the other data in the file.  Only character content of an element
can be encrypted; attribute content and XML elements themselves cannot
be encrypted.

By default, decryption is *strict*; that is, if any element cannot be
decrypted, parsing of the file is aborted.  If you wish for parsing to
continue, with unencryptable elements simply skipped, then you can set
decryption to *lax* in one of two ways:

* Set ``decrypt=lax`` in the ``[encryption]`` section of
  ``bcfg2.conf`` to set lax decryption on all files by default; or
* Set the ``decrypt="lax"`` attribute on the top-level ``Properties``
  tag of a Properties file to set lax decryption for a single file.

Note that you could, for instance, set lax decryption by default, and
then set strict decryption on individual files.

To encrypt or decrypt a file, use :ref:`bcfg2-crypt`.

See :ref:`server-encryption` for more details on encryption in Bcfg2
in general.


Accessing Properties contents from Genshi Templates
===================================================

Access contents of ``Properties/auth.xml``::

    ${metadata.Properties['auth.xml'].xdata.find('file').find('bcfg2.key').text}

Configuration
=============

``bcfg2.conf`` contains several miscellaneous configuration options
for the Properties plugin, which can be set in the ``[properties]``
section. Any booleans in the config file accept the values "1", "yes",
"true", and "on" for True, and "0", "no", "false", and "off" for
False.

It understands the following directives:

* ``automatch``: Enable
  :ref:`server-plugins-connectors-properties-automatch`.  Default is
  false.
* ``writes_enabled``: Enable
  :ref:`server-plugins-connectors-properties-write-back`.  Default is
  true.

Module Documentation
====================

.. automodule:: Bcfg2.Server.Plugins.Properties