Add the ability to specify important entries via info/info.xml

We currently have 'important' entries specified only inside the code for
various client tools. This provides no room for growing that list
outside of adding things to the code itself. With this change, users can
now specify important entries via an additional attribute in their
info.xml files.

Signed-off-by: Sol Jerome <sol.jerome@gmail.com>

git-svn-id: https://svn.mcs.anl.gov/repos/bcfg/trunk/bcfg2@5957 ce84e21b-d406-0410-9b95-82705330c041

13 files changed, 154 insertions, 78 deletions

diff --git a/doc/server/configurationentries.txt b/doc/server/configurationentries.txt
index ba727ee22..059eb1288 100644
--- a/doc/server/configurationentries.txt
+++ b/doc/server/configurationentries.txt
@@ -1,11 +1,14 @@
 .. -*- mode: rst -*-
 
-.. _Base: server-plugins-structures-base
-.. _Bundler: server-plugins-structures-bundler-index
-.. _Cfg: server-plugins-generators-cfg
-.. _TGenshi: server-plugins-generators-tgenshi-index
-.. _TCheetah: server-plugins-generators-tcheetah
-.. _Rules: server-plugins-generators-rules
+.. NOTE: these are relative links (change when directory structure
+..       changes)
+
+.. _Base: plugins/structures/base
+.. _Bundler: plugins/structures/bundler
+.. _Cfg: plugins/generators/cfg
+.. _TGenshi: plugins/generators/tgenshi
+.. _TCheetah: plugins/generators/tcheetah
+.. _Rules: plugins/generators/rules
 
 .. _server-configurationentries:
 
@@ -90,8 +93,7 @@ the *type* and any other necessary attributes in `Rules`_.
 +-------------+----------------------+-----------------+--------------------------+
 
 Keep in mind that permissions for files served up by Cfg/TGenshi/TCheetah
-are still handled via the traditional mechanisms (info/:info/info.xml
-files in the plugin directory).
+are still handled via the traditional :ref:`server-info` mechanisms.
 
 .. _boundentries:
 
diff --git a/doc/server/index.txt b/doc/server/index.txt
index 9c351c652..b0c97ccae 100644
--- a/doc/server/index.txt
+++ b/doc/server/index.txt
@@ -13,4 +13,5 @@ Bcfg2 Server
    admin/index
    reports/index
    configurationentries
+   info
    snapshots/index
diff --git a/doc/server/info.txt b/doc/server/info.txt
new file mode 100644
index 000000000..0adf8ce51
--- /dev/null
+++ b/doc/server/info.txt
@@ -0,0 +1,97 @@
+.. -*- mode: rst -*-
+
+.. NOTE: these are relative links (change when directory structure
+..       changes)
+
+.. _Cfg: plugins/generators/cfg
+.. _TGenshi: plugins/generators/tgenshi
+.. _TCheetah: plugins/generators/tcheetah
+
+.. _server-info:
+
+====
+Info
+====
+
+Various file properties for entries served by the `Cfg`_, `TGenshi`_,
+and `TCheetah`_ plugins are controlled through the use of ``:info``,
+``info``, or ``info.xml`` files.
+
+By default, these plugins are set to write files to the filesystem with
+owner **root**, group **root**, and mode **644** (read and write for
+owner, read only for group and other). These options, and a few others,
+can be overridden through use of ``:info`` or ``info`` files. Each config
+file directory can have a ``:info`` or ``info`` file if needed. The
+possible fields in an info file are:
+
++------------+-------------------+----------------------------------+---------+
+| Field      | Possible values   | Description                      | Default |
++============+===================+==================================+=========+
+| encoding:  | ascii | base64    | Encoding of the file. Use        | ascii   |
+|            |                   | base64 for non-ASCII files       |         |
++------------+-------------------+----------------------------------+---------+
+| group:     | Any valid group   | Sets group of the file           | root    |
++------------+-------------------+----------------------------------+---------+
+| important: | true | false      | Important entries are            | root    |
+|            |                   | installed first during client    |         |
+|            |                   | execution                        |         |
++------------+-------------------+----------------------------------+---------+
+| owner:     | Any valid user    | Sets owner of the file           | root    |
++------------+-------------------+----------------------------------+---------+
+| paranoid:  | yes | no          | Backup file before replacement?  | no      |
++------------+-------------------+----------------------------------+---------+
+| perms:     | Numeric file mode | Sets the permissions of the file | 0644    |
++------------+-------------------+----------------------------------+---------+
+
+A sample info file for CGI script on a web server might look like::
+
+    owner: www
+    group: www
+    perms: 0755
+
+Back to the fstab example again, our final ``Cfg/etc/fstab/`` directory
+might look like::
+
+    :info
+    fstab
+    fstab.G50_server
+    fstab.G99_fileserver
+    fstab.H_host.example.com
+
+Important attribute
+===================
+
+.. versionadded:: 1.1.0
+
+Having important entries hardcoded into the various client tools has
+worked relatively well so far. However, this method allows for a bit
+more flexibility as the entries can be controlled via the configuration
+specification.
+
++------------+-------------------+----------------------------------+---------+
+| Field      | Possible values   | Description                      | Default |
++============+===================+==================================+=========+
+| important: | true | false      | Important entries are            | root    |
+|            |                   | installed first during client    |         |
+|            |                   | execution                        |         |
++------------+-------------------+----------------------------------+---------+
+
+info.xml files
+==============
+
+``info.xml`` files add the ability to specify different sets of file
+metadata on a group by group basis. These files are XML, and work
+similarly to those used by :ref:`Rules <server-plugins-generators-rules>`
+or :ref:`Pkgmgr <server-plugins-generators-pkgmgr>`.
+
+The following specifies a different global set of permissions
+(root/sys/0651) than on clients in group webserver (root/root/0652)
+
+.. code-block:: xml
+
+    <FileInfo>
+      <Group name='webserver'>
+        <Info owner='root' group='root' perms='0652'/>
+      </Group>
+      <Info owner='root' group='sys' perms='0651'/>
+    </FileInfo>
diff --git a/doc/server/plugins/generators/cfg.txt b/doc/server/plugins/generators/cfg.txt
index a79b93178..0d837f419 100644
--- a/doc/server/plugins/generators/cfg.txt
+++ b/doc/server/plugins/generators/cfg.txt
@@ -149,61 +149,9 @@ file. The reason the other deltas aren't applied to *foo.example.com*
 is because a **.H_** delta is more specific than a **.G##_** delta. Bcfg2
 applies all the deltas at the most specific level.
 
-Info files
-==========
-
-By default, Cfg writes files to the filesystem with owner *root*, group
-*root*, and mode *644* (read and write for owner, read only for group
-and other). These options, and a few others, can be overridden through
-use of ``:info`` or ``info`` files. Each config file directory can have
-a ``:info`` or ``info`` file if needed. The possible fields in an info
-file are:
-
-+-----------+-------------------+------------------------------------------------------+---------+
-| Field     | Possible values   | Description                                          | Default |
-+===========+===================+======================================================+=========+
-| owner:    | Any valid user    | Sets owner of the file                               | root    |
-+-----------+-------------------+------------------------------------------------------+---------+
-| group:    | Any valid group   | Sets group of the file                               | root    |
-+-----------+-------------------+------------------------------------------------------+---------+
-| perms:    | Numeric file mode | Sets the permissions of the file                     | 0644    |
-+-----------+-------------------+------------------------------------------------------+---------+
-| encoding: | ascii | base64    | Encoding of the file. Use base64 for non-ASCII files | ascii   |
-+-----------+-------------------+------------------------------------------------------+---------+
-| paranoid: | yes | no          | Backup file before replacement?                      | no      |
-+-----------+-------------------+------------------------------------------------------+---------+
-
-A sample info file for CGI script on a web server might look like::
-
-    owner: www
-    group: www
-    perms: 0755
-
-Back to the fstab example again, our final ``Cfg/etc/fstab/`` directory
-might look like::
-
-    :info
-    fstab
-    fstab.G50_server
-    fstab.G99_fileserver
-    fstab.H_host.example.com
-
-info.xml files
-==============
-
-info.xml files add the ability to specify different sets of file metadata
-on a group by group basis. These files are XML, and work similarly
-to those used by :ref:`Rules <server-plugins-generators-rules>` or
-:ref:`Pkgmgr <server-plugins-generators-pkgmgr>`.
-
-The following specifies a different global set of permissions
-(root/sys/0651) than on clients in group webserver (root/root/0652)
-
-.. code-block:: xml
+File permissions
+================
 
-    <FileInfo>
-      <Group name='webserver'>
-        <Info owner='root' group='root' perms='0652'/>
-      </Group>
-      <Info owner='root' group='sys' perms='0651'/>
-    </FileInfo>
+File permissions for entries handled by Cfg are controlled via the use
+of :ref:`server-info` files. Note that you **cannot** use both a
+Permissions entry and a Path entry to handle the same file.
diff --git a/doc/server/plugins/generators/tcheetah.txt b/doc/server/plugins/generators/tcheetah.txt
index 2daa18740..b5c99f33a 100644
--- a/doc/server/plugins/generators/tcheetah.txt
+++ b/doc/server/plugins/generators/tcheetah.txt
@@ -34,6 +34,13 @@ The ``info`` file is formatted like ``:info`` files from Cfg.
 
 Mostly, people will want to use client metadata.
 
+File permissions
+================
+
+File permissions for entries handled by TCheetah are controlled via the
+use of :ref:`server-info` files. Note that you **cannot** use both a
+Permissions entry and a Path entry to handle the same file.
+
 self.metadata variables
 =======================
 
diff --git a/doc/server/plugins/generators/tgenshi/index.txt b/doc/server/plugins/generators/tgenshi/index.txt
index 71d0acfad..57440f6df 100644
--- a/doc/server/plugins/generators/tgenshi/index.txt
+++ b/doc/server/plugins/generators/tgenshi/index.txt
@@ -129,6 +129,13 @@ allocations map to sudo and root access.
     ${"\n".join(metadata.BB['users'][user])}
     {% end %}
 
+File permissions
+================
+
+File permissions for entries handled by TGenshi are controlled via the
+use of :ref:`server-info` files. Note that you **cannot** use both a
+Permissions entry and a Path entry to handle the same file.
+
 FAQs
 ====
 
diff --git a/schemas/info.xsd b/schemas/info.xsd
index 972b55c8d..4028f5c15 100644
--- a/schemas/info.xsd
+++ b/schemas/info.xsd
@@ -9,10 +9,11 @@
   <xsd:include schemaLocation="atom.xsd"/>
 
   <xsd:complexType name='InfoType'>
-    <xsd:attribute name='owner' type='xsd:string'/>
+    <xsd:attribute name='encoding' type='xsd:string'/>
     <xsd:attribute name='group' type='xsd:string'/>
+    <xsd:attribute name='important' type='xsd:string'/>
+    <xsd:attribute name='owner' type='xsd:string'/>
     <xsd:attribute name='perms' type='xsd:string'/>
-    <xsd:attribute name='encoding' type='xsd:string'/>
     <xsd:attribute name='paranoid' type='xsd:string'/>
   </xsd:complexType>
 
diff --git a/src/lib/Client/Tools/APT.py b/src/lib/Client/Tools/APT.py
index 8157d9e6c..1c035f956 100644
--- a/src/lib/Client/Tools/APT.py
+++ b/src/lib/Client/Tools/APT.py
@@ -47,7 +47,8 @@ class APT(Bcfg2.Client.Tools.Tool):
 
     def __init__(self, logger, setup, config):
         Bcfg2.Client.Tools.Tool.__init__(self, logger, setup, config)
-        self.__important__ = ["%s/cache/debconf/config.dat" % var_path,
+        self.__important__ = self.__important__ + \
+                             ["%s/cache/debconf/config.dat" % var_path,
                               "%s/cache/debconf/templates.dat" % var_path,
                               '/etc/passwd', '/etc/group',
                               '%s/apt/apt.conf' % etc_path,
diff --git a/src/lib/Client/Tools/Portage.py b/src/lib/Client/Tools/Portage.py
index dad9c4f3a..765e981fe 100644
--- a/src/lib/Client/Tools/Portage.py
+++ b/src/lib/Client/Tools/Portage.py
@@ -9,7 +9,6 @@ class Portage(Bcfg2.Client.Tools.PkgTool):
     the rest from Toolset.Toolset."""
     name = 'Portage'
     __execs__ = ['/usr/bin/emerge', '/usr/bin/equery']
-    __important__ = ['/etc/make.conf']
     __handles__ = [('Package', 'ebuild')]
     __req__ = {'Package': ['name', 'version']}
     pkgtype = 'ebuild'
@@ -18,6 +17,7 @@ class Portage(Bcfg2.Client.Tools.PkgTool):
 
     def __init__(self, logger, cfg, setup):
         Bcfg2.Client.Tools.PkgTool.__init__(self, logger, cfg, setup)
+        self.__important__ = self.__important__ + ['/etc/make.conf']
         self.cfg = cfg
         self.installed = {}
         self.RefreshPackages()
diff --git a/src/lib/Client/Tools/YUMng.py b/src/lib/Client/Tools/YUMng.py
index de5b221c1..a203b6961 100644
--- a/src/lib/Client/Tools/YUMng.py
+++ b/src/lib/Client/Tools/YUMng.py
@@ -75,7 +75,8 @@ class YUMng(Bcfg2.Client.Tools.RPMng.RPMng):
 
     def __init__(self, logger, setup, config):
         Bcfg2.Client.Tools.RPMng.RPMng.__init__(self, logger, setup, config)
-        self.__important__ = [entry.get('name') for struct in config \
+        self.__important__ = self.__important__ + \
+                             [entry.get('name') for struct in config \
                               for entry in struct \
                               if entry.tag in ['Path', 'ConfigFile'] and \
                               (entry.get('name').startswith('/etc/yum.d') \
diff --git a/src/lib/Client/Tools/__init__.py b/src/lib/Client/Tools/__init__.py
index 8ab815022..59f66612d 100644
--- a/src/lib/Client/Tools/__init__.py
+++ b/src/lib/Client/Tools/__init__.py
@@ -102,6 +102,9 @@ class Tool:
     __important__ = []
 
     def __init__(self, logger, setup, config):
+        self.__important__ = [entry.get('name') for struct in config for entry in struct \
+                              if entry.tag == 'Path' and \
+                              entry.get('important') in ['true', 'True']]
         self.setup = setup
         self.logger = logger
         if not hasattr(self, '__ireq__'):
diff --git a/src/lib/Options.py b/src/lib/Options.py
index abdfedc51..11dd41205 100644
--- a/src/lib/Options.py
+++ b/src/lib/Options.py
@@ -181,18 +181,21 @@ DEBUG = Option("Enable debugging output", False, cmd='-d')
 VERBOSE = Option("Enable verbose output", False, cmd='-v')
 DAEMON = Option("Daemonize process, storing pid", False,
                 cmd='-D', odesc="<pidfile>")
-MDATA_OWNER = Option('Default ConfigFile owner',
+MDATA_OWNER = Option('Default Path owner',
                      default='root', cf=('mdata', 'owner'),
                      odesc='owner permissions')
-MDATA_GROUP = Option('Default ConfigFile group',
+MDATA_GROUP = Option('Default Path group',
                      default='root', cf=('mdata', 'group'),
                      odesc='group permissions')
-MDATA_PERMS = Option('Default ConfigFile permissions',
+MDATA_IMPORTANT = Option('Default Path priority (importance)',
+                     default='False', cf=('mdata', 'important'),
+                     odesc='Important entries are installed first')
+MDATA_PERMS = Option('Default Path permissions',
                      '644', cf=('mdata', 'perms'),
                      odesc='octal permissions')
-MDATA_PARANOID = Option('Default ConfigFile paranoid setting',
+MDATA_PARANOID = Option('Default Path paranoid setting',
                      'false', cf=('mdata', 'paranoid'),
-                     odesc='ConfigFile paranoid setting')
+                     odesc='Path paranoid setting')
 
 SERVER_REPOSITORY = Option('Server repository path', '/var/lib/bcfg2',
                            cf=('server', 'repository'), cmd='-Q',
diff --git a/src/lib/Server/Plugin.py b/src/lib/Server/Plugin.py
index 765d908dc..2c2f0a21d 100644
--- a/src/lib/Server/Plugin.py
+++ b/src/lib/Server/Plugin.py
@@ -18,6 +18,7 @@ import Bcfg2.Options
 # grab default metadata info from bcfg2.conf
 opts = {'owner': Bcfg2.Options.MDATA_OWNER,
         'group': Bcfg2.Options.MDATA_GROUP,
+        'important': Bcfg2.Options.MDATA_IMPORTANT,
         'perms': Bcfg2.Options.MDATA_PERMS,
         'paranoid': Bcfg2.Options.MDATA_PARANOID}
 mdata_setup = Bcfg2.Options.OptionParser(opts)
@@ -29,9 +30,13 @@ logger = logging.getLogger('Bcfg2.Plugin')
 default_file_metadata = mdata_setup
 
 info_regex = re.compile( \
-    '^owner:(\s)*(?P<owner>\S+)|group:(\s)*(?P<group>\S+)|' +
-    'perms:(\s)*(?P<perms>\w+)|encoding:(\s)*(?P<encoding>\w+)|' +
-    'paranoid:(\s)*(?P<paranoid>\S+)|mtime:(\s)*(?P<mtime>\w+)$')
+    'encoding:(\s)*(?P<encoding>\w+)|' +
+    'group:(\s)*(?P<group>\S+)|' +
+    'important:(\s)*(?P<important>\S+)|' +
+    'mtime:(\s)*(?P<mtime>\w+)$' +
+    '^owner:(\s)*(?P<owner>\S+)|' +
+    'paranoid:(\s)*(?P<paranoid>\S+)|' +
+    'perms:(\s)*(?P<perms>\w+)|')
 
 class PluginInitError(Exception):
     """Error raised in cases of Plugin initialization errors."""