diff options
Diffstat (limited to 'doc')
31 files changed, 825 insertions, 428 deletions
diff --git a/doc/_templates/indexsidebar.html b/doc/_templates/indexsidebar.html new file mode 100644 index 000000000..39916315d --- /dev/null +++ b/doc/_templates/indexsidebar.html @@ -0,0 +1,11 @@ +<!-- FIXME: Add download page with pdf/html/txt archives of these documents + <h3>Download</h3> + <p><a href="{{ pathto('download') }}">Download these documents</a></p> +--> + + <h3>Docs for other versions</h3> + <ul> + <li><a href="http://docs.bcfg2.org/1.1/">Bcfg2 1.1 (stable)</a></li> + <li><a href="http://docs.bcfg2.org/1.2/">Bcfg2 1.2 (stable)</a></li> + <li><a href="http://docs.bcfg2.org/dev/">Bcfg2 development documentation</a></li> + </ul> diff --git a/doc/appendix/configuration/mrepo.txt b/doc/appendix/configuration/mrepo.txt index 309cd6779..c1bdb3fed 100644 --- a/doc/appendix/configuration/mrepo.txt +++ b/doc/appendix/configuration/mrepo.txt @@ -7,7 +7,7 @@ mrepo ===== -This section describes how to setup an `mrepo`_ mirror. +.. This section describes how to setup an `mrepo`_ mirror. `mrepo`_ builds a local APT/Yum RPM repository from local ISO files, downloaded updates, and extra packages from 3rd party repositories. It @@ -68,3 +68,27 @@ Update the repositories :: mrepo -ug + +Example sources.xml file +------------------------ + +.. code-block:: xml + + <Sources> + <!-- CentOS (5.4) sources --> + <YUMSource> + <Group>centos-5.4</Group> + <RawURL>http://mrepo/centos5-x86_64/RPMS.os</RawURL> + <Arch>x86_64</Arch> + </YUMSource> + <YUMSource> + <Group>centos-5.4</Group> + <RawURL>http://mrepo/centos5-x86_64/RPMS.updates</RawURL> + <Arch>x86_64</Arch> + </YUMSource> + <YUMSource> + <Group>centos-5.4</Group> + <RawURL>http://mrepo/centos5-x86_64/RPMS.extras</RawURL> + <Arch>x86_64</Arch> + </YUMSource> + </Sources> diff --git a/doc/appendix/contributors.txt b/doc/appendix/contributors.txt index ebc49d364..1afc62706 100644 --- a/doc/appendix/contributors.txt +++ b/doc/appendix/contributors.txt @@ -11,7 +11,7 @@ Contributors .. This is list is no longer in chronological order like the - AUTHORS file because it's easier to maintain (for me). + AUTHORS file because it's easier to maintain. Automatically sorted. In alphabetical order of the given name: @@ -22,6 +22,8 @@ In alphabetical order of the given name: which was used in the Bcfg2 client. - Chris Vuletich <vuletich@mcs.anl.gov> wrote some SSL code and the verification debugging code. +- Chris St. Pierre <stpierreca@ornl.gov> (re)wrote bcfg2-lint and has + made other miscellaneous contributions. - Cory Lueninghoener <cory@mcs.anl.gov> wrote the showentries function in ``bcfg2-info``. - Daniel Clark <dclark@pobox.com> created encap packages for bcfg2 and @@ -31,7 +33,7 @@ In alphabetical order of the given name: - David Strauss worked on CentOS, RHEL, Yum, and Bazaar VCS support. - Ed Smith <esmith4@inf.ed.ac.uk> has done substantial hardening of the Bcfg client and server and implemented a common logging infrastructure. -- Fabian Affolter <fabian@bernewireless.net> made some patches, added +- Fabian Affolter <mail@fabian-affolter.ch> made some patches, added some new features and plugins, and restructured the manual for Bcfg2. - Jack Neely <jjneely@ncsu.edu> worked on YUMng. - James Yang <jjyang@mcs.anl.gov> worked on ``bcfg2-admin`` and @@ -45,7 +47,7 @@ In alphabetical order of the given name: - Ken Raffenetti <raffenet@mcs.anl.gov> and Rick Bradshaw have written the Hostbase plugin. - Michael Jinks <mjinks@uchicago.edu> wrote the Gentoo tool drivers. -- Narayan Desai <desai@mcs.anl.gov> has written most of bcfg2, including all +- Narayan Desai <desai@mcs.anl.gov> has written most of Bcfg2, including all parts not explicitly mentioned in this file. - Patrick Ruckstuhl fixed bugs in the templating. - Pedro Flores made the Reporting system design help. diff --git a/doc/appendix/files/ntp.txt b/doc/appendix/files/ntp.txt index ec1fa3094..53b3347c8 100644 --- a/doc/appendix/files/ntp.txt +++ b/doc/appendix/files/ntp.txt @@ -26,7 +26,7 @@ a client, a profile group, a list of packages, and a base configuration. .. code-block:: xml <Clients version='3.0'> - <Client profile='fedora' pingable='N' pingtime='0' name='foo.bar.com'/> + <Client profile='fedora' name='foo.bar.com'/> </Clients> ``Metadata/groups.xml``: diff --git a/doc/appendix/guides/centos.txt b/doc/appendix/guides/centos.txt index 2bb3edc2b..50334ccbc 100644 --- a/doc/appendix/guides/centos.txt +++ b/doc/appendix/guides/centos.txt @@ -17,7 +17,7 @@ Install Bcfg2 The fastest way to get Bcfg2 onto your system is to use Yum or your preferred package management tool. We'll be using the ones that are distributed through EPEL_, but depending on your aversion -to risk you could download an RPM from other places as well. See +to risk you could download an RPM from other places as well. See :ref:`getting_started-using_bcfg2-with-centos` for information about building Bcfg2 from source and making your own packages. @@ -102,20 +102,20 @@ Run bcfg2 to be sure you are able to communicate with the server:: Excluding Packages in global exclude list Finished Loaded tool drivers: - Action Chkconfig POSIX YUMng + Action Chkconfig POSIX YUMng Phase: initial - Correct entries: 0 - Incorrect entries: 0 - Total managed entries: 0 - Unmanaged entries: 208 + Correct entries: 0 + Incorrect entries: 0 + Total managed entries: 0 + Unmanaged entries: 208 Phase: final - Correct entries: 0 - Incorrect entries: 0 - Total managed entries: 0 - Unmanaged entries: 208 + Correct entries: 0 + Incorrect entries: 0 + Total managed entries: 0 + Unmanaged entries: 208 No ca is specified. Cannot authenticate the server with SSL. @@ -147,20 +147,20 @@ Now if you run the client, no more warning:: Excluding Packages in global exclude list Finished Loaded tool drivers: - Action Chkconfig POSIX YUMng + Action Chkconfig POSIX YUMng Phase: initial - Correct entries: 0 - Incorrect entries: 0 - Total managed entries: 0 - Unmanaged entries: 208 + Correct entries: 0 + Incorrect entries: 0 + Total managed entries: 0 + Unmanaged entries: 208 Phase: final - Correct entries: 0 - Incorrect entries: 0 - Total managed entries: 0 - Unmanaged entries: 208 + Correct entries: 0 + Incorrect entries: 0 + Total managed entries: 0 + Unmanaged entries: 208 Bring your first machine under Bcfg2 control ============================================ @@ -173,7 +173,7 @@ Setup the :ref:`server-plugins-generators-packages` plugin ---------------------------------------------------------- First, replace **Pkgmgr** with **Packages** in the plugins -line of ``bcfg2.conf``. Then create Packages layout (as per +line of ``bcfg2.conf``. Then create Packages layout (as per :ref:`packages-exampleusage`) in ``/var/lib/bcfg2`` .. note:: I am using the RawURL syntax here since we are using `mrepo`_ @@ -291,30 +291,30 @@ Now if we run the client, we can see what this has done for us.:: Excluding Packages in global exclude list Finished Loaded tool drivers: - Action Chkconfig POSIX YUMng + Action Chkconfig POSIX YUMng Package pam failed verification. Phase: initial - Correct entries: 94 - Incorrect entries: 1 - Total managed entries: 95 - Unmanaged entries: 113 + Correct entries: 94 + Incorrect entries: 1 + Total managed entries: 95 + Unmanaged entries: 113 In dryrun mode: suppressing entry installation for: Package:pam Phase: final - Correct entries: 94 - Incorrect entries: 1 + Correct entries: 94 + Incorrect entries: 1 Package:pam - Total managed entries: 95 - Unmanaged entries: 113 + Total managed entries: 95 + Unmanaged entries: 113 Interesting, our **pam** package failed verification. What does this mean? Let's have a look:: [root@centos ~]# rpm --verify pam - ....L... c /etc/pam.d/system-auth + ....L... c /etc/pam.d/system-auth Sigh, it looks like the default RPM install for pam fails to verify using its own verification process (trust me, it's not the only one). At @@ -336,23 +336,23 @@ entries?:: Excluding Packages in global exclude list Finished Loaded tool drivers: - Action Chkconfig POSIX YUMng + Action Chkconfig POSIX YUMng Extra Package openssh-clients 4.3p2-36.el5_4.4.x86_64. Extra Package libuser 0.54.7-2.1el5_4.1.x86_64. ... Phase: initial - Correct entries: 95 - Incorrect entries: 0 - Total managed entries: 95 - Unmanaged entries: 113 + Correct entries: 95 + Incorrect entries: 0 + Total managed entries: 95 + Unmanaged entries: 113 Phase: final - Correct entries: 95 - Incorrect entries: 0 - Total managed entries: 95 - Unmanaged entries: 113 + Correct entries: 95 + Incorrect entries: 0 + Total managed entries: 95 + Unmanaged entries: 113 Package:at Package:avahi Package:avahi-compat-libdns_sd @@ -394,22 +394,22 @@ package:: Excluding Packages in global exclude list Finished Loaded tool drivers: - Action Chkconfig POSIX YUMng + Action Chkconfig POSIX YUMng Extra Package gpg-pubkey e8562897-459f07a4.None. Extra Package gpg-pubkey 217521f6-45e8a532.None. Phase: initial - Correct entries: 187 - Incorrect entries: 0 - Total managed entries: 187 - Unmanaged entries: 16 + Correct entries: 187 + Incorrect entries: 0 + Total managed entries: 187 + Unmanaged entries: 16 Phase: final - Correct entries: 187 - Incorrect entries: 0 - Total managed entries: 187 - Unmanaged entries: 16 + Correct entries: 187 + Incorrect entries: 0 + Total managed entries: 187 + Unmanaged entries: 16 Package:gpg-pubkey Service:atd Service:avahi-daemon @@ -562,20 +562,20 @@ Now we run the client and see there are no more unmanaged entries!:: Excluding Packages in global exclude list Finished Loaded tool drivers: - Action Chkconfig POSIX YUMng + Action Chkconfig POSIX YUMng Phase: initial - Correct entries: 205 - Incorrect entries: 0 - Total managed entries: 205 - Unmanaged entries: 0 + Correct entries: 205 + Incorrect entries: 0 + Total managed entries: 205 + Unmanaged entries: 0 Phase: final - Correct entries: 205 - Incorrect entries: 0 - Total managed entries: 205 - Unmanaged entries: 0 + Correct entries: 205 + Incorrect entries: 0 + Total managed entries: 205 + Unmanaged entries: 0 .. warning:: diff --git a/doc/appendix/guides/gentoo.txt b/doc/appendix/guides/gentoo.txt index d635e310b..da4acef19 100644 --- a/doc/appendix/guides/gentoo.txt +++ b/doc/appendix/guides/gentoo.txt @@ -16,28 +16,38 @@ let the list know if you find errors or omissions. Installing Bcfg2 ================ -Early in July 2008, Bcfg2 was added to the Gentoo portage tree. So far -it's only keyworded for ~x86, but we hope to see it soon in the amd64 and -x64-solaris ports. If you're using Gentoo on some other architecture, it -should still work provided that you have a reasonably up to date Python; -try adding `app-admin/bcfg2 ~*` to your `/etc/portage/package.keywords` -file. +Early in July 2008, Bcfg2 was added to the Gentoo portage tree. If you don't use portage to install Bcfg2, you'll want to make sure you have all the prerequisites installed first. For a server, you'll need: -* ``app-admin/gamin`` or ``app-admin/fam`` +* ``dev-libs/libgamin[python]`` * ``dev-python/lxml`` Clients will need at least: * ``app-portage/gentoolkit`` +Portage installs from source +============================ + +.. versionadded:: 1.3.0 + +By default the client will run with the ``--gitbinpkgonly`` option. If +you want your client to install packages from source (rather than +having a binary build host as seen below), you can set the following in +``/etc/bcfg2.conf``.:: + + [Portage] + binpkgonly = false + Package Repository ================== +.. note: This is only necessary for using binary packages. + You’ll need (to make) at least one archive of binary packages. The -Portage driver calls ``emerge`` with the ``-getbinpkgonly`` option. See +Portage driver calls ``emerge`` with the ``--getbinpkgonly`` option. See :manpage:`make.conf(5)` and :manpage:`emerge(1)` manpages, specifically the :envvar:`PORTAGE_BINHOST` environment variable. @@ -109,60 +119,17 @@ Configuring Client Machines Set up ``/etc/bcfg2.conf`` the way you would for any other Bcfg2 client. In ``make.conf``, set *PORTAGE_BINHOST* to point to the URI of -your package repository. You may want to create versions of +your package repository. You may want to create versions of ``make.conf`` for each package repository you maintain, with appropriate *PORTAGE_BINHOST* URI's in each, and associated with that package archive's group under ``Cfg`` -- for example, we have -``Cfg/etc/make.conf/make.conf.G99_gentoo-200701-vmware``. If a client +``Cfg/etc/make.conf/make.conf.G99_gentoo-200701-vmware``. If a client host switches groups, and the new group needs a different set of packages, everything should just fall into place. Pitfalls ======== -Package Verification Issues ---------------------------- - -As of this writing (2007/01/31), we're aware of a number of packages -marked stable in the Gentoo x86 tree which, for one reason or another, -consistently fail to verify cleanly under ``equery check``. In some cases -(pam, openldap), files which don't (ever) exist on the system are -nonetheless recorded in the package database; in some (python, Bcfg2, -ahem), whole classes of files (.pyc and .pyo files) consistently fail -their md5sum checks; and in others, the problem appears to be a -discrepancy in the way that symlinks are created vs. the way they're -recorded in the database. For example, in the OpenSSH package, -/usr/bin/slogin is a symlink to ./ssh, but equery expects it to point to -an unadorned ssh. An analogous situation exists with their manpages, -leading to noise like this:: - - # equery check openssh - [ Checking net-misc/openssh-4.5_p1 ] - !!! /etc/ssh/sshd_config has incorrect md5sum - !!! /usr/bin/slogin does not point to ssh - !!! /usr/share/man/man1/slogin.1.gz does not point to ssh.1.gz - !!! /etc/ssh/ssh_config has incorrect md5sum - * 62 out of 66 files good - -We can ignore the lines for ``ssh_config`` and ``sshd_config``; those will -be caught by Bcfg2 as registered config files and handled appropriately. - -Because Bcfg2 relies on the client system's native package reporting -tool to judge the state of installed packages, complaints like these -about trivial or intractable verification failures can trigger unnecessary -bundle reinstalls when the Bcfg2 client runs. Bcfg2 will catch on after a -pass or two that the situation isn't getting any better with repeated -package installs, stop trying, and list those packages as "bad" in -the client system's statistics. - -Aside from filing bugs with the Gentoo package maintainers, your narrator -has been unable to come up with a good approach to this. Maybe write a -series of ``Rules`` definitions according to what the package database -thinks it should find, and/or stage copies of affected files under -``Cfg``, and associate those rules and files with the affected package in -a bundle? Annoying but possibly necessary if you want your stats file -to look good. - /boot ----- diff --git a/doc/appendix/guides/nat_howto.txt b/doc/appendix/guides/nat_howto.txt index 818d3e644..5bd3f7b13 100644 --- a/doc/appendix/guides/nat_howto.txt +++ b/doc/appendix/guides/nat_howto.txt @@ -43,14 +43,14 @@ the Client entry in clients.xml will look something like this: .. code-block:: xml - <Client profile="desktop" name="test1" pingable="N" + <Client profile="desktop" name="test1" uuid='9001ec29-1531-4b16-8198-a71bea093d0a' location='floating'/> Alternatively, the Client entry can be setup like this: .. code-block:: xml - <Client profile="desktop" name="test1" pingable="N" + <Client profile="desktop" name="test1" uuid='9001ec29-1531-4b16-8198-a71bea093d0a' address='ip-address-of-NAT'/> The difference between these definitions is explained in detail in the diff --git a/doc/appendix/guides/using-bcfg2-with-centos.txt b/doc/appendix/guides/using-bcfg2-with-centos.txt index cd00e5e79..6a48fc5da 100644 --- a/doc/appendix/guides/using-bcfg2-with-centos.txt +++ b/doc/appendix/guides/using-bcfg2-with-centos.txt @@ -23,11 +23,13 @@ While you can go about building all these things from source, this how to will try and meet the dependencies using packages from EPEL_ or RPMforge_. The *el5* package should be compatible with CentOS 5.x. -EPEL_:: +EPEL_ for 5.x :: + [root@centos ~]# rpm -Uvh http://download.fedora.redhat.com/pub/epel/5/i386/epel-release-5-4.noarch.rpm - [root@centos ~]# rpm -Uvh http://download.fedora.redhat.com/pub/epel/5/i386/epel-release-5-3.noarch.rpm +EPEL_ for 6.x :: + [root@centos ~]# rpm -Uvh http://download.fedoraproject.org/pub/epel/6/i386/epel-release-6-5.noarch.rpm -RPMforge_:: +RPMforge_ for 5.x :: [root@centos ~]# rpm -Uvh http://dag.wieers.com/rpm/packages/rpmforge-release/rpmforge-release-0.3.6-1.el5.rf.x86_64.rpm diff --git a/doc/client/modes.txt b/doc/client/modes.txt index 639387859..4cdc54960 100644 --- a/doc/client/modes.txt +++ b/doc/client/modes.txt @@ -33,7 +33,7 @@ How do I use it? ---------------- #. In the Bcfg2 repository, put `paranoid='true'` in the ``info.xml`` - file. + file (this is the default setting). #. On the client, create ``/var/cache/bcfg2`` (or specify an alternate path in the [paranoid] section of ``/etc/bcfg2.conf``). #. On the client, run `bcfg2` with the `-P` option (alternatively, you @@ -71,9 +71,7 @@ accomplish this by adding a global metadata override in your [mdata] paranoid=true -This will override the default value of "paranoid=false" and change it to -"true" which will cause every file you add or update in your Bcfg2 repo -to backup on the client as specified in your client config. +.. note:: This is the default setting. Overall client service mode =========================== diff --git a/doc/conf.py b/doc/conf.py index 38160715d..5903b009a 100644 --- a/doc/conf.py +++ b/doc/conf.py @@ -104,7 +104,9 @@ html_theme = 'default' # Theme options are theme-specific and customize the look and feel of a theme # further. For a list of options available for each theme, see the # documentation. -#html_theme_options = {} +html_theme_options = { + "collapsiblesidebar": "true" +} # Add any paths that contain custom themes here, relative to this directory. #html_theme_path = [] @@ -139,7 +141,9 @@ html_last_updated_fmt = '%b %d, %Y' #html_use_smartypants = True # Custom sidebar templates, maps document names to template names. -#html_sidebars = {} +html_sidebars = { + 'index': 'indexsidebar.html' +} # Additional templates that should be rendered to pages, maps page names to # template names. diff --git a/doc/development/index.txt b/doc/development/index.txt index 352000dc8..2a54bfad8 100644 --- a/doc/development/index.txt +++ b/doc/development/index.txt @@ -39,3 +39,4 @@ git access. Mail the :ref:`help-mailinglist` for details. testing documentation docstyleguide + unit-testing diff --git a/doc/development/plugins.txt b/doc/development/plugins.txt index 15b512365..b2b70f553 100644 --- a/doc/development/plugins.txt +++ b/doc/development/plugins.txt @@ -164,7 +164,6 @@ Example Connector Bcfg2.Server.Plugin.Connector): '''The Foo plugin is here to illustrate a barebones connector''' name = 'Foo' - version = '$Revision: $' experimental = True def __init__(self, core, datastore): @@ -195,13 +194,10 @@ do so. We will call our new plugin `MyMetadata`. .. code-block:: python - __revision__ = '$Revision$' - import Bcfg2.Server.Plugins.Metadata class MyMetadata(Bcfg2.Server.Plugins.Metadata.Metadata): '''This class contains data for bcfg2 server metadata''' - __version__ = '$Id$' __author__ = 'bcfg-dev@mcs.anl.gov' def __init__(self, core, datastore, watch_clients=True): diff --git a/doc/development/setup.txt b/doc/development/setup.txt index e9fc6e1e5..b04bce3fe 100644 --- a/doc/development/setup.txt +++ b/doc/development/setup.txt @@ -12,13 +12,8 @@ Checking Out a Copy of the Code git clone git://git.mcs.anl.gov/bcfg2.git -* Create link to :file:`src/lib`:: - - cd bcfg2 - ln -s src/lib Bcfg2 - * Add :file:`bcfg2/src/sbin` to your :envvar:`PATH` environment variable -* Add :file:`bcfg2` to your :envvar:`PYTHONPATH` environment variable +* Add :file:`bcfg2/src/lib` to your :envvar:`PYTHONPATH` environment variable Using a Virtual Environment for Development diff --git a/doc/development/unit-testing.txt b/doc/development/unit-testing.txt new file mode 100644 index 000000000..30217dcc5 --- /dev/null +++ b/doc/development/unit-testing.txt @@ -0,0 +1,25 @@ +.. -*- mode: rst -*- + +.. _development-unit-testing: + +================== +Bcfg2 unit testing +================== + +.. _Python Mock Module: http://python-mock.sourceforge.net/ +.. _Python Nose: http://readthedocs.org/docs/nose/en/latest/ + +You will first need to install the `Python Mock Module`_ and `Python +Nose`_ modules. You can then run the existing tests with the +following.:: + + cd testsuite + nosetests + +You should see output something like the following:: + + .................................................. + ---------------------------------------------------------------------- + Ran 50 tests in 0.121s + + OK diff --git a/doc/getting_started/index.txt b/doc/getting_started/index.txt index e7075c33f..034584014 100644 --- a/doc/getting_started/index.txt +++ b/doc/getting_started/index.txt @@ -236,7 +236,7 @@ Next Steps Several other utilities can help from this point on: :ref:`bcfg2-info <server-bcfg2-info>` is a utility that -instantiates a copy of the bcfg2 server core (minus the networking code) +instantiates a copy of the Bcfg2 server core (minus the networking code) for examination. From this, you can directly query: * Client Metadata diff --git a/doc/installation/distributions.txt b/doc/installation/distributions.txt index 383943e9f..3dcfd7721 100644 --- a/doc/installation/distributions.txt +++ b/doc/installation/distributions.txt @@ -10,6 +10,17 @@ The installation of Bcfg2 on a specific distribution depends on the package management tool and the availability of the package in the distribution's repository. +Alpine Linux +============ + +Packages for `Alpine Linux`_ are available in the `testing`_ repository. +Just use `apk` to perform the installation :: + + apk add bcfg2 bcfg2-server -U -X http://dl-3.alpinelinux.org/alpine/edge/testing/ --allow-untrusted + +.. _Alpine Linux: http://www.alpinelinux.org/ +.. _testing: http://git.alpinelinux.org/cgit/aports/tree/testing/bcfg2 + ArchLinux ========= @@ -101,13 +112,15 @@ RHEL / Centos / Scientific Linux While you can go about building all these things from source, this section will try and meet the dependencies using packages from EPEL_ -[#f1]_. The *el5* and the soon available *el6* package should be compatible +[#f1]_. The *el5* and the *el6* package should be compatible with `CentOS`_ 5.x/6.x and `Scientific Linux`_. -EPEL_:: - +EPEL_ for 5.x :: [root@centos ~]# rpm -Uvh http://download.fedora.redhat.com/pub/epel/5/i386/epel-release-5-4.noarch.rpm +EPEL_ for 6.x :: + [root@centos ~]# rpm -Uvh http://download.fedoraproject.org/pub/epel/6/i386/epel-release-6-5.noarch.rpm + Install the bcfg2-server and bcfg2 RPMs:: [root@centos ~]# yum install bcfg2-server bcfg2 diff --git a/doc/server/admin/bundle.txt b/doc/server/admin/bundle.txt index 101666bb9..e9cb79781 100644 --- a/doc/server/admin/bundle.txt +++ b/doc/server/admin/bundle.txt @@ -8,13 +8,17 @@ bundle For a list of all available xml bundles use ``list-xml``. ``list-genshi`` will list all available genshi bundles.:: - # bcfg2-admin bundle list-xml - # bcfg2-admin bundle list-genshi +.. code-block:: sh + + # bcfg2-admin bundles list-xml + # bcfg2-admin bundles list-genshi ``show`` provides an interactive dialog to get details about the available bundles.:: - # bcfg2-admin bundle show +.. code-block:: sh + + # bcfg2-admin bundles show Available bundles (Number of bundles: 4) ---------------------------------------- [0] motd.xml diff --git a/doc/server/info.txt b/doc/server/info.txt index 3ee12c54d..d949aab68 100644 --- a/doc/server/info.txt +++ b/doc/server/info.txt @@ -37,7 +37,7 @@ possible fields in an info file are: +------------+-------------------+----------------------------------+---------+ | owner: | Any valid user | Sets owner of the file | root | +------------+-------------------+----------------------------------+---------+ -| paranoid: | true | false | Backup file before replacement? | false | +| paranoid: | true | false | Backup file before replacement? | true | +------------+-------------------+----------------------------------+---------+ | perms: | Numeric file mode | Sets the permissions of the file | 0644 | | | | 'inherit' | (or inherits from the files on | | diff --git a/doc/server/plugins/connectors/properties.txt b/doc/server/plugins/connectors/properties.txt index 7695e902c..ca0e9cf63 100644 --- a/doc/server/plugins/connectors/properties.txt +++ b/doc/server/plugins/connectors/properties.txt @@ -33,6 +33,9 @@ be checked when running ``bcfg2-lint``. For instance, given:: ``dns-config.xml`` will be validated against ``dns-config.xsd``. +Although Properties files are technically freeform XML, the top-level +XML tag should be ``<Properties>``. + Usage ===== @@ -83,6 +86,75 @@ directly in one of several ways: top-level element. (I.e., everything directly under the ``<Properties>`` tag.) +.. _server-plugins-connectors-properties-encrypted: + +Encrypted Properties data +========================= + +.. versionadded:: 1.3.0 + +You can encrypt selected data in Properties files to protect that data +from other people who need access to the repository. See +:ref:`server-plugins-generators-cfg-configuring-encryption` for +details on configuring encryption passphrases. The data is decrypted +transparently on-the-fly by the server; you never need to decrypt the +data in your templates. + +.. note:: + + This feature is *not* intended to secure the files against a + malicious attacker who has gained access to your Bcfg2 server, as + the encryption passphrases are held in plaintext in + ``bcfg2.conf``. This is only intended to make it easier to use a + single Bcfg2 repository with multiple admins who should not + necessarily have access to each other's sensitive data. + +Properties files are encrypted on a per-element basis; that is, rather +than encrypting the whole file, only the character content of +individual elements is encrypted. This makes it easier to track +changes to the file in a VCS, and also lets unprivileged users work +with the other data in the file. Only character content of an element +can be encrypted; attribute content and XML elements themselves cannot +be encrypted. + +To encrypt a file, use ``bcfg2-crypt``, e.g.:: + + bcfg2-crypt foo.xml + +If the top-level tag of a Properties file is not ``<Properties>``, +then you need to use the ``--properties`` flag to ``bcfg2-crypt``:: + + bcfg2-crypt --properties foo.xml + +The first time you run ``bcfg2-crypt`` on a Properties file, it will +encrypt all character data of all elements. Additionally, it will add +``encrypted="<key name>"`` to each element that has encrypted character +data. It also adds ``encryption="true"`` to the top-level +``<Properties>`` tag as a flag to the server that it should try to +decrypt the data in that file. (If you are using Properties schemas, +you will need to make sure to add support for these attributes.) On +subsequent runs, only those elements flagged with ``encrypted="*"`` +are encrypted or decrypted. + +To decrypt a Properties file, simply re-run ``bcfg2-crypt``:: + + bcfg2-crypt foo.xml + +This decrypts the encrypted elements, but it does *not* remove the +``encrypted`` attribute; this way, you can decrypt a Properties +file, modify the contents, and then simply re-run ``bcfg2-crypt`` to +encrypt it again. If you added elements that you also want to be +encrypted, you can either add the ``encrypted`` attribute to +them manually, or run:: + + bcfg2-crypt --xpath '*' foo.xml + +You can also use the ``--xpath`` option to specify more restrictive +XPath expressions to only encrypt a subset of elements, or to encrypt +different elements with different passphrases. Alternatively, you can +manally set the ``encrypted`` attribute on various elements and +``bcfg2-crypt`` will automatically do the right thing. + Accessing Properties contents from TGenshi ========================================== diff --git a/doc/server/plugins/connectors/puppetenc.txt b/doc/server/plugins/connectors/puppetenc.txt new file mode 100644 index 000000000..dc472c546 --- /dev/null +++ b/doc/server/plugins/connectors/puppetenc.txt @@ -0,0 +1,123 @@ +.. -*- mode: rst -*- + +.. _server-plugins-connectors-puppetenc: + +========= +PuppetENC +========= + +PuppetENC is a connector plugin that adds support for Puppet External +Node Classifiers +(`<http://docs.puppetlabs.com/guides/external_nodes.html>`_), or ENCs. + +Output Format +============= + +The PuppetENC plugin implements the Puppet 2.6.5+ ENC output format +with some modifications. The basic output format is described `here +<http://docs.puppetlabs.com/guides/external_nodes.html#puppet-265-and-higher>`_. +The following modifications apply: + +* ``classes`` are considered to be Bcfg2 groups. (This is basically + just a difference in terminology between Puppet and Bcfg2; Bcfg2 + calls "groups" what Puppet calls "classes.") +* As an alternative to the Puppet-specific ``classes`` value, you may + use ``groups`` if you are writing an ENC from scratch specifically + for Bcfg2. +* Since Bcfg2 does not have the notion of parameterized classes, any + class parameters provided will be merged in with the ``parameters`` + dict. +* ``parameters`` are presented as connector data. (See Usage + below.) +* The ``environment`` value is not supported. If present, PuppetENC + will issue a warning and skip it. + +The ``parameters`` from separate ENCs are all merged together, +including parameters from any parameterized classes. This is a +shallow merge; in other words, only the top-level keys are +considered. For instance, assuming you had one ENC that produced:: + + parameters: + ntp_servers: + - 0.pool.ntp.org + - ntp1.example.com + +And another that produced:: + + parameters: + ntp_servers: + - ntp2.example.com + +This would result in connector data that included *either* the first +value of ``ntp_servers`` *or* the second, but not both; this would +depend on the order in which the ENCs were run, which is +non-deterministic and should not be relied upon. However, if you add +one ENC that produced:: + + parameters: + ntp_servers: + - 0.pool.ntp.org + - ntp1.example.com + +And another that produced:: + + parameters: + mail_servers: + - mail.example.com + +Then the connector data would consist of:: + + {"ntp_servers": ["0.pool.ntp.org", "ntp1.example.com"], + "mail_servers": ["mail.example.com"]} + +Usage +===== + +To use the PuppetENC plugin, first do ``mkdir +/var/lib/bcfg2/PuppetENC``. Add ``PuppetENC`` to your ``plugins`` +line in ``/etc/bcfg2.conf``. Now you can place any ENCs you wish to +run in ``/var/lib/bcfg2/PuppetENC``. Note that ENCs are run each time +client metadata is generated, so if you have a large number of ENCs or +ENCs that are very time-consuming, they could have a significant +impact on server performance. In that case, it could be worthwhile to +write a dedicated Connector plugin. + +PuppetENC parameters can be accessed in templates as +``metadata.PuppetENC``, which is a dict of all parameter data merged +together. For instance, given the following ENC output:: + + --- + classes: + common: + puppet: + ntp: + ntpserver: 0.pool.ntp.org + aptsetup: + additional_apt_repos: + - deb localrepo.example.com/ubuntu lucid production + - deb localrepo.example.com/ubuntu lucid vendor + parameters: + ntp_servers: + - 0.pool.ntp.org + - ntp.example.com + mail_server: mail.example.com + iburst: true + environment: production + +``metadata.PuppetENC`` would contain:: + + 'additional_apt_repos': ['deb localrepo.example.com/ubuntu lucid production', + 'deb localrepo.example.com/ubuntu lucid vendor'], + 'iburst': True, + 'mail_server': 'mail.example.com', + 'ntp_servers': ['0.pool.ntp.org', 'ntp.example.com'], + 'ntpserver': '0.pool.ntp.org'} + +(Note that the duplication of NTP server data doesn't make this an +especially *good* example; it's just the official Puppet example.) + +So, in a template you could do something like:: + + {% for repo in metadata.PuppetENC['additional_apt_repos'] %}\ + ${repo} + {% end %}\ diff --git a/doc/server/plugins/connectors/templatehelper.txt b/doc/server/plugins/connectors/templatehelper.txt new file mode 100644 index 000000000..24d7f18b5 --- /dev/null +++ b/doc/server/plugins/connectors/templatehelper.txt @@ -0,0 +1,74 @@ +.. -*- mode: rst -*- + +.. _server-plugins-connectors-templatehelper: + +============== +TemplateHelper +============== + +The TemplateHelper plugin is a connector plugin that adds Python +classes and methods to client metadata instances for use in +templates. This allows you to easily reuse code that is common +amongst multiple templates and add convenience methods. + +Using TemplateHelper +==================== + +First, ``mkdir /var/lib/bcfg2/TemplateHelper`` and add +**TemplateHelper** to your ``plugins`` line in ``/etc/bcfg2.conf``. +Restart ``bcfg2-server``. + +Now, any ``.py`` file placed in ``/var/lib/bcfg2/TemplateHelper/`` +will be read and added to matching client metadata objects. See +:ref:`writing-templatehelpers` below for more information on how to +write TemplateHelper scripts. + +TemplateHelper supports group- and host-specific helpers, so you could +create, e.g., ``foo.py.G80_test`` to create a helper that only applied +to machines in the group ``test``. + +.. _writing-templatehelpers: + +Writing Helpers +=============== + +A helper module is just a Python module with three special conditions: + +* The filename must end with ``.py`` (before any specificity + strings, e.g., ``.G80_foo`` or ``.H_blah.example.com`` +* The module must have an attribute, ``__export__``, that lists all of + the classes, functions, variables, or other symbols you wish to + export from the module. +* ``data``, ``handle_event``, ``name``, and ``specific`` are reserved + names. You should not include symbols with a reserved name in + ``__export__``. Additionally, including symbols that start with an + underscore or double underscore is bad form, and may also produce + errors. + +See ``examples/TemplateHelper`` for examples of helper modules. + +Usage +===== + +Specific helpers can be referred to in +templates as ``metadata.TemplateHelper[<modulename>]``. That accesses +a HelperModule object will has, as attributes, all symbols listed in +``__export__``. For example, consider this helper module:: + + __export__ = ["hello"] + + def hello(metadata): + return "Hello, %s!" % metadata.hostname + +To use this in a TGenshi template, we could do:: + + ${metadata.TemplateHelper['hello'].hello(metadata)} + +The template would produce:: + + Hello, foo.example.com! + +Note that the client metadata object is not passed to a helper module +in any magical way; if you want to access the client metadata object +in a helper function or class, you must pass the object to the +function manually. diff --git a/doc/server/plugins/generators/cfg.txt b/doc/server/plugins/generators/cfg.txt index 1c8f6f11a..6c848fddb 100644 --- a/doc/server/plugins/generators/cfg.txt +++ b/doc/server/plugins/generators/cfg.txt @@ -35,24 +35,24 @@ templating -- see below). Group-Specific Files ==================== -It is often that you want one version of a config file for all of your -machines except those in a particular group. For example, ``/etc/fstab`` -should look alike on all of your desktop machines, but should be -different on your file servers. Bcfg2 can handle this case through use -of group-specific files. +It is often the case that you want one version of a config file for +all of your machines except those in a particular group. For example, +``/etc/fstab`` should look alike on all of your desktop machines, but +should be different on your file servers. Bcfg2 can handle this case +through use of group-specific files. As mentioned above, all Cfg entries live in like-named directories at the end of their directory tree. In the case of fstab, the file at ``Cfg/etc/fstab/fstab`` will be handed out by default to any client that asks for a copy of ``/etc/fstab``. Group-specific files are located in -the same directory and are named with the syntax:: +the same directory and are named with the following syntax:: /path/to/filename/filename.GNN_groupname -in which **NN** is a priority number where **00** is lowest and -**99** is highest, and **groupname** is the name of a group defined in +**NN** is a priority number where **00** is lowest and **99** +is highest, and **groupname** is the name of a group defined in ``Metadata/groups.xml``. Back to our fstab example, we might have a -``Cfg/etc/fstab/`` directory that looks like:: +``Cfg/etc/fstab/`` directory that looks like this:: fstab fstab.G50_server @@ -93,9 +93,158 @@ classes. one. That way if a hostname changes or an extra copy of a particular client is built, it will get the same changes as the original. +Templates +========= + +Genshi Templates +---------------- + +Genshi templates maybe used for entries as well. Any file ending in .genshi +will be processed using the new template style (like .newtxt in the TGenshi +plugin). + +Cheetah Templates +----------------- + +Cheetah templates maybe used for entries as well. Simply name your file +with a .cheetah extenstion and it will be processed like the TCheetah +plugin. + +Notes on Using Templates +------------------------ + +Templates can be host and group specific as well. Deltas will not be +processed for any genshi or cheetah base file. + +.. note:: + + If you are using templating in combination with host-specific + or group-specific files, you will need to ensure that the ``.genshi`` + or ``.cheetah`` extension is at the **end** of the filename. Using the + examples from above for *host.example.com* and group *server* you would + have the following (using genshi only):: + + Cfg/etc/fstab/fstab.H_host.example.com.genshi + Cfg/etc/fstab/fstab.G50_server.genshi + +Genshi templates take precence over cheetah templates. For example, if +two files exist named + + Cfg/etc/fstab/fstab.genshi + Cfg/etc/fstab/fstab.cheetah + +the cheetah template is ignored. But you can mix genshi and cheetah when +using different host-specific or group-specific files. For example: + + Cfg/etc/fstab/fstab.H_host.example.com.genshi + Cfg/etc/fstab/fstab.G50_server.cheetah + +Encrypted Files +=============== + +.. versionadded:: 1.3.0 + +Bcfg2 allows you to encrypt files stored in ``Cfg/`` to protect the +data in them from other people who need access to the repository. See +also :ref:`server-plugins-connectors-properties-encrypted` for +information on encrypting elements in Properties files, which is often +more friendly for tracking changes in a VCS. + +.. note:: + + This feature is *not* intended to secure the files against a + malicious attacker who has gained access to your Bcfg2 server, as + the encryption passphrases are held in plaintext in + ``bcfg2.conf``. This is only intended to make it easier to use a + single Bcfg2 repository with multiple admins who should not + necessarily have access to each other's sensitive data. + +Encrypting Files +---------------- + +An encrypted file should end with ``.crypt``, e.g.:: + + Cfg/etc/foo.conf + Cfg/etc/foo.conf/foo.conf.crypt + Cfg/etc/foo.conf/foo.conf.G10_foo.crypt + +Encrypted Genshi or Cheetah templates can have the extensions in +either order, e.g.:: + + Cfg/etc/foo.conf/foo.conf.crypt.genshi + Cfg/etc/foo.conf/foo.conf.G10_foo.genshi.crypt + Cfg/etc/foo.conf/foo.conf.H_bar.example.com.crypt.cheetah + +To encrypt a file, you can use ``bcfg2-crypt``, e.g.:: + + bcfg2-crypt foo.conf + +Once you are satisfied that the file has been encrypted as you wish, +you can remove the plaintext version, or you can use the ``--remove`` +flag of ``bcfg2-crypt``. + +To decrypt a file, simply run ``bcfg2-crypt`` again:: + + bcfg2-crypt foo.conf + +See the ``bcfg2-crypt`` man page for more information. + +``bcfg2-crypt`` simply performs an AES256 encryption, and is +more-or-less equivalent to the following commands (encryption and +decryption, respectively:: + + openssl enc -aes-256-cbc -k <passphrase> -in foo.conf -out foo.conf.crypt -a + openssl enc -d -aes-256-cbc -k <passphrase> -in foo.conf.crypt -out foo.conf -a + +.. _server-plugins-generators-cfg-configuring-encryption: + +Configuring Encryption +---------------------- + +To configure encryption, add a ``[encryption]`` section to +``bcfg2.conf`` with any number of name-passphrase pairs. When +decrypting a file, _all_ passphrases will be tried; the passphrase +name is currently purely cosmetic, but at some point in the future the +ability to give Bcfg2 a "hint" about which passphrase to use will be +added. + +For instance:: + + [encryption] + foo_team=P4ssphr4se + bar_team=Pa55phra5e + +This would define two separate encryption passphrases, presumably for +use by two separate teams. The passphrase names are completely +arbitrary. + +Note that this does entail a chicken-and-egg problem. In order for +the Bcfg2 server to be able to decrypt encrypted files, the +passphrases must exist in ``bcfg2.conf`` in plaintext; but, if you're +encrypting data, presumably you don't want to include those plaintext +passphrases in your Bcfg2 repository, so you'll want to encrypt +``bcfg2.conf``. The best way to solve this is: + +#. On your Bcfg2 server, manually add the ``[encryption]`` section to + ``bcfg2.conf`` and restart the Bcfg2 server. +#. Update ``bcfg2.conf`` in your Bcfg2 repository with the + passphrases, and encrypt it. + +The first (manual) step breaks the mutual dependency. + Deltas ====== +.. note:: + + In Bcfg2 1.3 and newer, deltas are deprecated. It is recommended + that you use templates instead. The + :ref:`TemplateHelper plugin + <server-plugins-connectors-templatehelper>` comes with an example + helper that can be used to include other files easily, a subset of + cat file functionality. ``bcfg2-lint`` checks for deltas and + warns about them. + Bcfg2 has finer grained control over how to deliver configuration files to a host. Let's say we have a Group named file-server. Members of this group need the exact same ``/etc/motd`` as all other hosts except @@ -150,51 +299,50 @@ file. The reason the other deltas aren't applied to *foo.example.com* is because a **.H_** delta is more specific than a **.G##_** delta. Bcfg2 applies all the deltas at the most specific level. -Templates -========= - -Genshi Templates ----------------- +Content Validation +================== -Genshi templates maybe used for entries as well. Any file ending in .genshi -will be processed using the new template style (like .newtxt in the TGenshi -plugin). +To ensure that files with invalid content are not pushed out, you can +provide a content validation script that will be run against each +file. Create a file called ``:test`` inside the directory for the +file you want to test. For example:: -Cheetah Templates ------------------ + Cfg/etc/sudoers/:test -Cheetah templates maybe used for entries as well. Simply name your file -with a .cheetah extenstion and it will be processed like the TCheetah -plugin. +You can also create host- and group-specific validators:: -Notes on Using Templates ------------------------- + Cfg/etc/sudoers/:test.G80_foogroup + Cfg/etc/sudoers/:test.H_bar.example.com -Templates can be host and group specific as well. Deltas will not be -processed for any genshi or cheetah base file. +A validator script has the following attributes: -.. note:: +* It must be executable, or specify a valid bangpath; +* The entire content of the file is passed to the validator on + stdin; +* The validator is not called with any flags or arguments; +* The validator must return 0 on success and non-zero on failure; and +* The validator must output a sensible error message on failure. - If you are using templating in combination with host-specific - or group-specific files, you will need to ensure that the ``.genshi`` - or ``.cheetah`` extension is at the **end** of the filename. Using the - examples from above for *host.example.com* and group *server* you would - have the following (using genshi only):: +For ``sudoers``, a very simple validator is:: - Cfg/etc/fstab/fstab.H_host.example.com.genshi - Cfg/etc/fstab/fstab.G50_server.genshi + #!/bin/sh + visudo -cf - -Genshi templates take precence over cheetah templates. For example, if -two files exist named +This uses the ``visudo`` command's built-in validation. - Cfg/etc/fstab/fstab.genshi - Cfg/etc/fstab/fstab.cheetah +If you wish to disable validation, this can be done with the following +setting in ``bcfg2.conf``:: -the cheetah template is ignored. But you can mix genshi and cheetah when -using different host-specific or group-specific files. For example: + [cfg] + validation=no - Cfg/etc/fstab/fstab.H_host.example.com.genshi - Cfg/etc/fstab/fstab.G50_server.cheetah +If you have a very large number of validators, you may wish to disable +validation by default to avoid slowing down the generation of +configurations on the server, and use ``bcfg2-test`` (for instance, as +a post-commit hook or as part of a code review process) to run +validation. You can do this by setting ``validation=no`` in +``bcfg2.conf`` as described above, and then calling ``bcfg2-test`` +with the ``--cfg-validation`` flag. File permissions ================ diff --git a/doc/server/plugins/generators/nagiosgen.txt b/doc/server/plugins/generators/nagiosgen.txt index b839c10ca..f722909fc 100644 --- a/doc/server/plugins/generators/nagiosgen.txt +++ b/doc/server/plugins/generators/nagiosgen.txt @@ -20,7 +20,7 @@ Create the NagiosGen directory:: Create default host, and group specs in: -* /var/lib/bcfg2/NagiosGen/default-host.cfg:: +``/var/lib/bcfg2/NagiosGen/default-host.cfg``:: define host{ name default @@ -44,7 +44,7 @@ Create default host, and group specs in: retry_interval 1 } -* /var/lib/bcfg2/NagiosGen/default-group.cfg:: +``/var/lib/bcfg2/NagiosGen/default-group.cfg``:: define service{ name default-service @@ -73,7 +73,7 @@ Create default host, and group specs in: Create group configuration files (Named identical to Bcfg2 groups) and add services, and commands specific to the hostgroup (Bcfg2 group) in -* /var/lib/bcfg2/NagiosGen/base-group.cfg:: +``/var/lib/bcfg2/NagiosGen/base-group.cfg``:: define hostgroup{ hostgroup_name base @@ -100,7 +100,7 @@ add services, and commands specific to the hostgroup (Bcfg2 group) in hostgroup_name base } -* /var/lib/bcfg2/NagiosGen/web-server-group.cfg:: +``/var/lib/bcfg2/NagiosGen/web-server-group.cfg``:: define hostgroup{ hostgroup_name web-server diff --git a/doc/server/plugins/generators/packages.txt b/doc/server/plugins/generators/packages.txt index 54e92f1f6..38952de3e 100644 --- a/doc/server/plugins/generators/packages.txt +++ b/doc/server/plugins/generators/packages.txt @@ -56,19 +56,19 @@ member clients. .. versionadded:: 1.2.0 Magic OS groups can be disabled in Bcfg2 1.2 and greater by setting - ``magic_groups`` to ``0`` in ``Packages/packages.conf``. This may + ``magic_groups`` to ``0`` in ``Packages/packages.conf``. This may give you greater flexibility in determining which source types to - use for which OSes. Magic architecture groups cannot be disabled. + use for which OSes. Magic architecture groups cannot be disabled. Limiting sources to groups ========================== ``Packages/sources.xml`` processes ``<Group>`` and ``<Client>`` tags -just like Bundles. In addition to any groups or clients specified +just like Bundles. In addition to any groups or clients specified that way, clients must be a member of the appropriate architecture group as specified in a -Source stanza. In total, in order for a source to be associated with +Source stanza. In total, in order for a source to be associated with a client, the client must be in one of the magic groups (debian, ubuntu, or nexenta), any explicit groups or clients specified in ``sources.xml``, and any specified architecture groups. @@ -126,15 +126,15 @@ Disabling dependency resolution .. versionadded:: 1.1.0 -Dependency resolution can be disabled by adding this to -``Packages/packages.conf`` in the ``global`` section:: +Dependency resolution can be disabled by adding the following setting +to ``bcfg2.conf`` in the ``packages`` section:: - [global] + [packages] resolver=0 All metadata processing can be disabled as well:: - [global] + [packages] metadata=0 This setting implies disabling the resolver. @@ -147,7 +147,9 @@ Packages, please file a bug report so that we can fix the problem in future releases. In the meantime, you can work around this issue by blacklisting the offending Package in your Sources. The blacklist element should immediately follow the Component section of your source -and should look like the following:: +and should look like the following: + +.. code-block::xml <Blacklist>unwanted-packagename</Blacklist> @@ -163,8 +165,10 @@ Handling GPG Keys If you have yum libraries installed, Packages can automatically handle GPG signing keys for Yum and Pulp repositories. (You do not need to use the native yum resolver; if yum libraries are available, GPG -signing keys can be handled automatically.) Simply specify the URL to -the GPG key(s) for a repository in ``sources.xml``:: +signing keys can be handled automatically.) Simply specify the URL to +the GPG key(s) for a repository in ``sources.xml``: + +.. code-block::xml <Source type="yum" rawurl="http://mirror.example.com/centos6-x86_64/RPMS.os"> @@ -218,7 +222,9 @@ Example usage ============= Create a ``sources.xml`` file in the Packages directory that looks -something like this:: +something like this: + +.. code-block::xml <Sources> <Group name="ubuntu-intrepid"> @@ -237,14 +243,13 @@ something like this:: .. versionadded:: 1.1.0 - The default behavior of the Packages plugin is to not make - any assumptions about which packages you want to have added - automatically. For that reason, neither **Recommended** nor - **Suggested** packages are added as dependencies by default. You - will notice that the default behavior for apt is to add Recommended - packages as dependencies. You can configure the Packages plugin to - add recommended packages by adding the ``recommended`` attribute, - e.g.: + The default behavior of the Packages plugin is to not make any + assumptions about which packages you want to have added automatically + [#f1]_. For that reason, neither **Recommended** nor **Suggested** + packages are added as dependencies by default. You will notice + that the default behavior for apt is to add Recommended packages as + dependencies. You can configure the Packages plugin to add recommended + packages by adding the ``recommended`` attribute, e.g.: .. code-block:: xml @@ -253,7 +258,17 @@ something like this:: .. warning:: You must regenerate the Packages cache when adding or removing the recommended attribute. -Yum sources can be similarly specified:: + .. [#f1] Bcfg2 will by default add **Essential** packages to the + client specification. You can disable this behavior by + setting the ``essential`` attribute to *false*: + + .. code-block:: xml + + <Source type="apt" essential="false" ...> + +Yum sources can be similarly specified: + +.. code-block::xml <Sources> <Group name="centos-5.2"> @@ -276,6 +291,8 @@ also necessary. :ref:`Pulp sources <pulp-source-support>` are very simple to specify due to the amount of data that can be queried from Pulp itself:: +.. code-block::xml + <Sources> <Group name="centos-6-x86_64"> <Source type="yum" pulp_id="centos-6-x86_64-os"/> @@ -367,7 +384,7 @@ updated. Availability ============ -Support for clients using yum and apt is currently available. Support for +Support for clients using yum and apt is currently available. Support for other package managers (Portage, Zypper, IPS, etc) remain to be added. Validation @@ -378,20 +395,13 @@ be validated using ``bcfg2-lint``. .. note:: The schema requires that elements be specified in the above order. -Limitations -=========== - -Packages does not do traditional caching as other plugins -do. Modifying sources in the Packages ``sources.xml`` file requires a -server restart for the time being. You do not have to restart the -server after changing ``packages.conf`` or after adding new sources to -``sources.xml``. - Package Checking and Verification ================================= In order to do disable per-package verification Pkgmgr style, you will -need to use :ref:`BoundEntries <boundentries>`, e.g.:: +need to use :ref:`BoundEntries <boundentries>`, e.g.: + +.. code-block::xml <BoundPackage name="mem-agent" priority="1" version="auto" type="yum" verify="false"/> @@ -405,10 +415,10 @@ Generating Client APT/Yum Configurations .. versionadded:: 1.2.0 The Packages plugin has native support for generating Yum configs. -You must set ``yum_config`` in ``Packages/packages.conf`` to the path -to the yum config file you want to generate:: +You must set ``yum_config`` in ``bcfg2.conf`` to the path to the yum +config file you want to generate:: - [global] + [packages] yum_config=/etc/yum.repos.d/all.repo Then add the corresponding Path entry to your Yum bundle. @@ -416,9 +426,9 @@ Then add the corresponding Path entry to your Yum bundle. .. versionadded:: 1.1.0 APT repository information can be generated automatically from -software sources using :doc:`./tgenshi/index` or :doc:`./tcheetah`. A +software sources using :doc:`./tgenshi/index` or :doc:`./tcheetah`. A list of source urls are exposed in the client's metadata as -``metadata.Packages.sources``. E.g.:: +``metadata.Packages.sources``. E.g.:: # bcfg2 maintained apt @@ -436,10 +446,10 @@ Using Native Yum Libraries By default, Bcfg2 uses an internal implementation of Yum's dependency resolution and other routines so that the Bcfg2 server can be run on a -host that does not support Yum itself. If you run the Bcfg2 server on +host that does not support Yum itself. If you run the Bcfg2 server on a machine that does have Yum libraries, however, you can enable use of those native libraries in Bcfg2 by setting ``use_yum_libraries`` to -``1`` in the ``[yum]`` section of ``Packages/packages.conf``. +``1`` in the ``[packages:yum]`` section of ``bcfg2.conf``. Benefits to this include: @@ -450,7 +460,7 @@ Benefits to this include: Drawbacks include: -* More disk I/O. In some cases, you may have to raise the open file +* More disk I/O. In some cases, you may have to raise the open file limit for the user who runs your Bcfg2 server process, particularly if you have a lot of repositories. * Resolution of package dependencies is slower in some cases, @@ -468,24 +478,24 @@ bcfg2-server process uses an external short-lived helper, library support. By default, Bcfg2 looks for this helper in ``$PATH``, or, failing that, at ``/usr/sbin/bcfg2-yum-helper``. If you have installed the helper elsewhere, you will need to configure -that location with the ``helper`` option in the ``[yum]`` section, -e.g.:: +that location with the ``helper`` option in the ``[packages:yum]`` +section, e.g.:: - [yum] + [packages:yum] use_yum_libraries = 1 helper = /usr/local/sbin/bcfg2-yum-helper Setting Yum Options ------------------- -In ``Packages/packages.conf``, any options you set in the ``[yum]`` +In ``bcfg2.conf``, any options you set in the ``[packages:yum]`` section other than ``use_yum_libraries`` and ``helper`` will be passed along verbatim to the configuration of the Yum objects used in the -Bcfg2 server. The following options are set by default, and should -not generally be overridden: +Bcfg2 server. The following options are set by default, and should not +generally be overridden: * ``cachedir`` is set to a hashed value unique to each distinct Yum - configuration. Don't set this unless you know what you're doing. + configuration. Don't set this unless you know what you're doing. * ``keepcache`` is set to ``0``; there is no benefit to changing this. * ``sslverify`` is set to ``0``; change this if you know what you're doing. @@ -495,17 +505,21 @@ not generally be overridden: Package Groups -------------- -Yum package groups are supported by the native Yum libraries. To +Yum package groups are supported by the native Yum libraries. To include a package group, use the ``group`` attribute of the -``Package`` tag. You can use either the short group ID or the long -group name:: +``Package`` tag. You can use either the short group ID or the long +group name: + +.. code-block::xml <Package group="SNMP Support"/> <Package group="system-management-snmp"/> By default, only those packages considered the "default" packages in a -group will be installed. You can change this behavior using the -"type" attribute:: +group will be installed. You can change this behavior using the +"type" attribute: + +.. code-block::xml <Package group="development" type="optional"/> <Package group="Administration Tools" type="mandatory"/> @@ -518,7 +532,7 @@ Valid values of "type" are: including mandatory, default, and optional packages. You can view the packages in a group by category with the ``yum -groupinfo`` command. More information about the different levels can +groupinfo`` command. More information about the different levels can be found at http://fedoraproject.org/wiki/How_to_use_and_edit_comps.xml_for_package_groups#Installation @@ -530,9 +544,11 @@ Pulp Support .. versionadded:: 1.2.0 Bcfg2 contains explicit support for repositories managed by Pulp -(http://pulpproject.org/). Due to the amount of data about a +(http://pulpproject.org/). Due to the amount of data about a repository that can be retrieved directly from Pulp, the only thing -necessary to configure a Pulp repo is the repo ID:: +necessary to configure a Pulp repo is the repo ID: + +.. code-block::xml <Sources> <Group name="centos-6-x86_64"> @@ -542,17 +558,17 @@ necessary to configure a Pulp repo is the repo ID:: </Group> </Sources> -Pulp sources require some additional configuration. First, the Bcfg2 +Pulp sources require some additional configuration. First, the Bcfg2 server must have a valid ``/etc/pulp/consumer/consumer.conf`` that is readable by the user your Bcfg2 server runs as; the Pulp server, URLs, and so on, are determined from this. -Secondly, in ``Packages/packages.conf`` you must set the following -options in the ``[pulp]`` section: +Secondly, in ``bcfg2.conf`` you must set the following +options in the ``[packages:pulp]`` section: * ``username`` and ``password``: The username and password of a Pulp user that will be used to register new clients and bind them to - repositories. Membership in the default ``consumer-users`` role is + repositories. Membership in the default ``consumer-users`` role is sufficient. Bcfg2 clients using Pulp sources will be registered to the Pulp server @@ -669,20 +685,20 @@ multiple data sources need to be multiplexed. The APT source in ``src/lib/Server/Plugins/Packages.py`` provides a relatively simple implementation of a source. -packages.conf +Configuration ============= -``packages.conf`` contains miscellaneous configuration options for the -Packages plugin. Any booleans in the config file accept the values +``bcfg2.conf`` contains miscellaneous configuration options for the +Packages plugin. Any booleans in the config file accept the values "1", "yes", "true", and "on" for True, and "0", "no", "false", and -"off" for False +"off" for False. It understands the following directives: -[global] section ----------------- +[packages] section +------------------ -* ``resolver``: Enable dependency resolution. Default is ``1`` +* ``resolver``: Enable dependency resolution. Default is ``1`` (true). For historical reasons, this also accepts "enabled" and "disabled". * ``metadata``: Enable metadata processing. Default is ``1`` @@ -691,28 +707,31 @@ It understands the following directives: "enabled" and "disabled". * ``yum_config``: The path at which to generate Yum configs. No default. -* ``apt_config``: The path at which to generate APT configs. No +* ``apt_config``: The path at which to generate APT configs. No default. * ``gpg_keypath``: The path on the client RPM GPG keys will be copied - to before they are imported on the client. Default is + to before they are imported on the client. Default is "/etc/pki/rpm-gpg". * ``version``: Set the version attribute used when binding Packages. Default is ``auto``. -[yum] section -------------- +[packages:yum] section +---------------------- * ``use_yum_libraries``: Whether or not to use the :ref:`native yum - library support <native-yum-libraries>`. Default is ``0`` (false). + library support <native-yum-libraries>`. Default is ``0`` (false). +* ``helper``: Path to ``bcfg2-yum-helper``. By default, Bcfg2 looks + first in ``$PATH`` and then in ``/usr/sbin/bcfg2-yum-helper`` for + the helper. -All other options in the ``[yum]`` section will be passed along -verbatim to the Yum configuration if you are using the native Yum -library support. +All other options in the ``[packages:yum]`` section will be passed +along verbatim to the Yum configuration if you are using the native +Yum library support. -[pulp] section --------------- +[packages:pulp] section +----------------------- * ``username`` and ``password``: The username and password of a Pulp user that will be used to register new clients and bind them to - repositories. Membership in the default ``consumer-users`` role is + repositories. Membership in the default ``consumer-users`` role is sufficient. diff --git a/doc/server/plugins/generators/rules.txt b/doc/server/plugins/generators/rules.txt index c084c5681..3b1d94480 100644 --- a/doc/server/plugins/generators/rules.txt +++ b/doc/server/plugins/generators/rules.txt @@ -118,7 +118,14 @@ Service Tag +------------+-------------------------------+---------------------------------------------------------+ | Name | Description | Values | +============+===============================+=========================================================+ -| mode | Per Service Mode (New in 1.0) | (manual | default | supervised | interactive_only ) | +| restart | Whether to restart the | ( true | false | interactive ) | +| | service when the bundle | | +| | changes (new in 1.3; replaces | | +| | "mode" attribute) | | ++------------+-------------------------------+---------------------------------------------------------+ +| install | Whether to install the | ( true | false ) | +| | service (new in 1.3; replaces | | +| | "mode" attribute) | | +------------+-------------------------------+---------------------------------------------------------+ | name | Service name or regular | String or regex | | | expression | | @@ -139,29 +146,33 @@ Service Tag | | (Upstart services only) | | +------------+-------------------------------+---------------------------------------------------------+ -Service mode descriptions -^^^^^^^^^^^^^^^^^^^^^^^^^ - -.. versionadded:: 1.0.0 - -* manual +Service mode specification +^^^^^^^^^^^^^^^^^^^^^^^^^^ - * do not start/stop/restart this service - * service installation is not performed +.. versionadded:: 1.3.0 -* interactive_only +In the 1.3.0 release, the "mode" attribute has been replaced by a pair +of attributes, "restart" and "install," which control how a service is +handled more granularly than the old "mode" attribute. The old "mode" +attribute values are equivalent as follows: - * only attempt to start/stop/restart this service if the client is run interactively - * service installation is performed ++-----------------------------+------------------------------------------+ +| Mode attribute | Equivalent | ++=============================+==========================================+ +| ``mode="default"`` | ``restart="true" install="true"`` | ++-----------------------------+------------------------------------------+ +| ``mode="interactive_only"`` | ``restart="interactive" install="true"`` | ++-----------------------------+------------------------------------------+ +| ``mode="supervised"`` | ``restart="true" install="true"`` | ++-----------------------------+------------------------------------------+ +| ``mode="manual"`` | ``restart="false" install="false"`` | ++-----------------------------+------------------------------------------+ -* default +The default is ``restart="true" install="true"`` - * perform appropriate service operations - -* supervised - - * default and ensure service is running (or stopped) when verification is performed - * deprecates supervised='true' +Previously, "supervised" could be used to start a service during the +verification phase; this is no longer supported. Services that have +been stopped on a client will be started during the install phase. Service status descriptions ^^^^^^^^^^^^^^^^^^^^^^^^^^^ @@ -356,14 +367,11 @@ Using Regular Expressions in Rules If you wish, you can configure the Rules plugin to support regular expressions. This entails a small performance and memory usage -penalty. To do so, create a file, "Rules/rules.conf", and add the -following text:: +penalty. To do so, add the following setting to ``bcfg2.conf``:: [rules] regex = yes -You will have to restart the Bcfg2 server after making that change. - With regular expressions enabled, you can use a regex in the ``name`` attribute to match multiple abstract configuration entries. @@ -372,4 +380,4 @@ name="bcfg2".../>`` will *not* match a Service named ``bcfg2-server``; you'd have to explicitly specify ``<Service name="bcfg2.*".../>``. Note that only one Rule can apply to any abstract entry, so you cannot -specify multiple regexs to match the same rule. +specify multiple regexes to match the same rule. diff --git a/doc/server/plugins/generators/tgenshi/index.txt b/doc/server/plugins/generators/tgenshi/index.txt index 21ef8f17f..0d4a7ffb0 100644 --- a/doc/server/plugins/generators/tgenshi/index.txt +++ b/doc/server/plugins/generators/tgenshi/index.txt @@ -17,7 +17,7 @@ on the client in the created files. To begin, you will need to download and install the Genshi templating engine. -To install on CentOS or RHEL 5, run:: +To install on CentOS or RHEL, run:: sudo yum install python-genshi diff --git a/doc/server/plugins/grouping/metadata.txt b/doc/server/plugins/grouping/metadata.txt index 305857578..2c05e9e7e 100644 --- a/doc/server/plugins/grouping/metadata.txt +++ b/doc/server/plugins/grouping/metadata.txt @@ -37,11 +37,11 @@ describe one host. A sample file is below: .. code-block:: xml <Clients version="3.0"> - <Client profile="backup-server" pingable="Y" pingtime="0" name="backup.example.com"/> - <Client profile="console-server" pingable="Y" pingtime="0" name="con.example.com"/> - <Client profile="kerberos-master" pingable="Y" pingtime="0" name="kdc.example.com"/> - <Client profile="mail-server" pingable="Y" pingtime="0" name="mail.example.com"/> - <Client name='foo' address='10.0.0.1' pingable='N' pingtime='-1'> + <Client profile="backup-server" name="backup.example.com"/> + <Client profile="console-server" name="con.example.com"/> + <Client profile="kerberos-master" name="kdc.example.com"/> + <Client profile="mail-server" name="mail.example.com"/> + <Client name='foo' address='10.0.0.1'> <Alias name='foo-mgmt' address='10.1.0.1'/> </Client> </Clients> @@ -93,12 +93,6 @@ Additionally, the following properties can be specified: | | can be used instead of the global | | | | password. | | +----------+----------------------------------------+----------------+ -| pingable | If the client is pingable (deprecated; | Y|N | -| | for old reporting system) | | -+----------+----------------------------------------+----------------+ -| pingtime | Last time the client was pingable | String | -| | (deprecated; for old reporting system) | | -+----------+----------------------------------------+----------------+ | secure | Requires the use of the per-client | true|false | | | password for this client. | | +----------+----------------------------------------+----------------+ diff --git a/doc/server/plugins/misc/trigger.txt b/doc/server/plugins/misc/trigger.txt index 7547f2fdd..224b7444b 100644 --- a/doc/server/plugins/misc/trigger.txt +++ b/doc/server/plugins/misc/trigger.txt @@ -6,8 +6,8 @@ Trigger ======= -Trigger is a plugin that calls external scripts (on the server) when -clients are configured. +Trigger is a plugin that calls external scripts (on the server) at the +end of each client run. Setup ===== diff --git a/doc/server/plugins/plugin-roles.txt b/doc/server/plugins/plugin-roles.txt index 2ce7e21ff..58dc36296 100644 --- a/doc/server/plugins/plugin-roles.txt +++ b/doc/server/plugins/plugin-roles.txt @@ -6,124 +6,41 @@ Plugin Roles ============ -This documents available plugin roles. - -1. list of plugin roles - - +---------------+--------------------+--------+ - | Role | Class | Status | - +===============+====================+========+ - | Metadata | Metadata | done | - +---------------+--------------------+--------+ - | Connector | Connector | done | - +---------------+--------------------+--------+ - | Probing | Probing | done | - +---------------+--------------------+--------+ - | Structure | Structure | done | - +---------------+--------------------+--------+ - | Structure Val | StructureValidator | done | - +---------------+--------------------+--------+ - | Generator | Generator | done | - +---------------+--------------------+--------+ - | Goals Val | GoalValidator | done | - +---------------+--------------------+--------+ - | Statistics | Statistics | done | - +---------------+--------------------+--------+ - | Pull Source | PullSource | done | - +---------------+--------------------+--------+ - | Pull Target | PullTarget | done | - +---------------+--------------------+--------+ - | Version | Version | done | - +---------------+--------------------+--------+ - | Decision | Decision | done | - +---------------+--------------------+--------+ - | Remote | Remote | none | - +---------------+--------------------+--------+ - | Syncing | Syncing | none | - +---------------+--------------------+--------+ - -2. Plugin Capabilities - - * Metadata - - * Initial metadata construction - * Connector data accumulation - * ClientMetadata instance delivery - * Introspection interface (for bcfg2-info & co) - - * Connector - - * Provide additional data for ClientMetadata instances - - * Probing - - * send executable probes to clients and receive data responses - - * Structure - - * Produce a list of configuration entries that should be included in client configurations - * Each structure plugin is produces a list of structures - * Core verifies that each bundle listed has been constructed - - * Structure Validation - - * Validate a client entry list's internal consistency, modifying if needed - - * Generator - * Goals Validation - - * Validate client goals, modifying if needed - - * Pull Source - - * Plugin can provide entry information about clients - - * Pull Target - - * Plugin can accept entry data and merge it into the specification - - * Version - - * Plugin can read revision information from VCS of choice - * Will provide an interface for producing commits made by the bcfg2-server - - * Decision - -3. Configuration of plugins - - Plugin configuration will be simplified substantially. Now, a single - list of plugins (including plugins of all capabilities) is specified - upon startup (either via bcfg2.conf or equivalent). This mechanism - replaces the current split configuration mechanism where generators, - structures, and other plugins are listed independently. Instead, all - plugins included in the startup list will be initialized, and each - will be enabled in all roles that it supports. This will remove a - current source of confusion and potential configuration errors, - wherein a plugin is enabled for an improper set of goals. (ie Cfg - enabled as a structure, etc) This does remove the possibility of - partially enabling a plugin for one of its roles without activating it - across the board, but I think this is a corner case, which will be - poorly supported by plugin implementers. If needed, this use case can - be explicitly supported by the plugin author, through use of a config - file directive. - -4. User Visible Changes - - Connector data is added to ClientMetadata instances using the name of - the connector plugin. This means that the dictionary of key/val probe - pairs included with metadata is now available as metadata.Probes - (instead of metadata.probes). Once properties are available the same - way, they will likewise change names to metadata.Properties from their - current name. - - Plugin configuration will change. A single field "plugins" in - bcfg2.conf will supercede the combination of the "generators" and - "structures" fields. - - Default loading of needed plugins is now explicit; this means that - Statistics (if used) should be listed in the plugins line of - bcfg2.conf. - -5. Notes - - * Need to ensure bundle accumulation occurs with connector groups +* Metadata + * Initial metadata construction + * Connector data accumulation + * ClientMetadata instance delivery + * Introspection interface (for bcfg2-info & co) +* Connector + * Provide additional data for ClientMetadata instances +* Probing + * send executable probes to clients and receive data responses +* Structure + * Produce a list of configuration entries that should be included in + client configurations + * Each structure plugin is produces a list of structures + * Core verifies that each bundle listed has been constructed +* StructureValidator + * Validate a client entry list's internal consistency, modifying if + needed +* Generator +* GoalValidator + * Validate client goals, modifying if needed +* PullSource + * Plugin can provide entry information about clients +* PullTarget + * Plugin can accept entry data and merge it into the specification +* Version + * Plugin can read revision information from VCS of choice + * Will provide an interface for producing commits made by the bcfg2-server +* Decision +* ClientRunHooks + * Provides hooks executed at the start and end of each client run + +Configuration of plugins +======================== + +A single list of plugins (including plugins of all capabilities) is +specified upon startup (either via bcfg2.conf or equivalent). All +plugins included in the startup list are initialized, and each is +enabled in all roles that it supports. diff --git a/doc/server/plugins/probes/group.txt b/doc/server/plugins/probes/group.txt index 5c4d6ecb1..03c13db42 100644 --- a/doc/server/plugins/probes/group.txt +++ b/doc/server/plugins/probes/group.txt @@ -52,7 +52,7 @@ Probe used to dynamically set client groups based on OS/distro. # redhat based if [ -x /bin/rpm ]; then OUTPUT="${OUTPUT}\ngroup:rpm" - OS_GROUP=`/bin/rpm -q --qf "%{NAME}" --whatprovides redhat-release | sed 's/-release.*//' | tr '[A-Z]' '[a-z]'` + OS_GROUP=`/bin/rpm -q --qf "%{NAME}" --whatprovides redhat-release | grep -vi 'freeing read locks for locker' | sed 's/-release.*//' | tr '[A-Z]' '[a-z]'` REDHAT_VERSION=`/bin/rpm -q --qf "%{VERSION}" --whatprovides redhat-release` case "$OS_GROUP" in "centos" | "fedora" | "sl") diff --git a/doc/server/plugins/probes/index.txt b/doc/server/plugins/probes/index.txt index 33d6aa7d7..cacc42bc1 100644 --- a/doc/server/plugins/probes/index.txt +++ b/doc/server/plugins/probes/index.txt @@ -208,7 +208,7 @@ look something like: <FileProbes> <FileProbe name="/etc/foo.conf"/> <Group name="blah-servers"> - <FileProbe name="/etc/blah.conf" update="true" + <FileProbe name="/etc/blah.conf" update="true"/> </Group> <Client name="bar.example.com"> <FileProbe name="/var/lib/bar.gz" encoding="base64"/> |