diff options
Diffstat (limited to 'doc')
85 files changed, 2156 insertions, 3070 deletions
diff --git a/doc/appendix/files/mysql.txt b/doc/appendix/files/mysql.txt index 5adf2e27f..0dbbe9b05 100644 --- a/doc/appendix/files/mysql.txt +++ b/doc/appendix/files/mysql.txt @@ -8,14 +8,14 @@ MySQL example ============= -I had some time ago to continue with putting my configuration into +I had some time ago to continue with putting my configuration into Bcfg2 and maybe this helps someone else. I added a new bundle: .. code-block:: xml - <Bundle name="mysql-server" version="3.0"> + <Bundle> <Path name="/root/bcfg2-install/mysql/users.sh"/> <Path name="/root/bcfg2-install/mysql/users.sql"/> <Action name="users.sh"/> @@ -32,9 +32,9 @@ The ``users.sh`` script looks like this: mysql --defaults-extra-file=/etc/mysql/debian.cnf mysql \ < /root/bcfg2-install/mysql/users.sql -On debian there is a user account in ``/etc/mysql/debian.cnf`` -automatically created, but you could also (manually) create a -user in the database that has enough permissions and add the +On debian there is a user account in ``/etc/mysql/debian.cnf`` +automatically created, but you could also (manually) create a +user in the database that has enough permissions and add the login information in a file yourself. This file looks like this:: [client] diff --git a/doc/appendix/files/ntp.txt b/doc/appendix/files/ntp.txt index e14816f6e..c999841da 100644 --- a/doc/appendix/files/ntp.txt +++ b/doc/appendix/files/ntp.txt @@ -13,7 +13,7 @@ another layer of functionality. * After each change, run ``bcfg-repo-validate -v`` * Run the server with ``bcfg2-server -v`` * Update the client with ``bcfg2 -v -d -n`` (will not actually make - client changes) + client changes) Package only ------------ @@ -43,7 +43,7 @@ a client, a profile group, a list of packages, and an NTP bundle. .. code-block:: xml - <Bundle name="ntp"> + <Bundle> <Package name='ntp'/> </Bundle> @@ -75,7 +75,7 @@ Configure the service, and add it to Rules. .. code-block:: xml - <Bundle name="ntp"> + <Bundle> <Package name='ntp'/> <Service name='ntpd'/> </Bundle> @@ -85,16 +85,14 @@ Add config file Setup an ``etc/`` directory structure, and add it to the base:: - # cat Cfg/etc/ntp.conf/ntp.conf + # cat Cfg/etc/ntp.conf/ntp.conf server ntp1.utexas.edu -``Base/base.xml``: - ``Bundler/ntp.xml``: .. code-block:: xml - <Bundle name="ntp"> + <Bundle> <Package name='ntp'/> <Service name='ntpd'/> <Path name='/etc/ntp.conf'/> @@ -114,18 +112,18 @@ used to provide a single service. This is done for several reasons: packages are upgraded, so that they can be repaired if the package install clobbered them. * Services associated with a bundle get restarted whenever any entity - in that bundle is modified. This ensures that new configuration - files and software are used after installation. + in that bundle is modified. This ensures that new configuration + files and software are used after installation. The config file, package, and service are really all related -components describing the idea of an ntp client, so they should be +components describing the idea of an ntp client, so they should be logically grouped together. We use a bundle to accomplish this. ``Bundler/ntp.xml``: .. code-block:: xml - <Bundle name='ntp'> + <Bundle> <Package name='ntp'/> <Service name='ntpd'/> <Path name='/etc/ntp.conf'/> diff --git a/doc/appendix/guides/centos.txt b/doc/appendix/guides/centos.txt index febdf5769..3a35627a8 100644 --- a/doc/appendix/guides/centos.txt +++ b/doc/appendix/guides/centos.txt @@ -102,7 +102,7 @@ Run bcfg2 to be sure you are able to communicate with the server:: Excluding Packages in global exclude list Finished Loaded tool drivers: - Action Chkconfig POSIX YUMng + Action Chkconfig POSIX YUM Phase: initial Correct entries: 0 @@ -147,7 +147,7 @@ Now if you run the client, no more warning:: Excluding Packages in global exclude list Finished Loaded tool drivers: - Action Chkconfig POSIX YUMng + Action Chkconfig POSIX YUM Phase: initial Correct entries: 0 @@ -176,7 +176,7 @@ First, replace **Pkgmgr** with **Packages** in the plugins line of ``bcfg2.conf``. Then create Packages layout (as per :ref:`packages-exampleusage`) in ``/var/lib/bcfg2`` -.. note:: I am using the RawURL syntax here since we are using `mrepo`_ +.. note:: I am using the rawurl syntax here since we are using `mrepo`_ to manage our yum mirrors. .. _mrepo: http://dag.wieers.com/home-made/mrepo/ @@ -184,37 +184,36 @@ line of ``bcfg2.conf``. Then create Packages layout (as per .. code-block:: xml <Sources> - <!-- CentOS (5.4) sources --> - <Source type="yum" rawurl="http://mrepo/centos5-x86_64/RPMS.os"> - <Arch>x86_64</Arch> + <Group name="centos5"> + <!-- CentOS 5 sources --> + <Source type="yum" + rawurl="http://mrepo/centos5-x86_64/RPMS.os"> + <Arch>x86_64</Arch> </Source> - <Source type="yum" rawurl="http://mrepo/centos5-x86_64/RPMS.updates"> - <Arch>x86_64</Arch> + <Source type="yum" + rawurl="http://mrepo/centos5-x86_64/RPMS.updates"> + <Arch>x86_64</Arch> </Source> - <Source type="yum" rawurl="http://mrepo/centos5-x86_64/RPMS.extras"> - <Arch>x86_64</Arch> + <Source type="yum" + rawurl="http://mrepo/centos5-x86_64/RPMS.extras"> + <Arch>x86_64</Arch> </Source> + </Group> </Sources> -Due to the :ref:`server-plugins-generators-packages-magic-groups`, -we need to modify our Metadata. Let's add a **centos5.4** group which -inherits a **centos** group (this should replace the existing **redhat** -group) present in ``/var/lib/bcfg2/Metadata/groups.xml``. The resulting -file should look something like this - -.. note:: - - The reason we are creating a release-specific group in this case is - that the YUMSource above is specific to the 5.4 release of centos. - That is, it should not apply to other releases (5.1, 5.3, etc). +To make these sources apply to our centos 5 clients, we need to modify +our Metadata. Let's add a **centos5** group which inherits a +**centos** group (this should replace the existing **redhat** group) +present in ``/var/lib/bcfg2/Metadata/groups.xml``. The resulting file +should look something like this .. code-block:: xml <Groups version='3.0'> <Group profile='true' public='true' default='true' name='basic'> - <Group name='centos-5.4'/> + <Group name='centos-5'/> </Group> - <Group name='centos-5.4'> + <Group name='centos-5'> <Group name='centos'/> </Group> <Group name='ubuntu'/> @@ -237,7 +236,7 @@ arch group membership. For this, we will make use of the Probes to your plugins line in ``bcfg2.conf`` and create the Probe.:: [root@centos ~]# grep plugins /etc/bcfg2.conf - plugins = Base,Bundler,Cfg,...,Probes + plugins = Bundler,Cfg,...,Probes [root@centos ~]# mkdir /var/lib/bcfg2/Probes [root@centos ~]# cat /var/lib/bcfg2/Probes/groups #!/bin/sh @@ -259,9 +258,8 @@ it with the *yum* package. .. code-block:: xml - [root@centos ~]# cat /var/lib/bcfg2/Bundler/base-packages.xml - <Bundle name='base-packages'> - <Package name='yum'/> + <Bundle> + <Package name='yum'/> </Bundle> You need to reference the bundle from your Metadata. The resulting @@ -271,7 +269,7 @@ profile group might look something like this <Group profile='true' public='true' default='true' name='basic'> <Bundle name='base-packages'/> - <Group name='centos5.4'/> + <Group name='centos5'/> </Group> Now if we run the client, we can see what this has done for us.:: @@ -285,7 +283,7 @@ Now if we run the client, we can see what this has done for us.:: Excluding Packages in global exclude list Finished Loaded tool drivers: - Action Chkconfig POSIX YUMng + Action Chkconfig POSIX YUM Package pam failed verification. Phase: initial @@ -330,7 +328,7 @@ entries?:: Excluding Packages in global exclude list Finished Loaded tool drivers: - Action Chkconfig POSIX YUMng + Action Chkconfig POSIX YUM Extra Package openssh-clients 4.3p2-36.el5_4.4.x86_64. Extra Package libuser 0.54.7-2.1el5_4.1.x86_64. ... @@ -358,22 +356,22 @@ looks like this .. code-block:: xml - <Bundle name='base-packages'> - <Package name='bcfg2-server'/> - <Package name='exim'/> - <Package name='grub'/> - <Package name='kernel'/> - <Package name='krb5-workstation'/> - <Package name='m2crypto'/> - <Package name='openssh-clients'/> - <Package name='openssh-server'/> - <Package name='prelink'/> - <Package name='redhat-lsb'/> - <Package name='rpm-build'/> - <Package name='rsync'/> - <Package name='sysklogd'/> - <Package name='vim-enhanced'/> - <Package name='yum'/> + <Bundle> + <Package name='bcfg2-server'/> + <Package name='exim'/> + <Package name='grub'/> + <Package name='kernel'/> + <Package name='krb5-workstation'/> + <Package name='m2crypto'/> + <Package name='openssh-clients'/> + <Package name='openssh-server'/> + <Package name='prelink'/> + <Package name='redhat-lsb'/> + <Package name='rpm-build'/> + <Package name='rsync'/> + <Package name='sysklogd'/> + <Package name='vim-enhanced'/> + <Package name='yum'/> </Bundle> Now when I run the client, you can see I have only one unmanaged @@ -388,9 +386,7 @@ package:: Excluding Packages in global exclude list Finished Loaded tool drivers: - Action Chkconfig POSIX YUMng - Extra Package gpg-pubkey e8562897-459f07a4.None. - Extra Package gpg-pubkey 217521f6-45e8a532.None. + Action Chkconfig POSIX YUM Phase: initial Correct entries: 187 @@ -404,96 +400,11 @@ package:: Incorrect entries: 0 Total managed entries: 187 Unmanaged entries: 16 - Package:gpg-pubkey Service:atd Service:avahi-daemon Service:bcfg2-server ... -The gpg-pubkey packages are special in that they are not really -packages. Currently, the way to manage them is using :ref:`BoundEntries -<boundentries>`. So, after adding them, our Bundle now looks like this - -.. note:: This does not actually control the contents of the files, - you will need to do this part separately (see below). - -.. code-block:: xml - - <Bundle name='base-packages'> - <BoundPackage name="gpg-pubkey" type="rpm" version="foo"> - <Instance simplefile="/etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-5" version="e8562897" release="459f07a4"/> - <Instance simplefile="/etc/pki/rpm-gpg/RPM-GPG-KEY-EPEL" version="217521f6" release="45e8a532"/> - </BoundPackage> - <Package name='bcfg2-server'/> - <Package name='exim'/> - <Package name='grub'/> - <Package name='kernel'/> - <Package name='krb5-workstation'/> - <Package name='m2crypto'/> - <Package name='openssh-clients'/> - <Package name='openssh-server'/> - <Package name='prelink'/> - <Package name='redhat-lsb'/> - <Package name='rpm-build'/> - <Package name='rsync'/> - <Package name='sysklogd'/> - <Package name='vim-enhanced'/> - <Package name='yum'/> - </Bundle> - -.. note:: - - version="foo" is just a dummy attribute for the gpg-pubkey Package - -To actually push the gpg keys out via Bcfg2, you will need to manage the -files as well. This can be done by adding Path entries for each of the -gpg keys you want to manage - -.. code-block:: xml - - <Bundle name='base-packages'> - <Path name='/etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-5'/> - <Path name='/etc/pki/rpm-gpg/RPM-GPG-KEY-EPEL'/> - <BoundPackage name="gpg-pubkey" type="rpm" version="foo"> - <Instance simplefile="/etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-5" version="e8562897" release="459f07a4"/> - <Instance simplefile="/etc/pki/rpm-gpg/RPM-GPG-KEY-EPEL" version="217521f6" release="45e8a532"/> - </BoundPackage> - <Package name='bcfg2-server'/> - <Package name='exim'/> - <Package name='grub'/> - <Package name='kernel'/> - <Package name='krb5-workstation'/> - <Package name='m2crypto'/> - <Package name='openssh-clients'/> - <Package name='openssh-server'/> - <Package name='prelink'/> - <Package name='redhat-lsb'/> - <Package name='rpm-build'/> - <Package name='rsync'/> - <Package name='sysklogd'/> - <Package name='vim-enhanced'/> - <Package name='yum'/> - </Bundle> - -Then add the files to Cfg:: - - mkdir -p Cfg/etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-5 - cp /etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-5 !$/RPM-GPG-KEY-CentOS-5 - mkdir -p Cfg/etc/pki/rpm-gpg/RPM-GPG-KEY-EPEL - cp /etc/pki/rpm-gpg/RPM-GPG-KEY-EPEL !$/RPM-GPG-KEY-EPEL - -You will also want to add an *important* attribute to these files so -that they are installed on the client prior to any attempts to install -the **gpg-pubkey** rpm packages. This is especially important during the -bootstrapping phase and can be accomplished using an :ref:`server-info` -file that looks like the following: - -.. code-block:: xml - - <FileInfo> - <Info owner='root' group='root' mode='0644' important='true'/> - </FileInfo> - Now, running the client shows only unmanaged Service entries. Woohoo! Manage services @@ -527,22 +438,22 @@ entries to our bundle. [root@centos ~]# cat /var/lib/bcfg2/Rules/services.xml <Rules priority='1'> - <!-- basic services --> - <Service type='chkconfig' status='on' name='atd'/> - <Service type='chkconfig' status='on' name='avahi-daemon'/> - <Service type='chkconfig' status='on' name='bcfg2-server'/> - <Service type='chkconfig' status='on' name='crond'/> - <Service type='chkconfig' status='on' name='cups'/> - <Service type='chkconfig' status='on' name='gpm'/> - <Service type='chkconfig' status='on' name='lvm2-monitor'/> - <Service type='chkconfig' status='on' name='mcstrans'/> - <Service type='chkconfig' status='on' name='messagebus'/> - <Service type='chkconfig' status='on' name='netfs'/> - <Service type='chkconfig' status='on' name='network'/> - <Service type='chkconfig' status='on' name='postfix'/> - <Service type='chkconfig' status='on' name='rawdevices'/> - <Service type='chkconfig' status='on' name='sshd'/> - <Service type='chkconfig' status='on' name='syslog'/> + <!-- basic services --> + <Service type='chkconfig' status='on' name='atd'/> + <Service type='chkconfig' status='on' name='avahi-daemon'/> + <Service type='chkconfig' status='on' name='bcfg2-server'/> + <Service type='chkconfig' status='on' name='crond'/> + <Service type='chkconfig' status='on' name='cups'/> + <Service type='chkconfig' status='on' name='gpm'/> + <Service type='chkconfig' status='on' name='lvm2-monitor'/> + <Service type='chkconfig' status='on' name='mcstrans'/> + <Service type='chkconfig' status='on' name='messagebus'/> + <Service type='chkconfig' status='on' name='netfs'/> + <Service type='chkconfig' status='on' name='network'/> + <Service type='chkconfig' status='on' name='postfix'/> + <Service type='chkconfig' status='on' name='rawdevices'/> + <Service type='chkconfig' status='on' name='sshd'/> + <Service type='chkconfig' status='on' name='syslog'/> </Rules> Now we run the client and see there are no more unmanaged entries!:: @@ -556,7 +467,7 @@ Now we run the client and see there are no more unmanaged entries!:: Excluding Packages in global exclude list Finished Loaded tool drivers: - Action Chkconfig POSIX YUMng + Action Chkconfig POSIX YUM Phase: initial Correct entries: 205 diff --git a/doc/appendix/guides/converging_rhel5.txt b/doc/appendix/guides/converging_rhel5.txt index d6883c778..38d8761cb 100644 --- a/doc/appendix/guides/converging_rhel5.txt +++ b/doc/appendix/guides/converging_rhel5.txt @@ -24,7 +24,8 @@ Unmanaged entries sudo yum remove PACKAGE - #. Otherwise, add ``<Package name="PACKAGE" />`` to the Base or Bundler configuration. + #. Otherwise, add ``<Package name="PACKAGE" />`` to the Bundler + configuration. * Package (dependency) @@ -38,7 +39,7 @@ Unmanaged entries * Service - #. Add ``<Service name="SERVICE" />`` to the Base or Bundler configuration. + #. Add ``<Service name="SERVICE" />`` to the Bundler configuration. #. Add ``<Service name="SERVICE" status="on" type="chkconfig" />`` to ``/var/lib/bcfg2/Rules/services.xml``. @@ -57,8 +58,8 @@ For a "Package" * For example, ``/etc/motd`` to ``/var/lib/bcfg2/Cfg/etc/motd/motd``. Yes, there is an extra directory level named after the file. - #. Specify configuration files as ``<Path name='PATH' />`` in the Base - or Bundler configuration. + #. Specify configuration files as ``<Path name='PATH' />`` in the + Bundler configuration. #. Add directories to ``/var/lib/bcfg2/Rules/directories.xml``. For example: @@ -73,13 +74,13 @@ For a "Package" * Option A: Explicitly list the instances - #. Drop the ``<Package />`` from the Base or Bundler configuration. + #. Drop the ``<Package />`` from the Bundler configuration. #. Add an explicit ``<BoundPackage>`` and ``<Instance />`` configuration to a new Bundle, like the following: .. code-block:: xml - <Bundle name='keys'> + <Bundle> <!-- GPG keys --> <BoundPackage name="gpg-pubkey" type="rpm" version="foo"> <Instance simplefile="/etc/pki/rpm-gpg/RPM-GPG-KEY-EPEL" version="217521f6" release="45e8a532"/> diff --git a/doc/appendix/guides/fedora.txt b/doc/appendix/guides/fedora.txt deleted file mode 100644 index 1e49084ef..000000000 --- a/doc/appendix/guides/fedora.txt +++ /dev/null @@ -1,492 +0,0 @@ -.. -*- mode: rst -*- - -.. This guide is based on the Centos guide. - -.. _guide-fedora: - -====== -Fedora -====== - -This guide is work in progess. - - -This is a complete getting started guide for Fedora. With this -document you should be able to install a Bcfg2 server, a Bcfg2 client, -and change the ``/etc/motd`` file on the client. - -Prerequisites -============= - -To setup a configuration management system based on Bcfg2 only a few -prerequisites need to be fullfilled. - -* A server machine that can host the Bcfg2 -* Internet access for the installation process -* A working network with DNS - - -Install Bcfg2 From RPM -====================== - -The fastest way to get Bcfg2 onto your system is to use ``yum`` -or PackageKit. ``yum`` will pull all dependencies of Bcfg2 -automatically in. :: - - $ su -c 'yum install bcfg2-server bcfg2' - -Your system should now have the necessary software to use Bcfg2. -The next step is to set up your Bcfg2 :term:`repository`. - - -Initialize your repository -========================== - -Now that you're done with the install, you need to initialize your -repository and setup your ``/etc/bcfg2.conf``. ``bcfg2-admin init`` -is a tool which allows you to automate this: - -.. code-block:: sh - - # bcfg2-admin init - Store bcfg2 configuration in [/etc/bcfg2.conf]: - Location of bcfg2 repository [/var/lib/bcfg2]: - Directory /var/lib/bcfg2 exists. Overwrite? [y/N]:y - Input password used for communication verification (without echoing; leave blank for a random): - What is the server's hostname: [config01.local.net] - Input the server location [https://config01.local.net:6789]: - Input base Operating System for clients: - 1: Red Hat/Fedora/RHEL/RHAS/Centos - 2: SUSE/SLES - 3: Mandrake - 4: Debian - 5: Ubuntu - 6: Gentoo - 7: FreeBSD - : 1 - Generating a 1024 bit RSA private key - .......................................................++++++ - .....++++++ - writing new private key to '/etc/bcfg2.key' - ----- - Signature ok - subject=/C=US/ST=Illinois/L=Argonne/CN=config01.local.net - Getting Private key - Repository created successfuly in /var/lib/bcfg2 - -Change responses as necessary. - -Start the server -================ - -You are now ready to start your Bcfg2 server for the first time:: - - $ su -c '/etc/init.d/bcfg2-server start' - Starting Configuration Management Server: bcfg2-server [ OK ] - -To verify that everything started ok, look for the running daemon and -check the logs: - -.. code-block:: sh - - $ su -c 'tail /var/log/messages' - May 16 14:14:57 config01 bcfg2-server[2746]: service available at https://config01.local.net:6789 - May 16 14:14:57 config01 bcfg2-server[2746]: serving bcfg2-server at https://config01.local.net:6789 - May 16 14:14:57 config01 bcfg2-server[2746]: serve_forever() [start] - May 16 14:14:57 config01 bcfg2-server[2746]: Handled 16 events in 0.009s - - -Run ``bcfg2`` to be sure you are able to communicate with the server: - -.. code-block:: sh - - $ su -c 'bcfg2 -vqne' - - /usr/lib/python2.6/site-packages/Bcfg2/Client/Tools/rpmtools.py:23: DeprecationWarning: the md5 module is deprecated; use hashlib instead - import md5 - Loaded plugins: presto, refresh-packagekit - Loaded tool drivers: - Action Chkconfig POSIX YUMng - Extra Package imsettings-libs 0.108.0-2.fc13.i686. - Extra Package PackageKit-device-rebind 0.6.4-1.fc13.i686. - ... - Extra Package newt-python 0.52.11-2.fc13.i686. - Extra Package pulseaudio-gdm-hooks 0.9.21-6.fc13.i686. - - Phase: initial - Correct entries: 0 - Incorrect entries: 0 - Total managed entries: 0 - Unmanaged entries: 1314 - - - Phase: final - Correct entries: 0 - Incorrect entries: 0 - Total managed entries: 0 - Unmanaged entries: 1314 - Package:ConsoleKit Package:jasper-libs Package:pcsc-lite-libs - Package:ConsoleKit-libs Package:java-1.5.0-gcj Package:perf - ... - Package:iw Package:pcre Service:sshd - Package:jack-audio-connection-kit Package:pcsc-lite Service:udev-post - -The ``bcfg2.conf`` file contains only standard plugins so far. - -.. code-block:: sh - - $ su -c 'cat /etc/bcfg2.conf' - - [server] - repository = /var/lib/bcfg2 - plugins = SSHbase,Cfg,Pkgmgr,Rules,Metadata,Base,Bundler - - [statistics] - sendmailpath = /usr/lib/sendmail - - [database] - engine = sqlite3 - # 'postgresql', 'mysql', 'mysql_old', 'sqlite3' or 'ado_mssql'. - name = - # Or path to database file if using sqlite3. - #<repository>/etc/brpt.sqlite is default path if left empty - user = - # Not used with sqlite3. - password = - # Not used with sqlite3. - host = - # Not used with sqlite3. - port = - - [communication] - protocol = xmlrpc/ssl - password = test1234 - certificate = /etc/bcfg2.crt - key = /etc/bcfg2.key - ca = /etc/bcfg2.crt - - [components] - bcfg2 = https://config01.local.net:6789 - - -Add the machines to Bcfg2 -------------------------- - -``bcfg2-admin`` can be used to add a machine to Bcfg2 easily. You -need to know the Fully Qualified Domain Name (FQDN) of ever system -you want to control through Bcfg2. :: - - bcfg2-admin client add <FQDN machine> - -Bring your first machine under Bcfg2 control --------------------------------------------- - -Now it is time to get the first machine's configuration into the -Bcfg2 repository. The server will be the first machine. It's -already in the ``Metadata/client.xml``. - - -Setup the :ref:`server-plugins-generators-packages` plugin -++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ - -First, replace **Pkgmgr** with **Packages** in the plugins -line of ``bcfg2.conf``. Then create a `Packages/` directory in -``/var/lib/bcfg2`` :: - - $ su -c 'mkdir /var/lib/bcfg2/Packages' - -Create a ``packages.conf`` in the ``/var/lib/bcfg2/Packages`` directory -with the following contents:: - - [global] - -Create a ``sources.xml`` file for the packages in -``/var/lib/bcfg2/Packages`` with the following content. Choose a mirror -near your location according the `Mirror list`_ . - -.. _Mirror list: http://mirrors.fedoraproject.org/publiclist/ - -.. code-block:: xml - - <Sources> - <Group name="fedora-13"> - <Source type="yum" url="ftp://fedora.tu-chemnitz.de/pub/linux/fedora/linux/releases/" version="13"> - <Component>Fedora</Component> - <Arch>i386</Arch> - <Arch>x86_64</Arch> - <Source> - </Group> - </Sources> - - -Due to the :ref:`server-plugins-generators-packages-magic-groups`, -we need to modify our Metadata. Let's add a **fedora13** group which -inherits a **fedora** group (this should replace the existing **redhat** -group) present in ``/var/lib/bcfg2/Metadata/groups.xml``. The resulting -file should look something like this - -.. note:: - - The reason we are creating a release-specific group in this case is - that the YUMSource above is specific to the 13th release of fedora. - That is, it should not apply to other releases (14, 15, etc). - -.. code-block:: xml - - <Groups version='3.0'> - <Group profile='true' public='true' default='true' name='basic'> - <Group name='fedora13'/> - </Group> - <Group name='fedora13'/> - <Group name='fedora'/> - <Group name='ubuntu'/> - <Group name='debian'/> - <Group name='freebsd'/> - <Group name='gentoo'/> - <Group name='fedora'/> - <Group name='suse'/> - <Group name='mandrake'/> - <Group name='solaris'/> - </Groups> - -.. note:: - When editing your xml files by hand, it is useful to occasionally - run ``bcfg2-lint`` to ensure that your xml validates properly. - -Add a probe -+++++++++++ - -The next step for the client will be to have the proper -arch group membership. For this, we will make use of the -:ref:`unsorted-dynamic_groups` capabilities of the Probes plugin. Add -**Probes** to your plugins line in ``bcfg2.conf`` and create the Probe: - -.. code-block:: sh - - $ su -c 'mkdir /var/lib/bcfg2/Probes' - $ su -c 'cat /var/lib/bcfg2/Probes/groups' - #!/bin/sh - - echo "group:`uname -m`" - -Now a restart of ``bcfg2-server`` is needed:: - - $ su -c '/etc/init.d/bcfg2-server restart' - -To test the Probe just run ``bcfg2 -vqn``. - -.. code-block:: xml - - $ su -c 'bcfg2 -vqn' - Running probe group - Probe group has result: - group:i686 - ... - -Start managing packages -+++++++++++++++++++++++ - -Add a base-packages bundle. Let's see what happens when we just populate -it with the *yum* package. Create the ``base-packages.xml`` in your -``Bundler/`` directory with a entry for ``yum``. - -.. code-block:: xml - - $ cat /var/lib/bcfg2/Bundler/base-packages.xml - <Bundle name='base-packages'> - <Package name='yum'/> - </Bundle> - -You need to reference the bundle from your ``group.xml``. The resulting -profile group might look something like this - -.. code-block:: xml - - <Group profile='true' public='true' default='true' name='basic'> - <Bundle name='base-packages'/> - <Group name='fedora13'/> - </Group> - -Now if we run the client, we can see what this has done for us.:: - - output - -As you can see, the Packages plugin has generated the dependencies -required for the yum package automatically. The ultimate goal should -be to move all the packages from the **Unmanaged** entries section -to the **Managed** entries section. So, what exactly *are* those -Unmanaged entries?:: - - output - -Now you can go through these and continue adding the packages you -want to your Bundle. After a while, I ended up with a minimal bundle -that looks like this - -.. code-block:: xml - - <Bundle name='base-packages'> - - </Bundle> - -Now when I run the client, you can see I have only one unmanaged -package:: - - outout - -The gpg-pubkey packages are special in that they are not really -packages. Currently, the way to manage them is using -:ref:`BoundEntries <boundentries>`. So, after adding them, our -Bundle now looks like this - -.. note:: This does not actually control the contents of the files, - you will need to do this part separately (see below). - -.. code-block:: xml - - <Bundle name='base-packages'> - <BoundPackage name="gpg-pubkey" type="rpm" version="foo"> - <Instance simplefile="/etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-5" version="e8562897" release="459f07a4"/> - <Instance simplefile="/etc/pki/rpm-gpg/RPM-GPG-KEY-EPEL" version="217521f6" release="45e8a532"/> - </BoundPackage> - <Package name='bcfg2-server'/> - <Package name='exim'/> - <Package name='grub'/> - <Package name='kernel'/> - <Package name='krb5-workstation'/> - <Package name='m2crypto'/> - <Package name='openssh-clients'/> - <Package name='openssh-server'/> - <Package name='prelink'/> - <Package name='redhat-lsb'/> - <Package name='rpm-build'/> - <Package name='rsync'/> - <Package name='sysklogd'/> - <Package name='vim-enhanced'/> - <Package name='yum'/> - </Bundle> - -.. note:: - - version="foo" is just a dummy attribute for the gpg-pubkey Package - -To actually push the gpg keys out via Bcfg2, you will need to manage -the files as well. This can be done by adding Path entries for each -of the gpg keys you want to manage - -.. code-block:: xml - - <Bundle name='base-packages'> - <Path name='/etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-5'/> - <Path name='/etc/pki/rpm-gpg/RPM-GPG-KEY-EPEL'/> - <BoundPackage name="gpg-pubkey" type="rpm" version="foo"> - <Instance simplefile="/etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-5" version="e8562897" release="459f07a4"/> - <Instance simplefile="/etc/pki/rpm-gpg/RPM-GPG-KEY-EPEL" version="217521f6" release="45e8a532"/> - </BoundPackage> - <Package name='bcfg2-server'/> - <Package name='exim'/> - <Package name='grub'/> - <Package name='kernel'/> - <Package name='krb5-workstation'/> - <Package name='m2crypto'/> - <Package name='openssh-clients'/> - <Package name='openssh-server'/> - <Package name='prelink'/> - <Package name='redhat-lsb'/> - <Package name='rpm-build'/> - <Package name='rsync'/> - <Package name='sysklogd'/> - <Package name='vim-enhanced'/> - <Package name='yum'/> - </Bundle> - -Then add the files to Cfg:: - - mkdir -p Cfg/etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-5 - cp /etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-5 !$/RPM-GPG-KEY-CentOS-5 - mkdir -p Cfg/etc/pki/rpm-gpg/RPM-GPG-KEY-EPEL - cp /etc/pki/rpm-gpg/RPM-GPG-KEY-EPEL !$/RPM-GPG-KEY-EPEL - -Now, running the client shows only unmanaged Service entries. Woohoo! - -Manage services -+++++++++++++++ - -Now let's clear up the unmanaged service entries by adding the -following entries to our bundle... - -.. code-block:: xml - - <!-- basic services --> - <Service name='atd'/> - <Service name='avahi-daemon'/> - <Service name='bcfg2-server'/> - <Service name='crond'/> - <Service name='cups'/> - <Service name='gpm'/> - <Service name='lvm2-monitor'/> - <Service name='mcstrans'/> - <Service name='messagebus'/> - <Service name='netfs'/> - <Service name='network'/> - <Service name='postfix'/> - <Service name='rawdevices'/> - <Service name='sshd'/> - <Service name='syslog'/> - -...and bind them in Rules - -.. code-block:: xml - - [root@centos ~]# cat /var/lib/bcfg2/Rules/services.xml - <Rules priority='1'> - <!-- basic services --> - <Service type='chkconfig' status='on' name='atd'/> - <Service type='chkconfig' status='on' name='avahi-daemon'/> - <Service type='chkconfig' status='on' name='bcfg2-server'/> - <Service type='chkconfig' status='on' name='crond'/> - <Service type='chkconfig' status='on' name='cups'/> - <Service type='chkconfig' status='on' name='gpm'/> - <Service type='chkconfig' status='on' name='lvm2-monitor'/> - <Service type='chkconfig' status='on' name='mcstrans'/> - <Service type='chkconfig' status='on' name='messagebus'/> - <Service type='chkconfig' status='on' name='netfs'/> - <Service type='chkconfig' status='on' name='network'/> - <Service type='chkconfig' status='on' name='postfix'/> - <Service type='chkconfig' status='on' name='rawdevices'/> - <Service type='chkconfig' status='on' name='sshd'/> - <Service type='chkconfig' status='on' name='syslog'/> - </Rules> - -Now we run the client and see there are no more unmanaged entries! :: - - $ su -c 'bcfg2 -veqn' - - -Adding Plugins -++++++++++++++ - -Git ---- - -.. _Git tutorial: http://www.kernel.org/pub/software/scm/git/docs/gittutorial.html - -Adding the :ref:`server-plugins-version-git` plugins can preserve -versioning information. The first step is to add *Git* to your -plugin line:: - - plugins = Base,Bundler,Cfg,...,Git - -For tracking the configuration files in the ``/var/lib/bcfg2`` -directory a git repository need to be established:: - - git init - -For more detail about the setup of git please refer to a `git tutorial`_. -The first commit can be the empty or the allready populated directory:: - - git add . && git commit -a - -While running ``bcfg2-info`` the following line will show up:: - - Initialized git plugin with git directory = /var/lib/bcfg2/.git diff --git a/doc/appendix/guides/import-existing-ssh-keys.txt b/doc/appendix/guides/import-existing-ssh-keys.txt index 64a1b62cd..6ce41ba60 100644 --- a/doc/appendix/guides/import-existing-ssh-keys.txt +++ b/doc/appendix/guides/import-existing-ssh-keys.txt @@ -21,10 +21,11 @@ Add a bundle for ssh After verifying that SSHbase is listed on the plugins line in ``/etc/bcfg2.conf``, you need to create a bundle containing the -appropriate entries.:: +appropriate entries. - cat > /tmp/ssh.xml << EOF - <Bundle name='ssh'> +.. code-block:: xml + + <Bundle> <Path name='/etc/ssh/ssh_host_dsa_key'/> <Path name='/etc/ssh/ssh_host_rsa_key'/> <Path name='/etc/ssh/ssh_host_dsa_key.pub'/> @@ -34,10 +35,6 @@ appropriate entries.:: <Path name='/etc/ssh/ssh_known_hosts'/> </Bundle> -:: - - mv /tmp/ssh.xml /var/lib/bcfg2/Bundle - Next, you need to add the ssh bundle to the client's metadata in groups.xml. diff --git a/doc/appendix/guides/sslca_howto.txt b/doc/appendix/guides/sslca_howto.txt new file mode 100644 index 000000000..9c939dcd3 --- /dev/null +++ b/doc/appendix/guides/sslca_howto.txt @@ -0,0 +1,183 @@ +.. -*- mode: rst -*- + +.. _appendix-guides-sslca_howto: + +==================================== + Automated Bcfg2 SSL Authentication +==================================== + +This how-to describes one possible scenario for automating SSL +certificate generation and distribution for bcfg2 client/server +communication using the :ref:`SSL CA feature +<server-plugins-generators-cfg-ssl-certificates>` of +:ref:`server-plugins-generators-cfg`. The process involves configuring +a certificate authority (CA), generating the CA cert and key pair, +configuring the Cfg SSL CA feature and a Bundle to use the generated +certs to authenticate the Bcfg2 client and server. + +OpenSSL CA +========== + +If you already have a SSL CA available you can skip this section, +otherwise you can easily build one on the server using openssl. The +paths should be adjusted to suite your preferences. + +#. Prepare the directories and files:: + + mkdir -p /etc/pki/CA/newcerts + mkdir /etc/pki/CA/crl + echo '01' > /etc/pki/CA/serial + touch /etc/pki/CA/index.txt + touch /etc/pki/CA/crlnumber + +#. Edit the ``openssl.cnf`` config file, and in the **[ CA_default ]** + section adjust the following parameters:: + + dir = /etc/pki # Where everything is kept + certs = /etc/pki/CA/certs # Where the issued certs are kept + database = /etc/pki/CA/index.txt # database index file. + new_certs_dir = /etc/pki/CA/newcerts # default place for new certs. + certificate = /etc/pki/CA/certs/bcfg2ca.crt # The CA certificate + serial = /etc/pki/CA/serial # The current serial number + crl_dir = /etc/pki/CA/crl # Where the issued crl are kept + crlnumber = /etc/pki/CA/crlnumber # the current crl number + crl = /etc/pki/CA/crl.pem # The current CRL + private_key = /etc/pki/CA/private/bcfg2ca.key # The private key + +#. Create the CA root certificate and key pair. You'll be asked to + supply a passphrase, and some organizational info. The most + important bit is **Common Name** which you should set to be the + hostname of your bcfg2 server that your clients will see when doing + a reverse DNS query on it's ip address.:: + + openssl req -new -x509 -extensions v3_ca -keyout bcfg2ca.key \ + -out bcfg2ca.crt -days 3650 + +#. Move the generated cert and key to the locations specified in + ``openssl.cnf``:: + + mv bcfg2ca.key /etc/pki/CA/private/ + mv bcfg2ca.crt /etc/pki/CA/certs/ + +Your self-signing CA is now ready to use. + +Bcfg2 +===== + +SSL CA Feature +-------------- + +The SSL CA feature of Cfg was not designed specifically to manage +Bcfg2 client/server communication, though it is certainly able to +provide certificate generation and management services for that +purpose. You'll need to configure Cfg as described in +:ref:`server-plugins-generators-cfg-ssl-certificates`, including: + +* Configuring a ``[sslca_default]`` section in ``bcfg2.conf`` that + describes the CA you created above; +* Creating ``Cfg/etc/pki/tls/certs/bcfg2client.crt/sslcert.xml`` and + ``Cfg/etc/pki/tls/private/bcfg2client.key/sslkey.xml`` to describe + the key and cert you want generated. + +In general, the defaults in ``sslcert.xml`` and ``sslkey.xml`` should +be fine, so those files can look like this: + +``Cfg/etc/pki/tls/certs/bcfg2client.crt/sslcert.xml``: + +.. code-block:: xml + + <CertInfo> + <Cert key="/etc/pki/tls/private/bcfg2client.key"/> + </CertInfo> + +``Cfg/etc/pki/tls/private/bcfg2client.key/sslkey.xml``: + +.. code-block:: xml + + <KeyInfo/> + +Client Bundle +------------- + +To automate the process of generating and distributing certs to the +clients we need define at least the cert and key paths created by Cfg, +as well as the CA certificate path in a Bundle. For example: + +.. code-block:: xml + + <Path name='/etc/pki/tls/certs/bcfg2ca.crt'/> + <Path name='/etc/pki/tls/bcfg2client.crt'/> + <Path name='/etc/pki/tls/private/bcfg2client.key'/> + +Here's a more complete example bcfg2-client bundle: + +.. code-block:: xml + + <Bundle> + <Path name='/etc/bcfg2.conf'/> + <Path name='/etc/cron.d/bcfg2-client'/> + <Package name='bcfg2'/> + <Service name='bcfg2'/> + <Group name='rpm'> + <Path name='/etc/sysconfig/bcfg2'/> + <Path name='/etc/pki/tls/certs/bcfg2ca.crt'/> + <Path name='/etc/pki/tls/certs/bcfg2client.crt'/> + <Path name='/etc/pki/tls/private/bcfg2client.key'/> + </Group> + <Group name='deb'> + <Path name='/etc/default/bcfg2' altsrc='/etc/sysconfig/bcfg2'/> + <Path name='/etc/ssl/certs/bcfg2ca.crt' altsrc='/etc/pki/tls/certs/bcfg2ca.crt'/> + <Path name='/etc/ssl/certs/bcfg2client.crt' altsrc='/etc/pki/tls/certs/bcfg2client.crt'/> + <Path name='/etc/ssl/private/bcfg2client.key' altsrc='/etc/pki/tls/private/bcfg2client.key'/> + </Group> + </Bundle> + +The ``bcfg2.conf`` client config needs at least 5 parameters set for +SSL auth. + +#. ``key`` : This is the host specific key that Cfg will create. +#. ``certificate`` : This is the host specific cert that Cfg will + create. +#. ``ca`` : This is a copy of your CA certificate. Not generated by + Cfg. +#. ``password`` : Set to arbitrary string when using certificate + auth. This also *shouldn't* be required. See: + http://trac.mcs.anl.gov/projects/bcfg2/ticket/1019 + +Here's what a functional **[communication]** section in a +``bcfg2.conf`` genshi template for clients might look like.:: + + [communication] + protocol = xmlrpc/ssl + {% if metadata.uuid != None %}\ + user = ${metadata.uuid} + {% end %}\ + password = DUMMYPASSWORDFORCERTAUTH + {% choose %}\ + {% when 'rpm' in metadata.groups %}\ + certificate = /etc/pki/tls/certs/bcfg2client.crt + key = /etc/pki/tls/private/bcfg2client.key + ca = /etc/pki/tls/certs/bcfg2ca.crt + {% end %}\ + {% when 'deb' in metadata.groups %}\ + certificate = /etc/ssl/certs/bcfg2client.crt + key = /etc/ssl/private/bcfg2client.key + ca = /etc/ssl/certs/bcfg2ca.crt + {% end %}\ + {% end %}\ + +As a client will not be able to authenticate with certificates it does +not yet posses we need to overcome the chicken and egg scenario the +first time we try to connect such a client to the server. We can do so +using password based auth to bootstrap the client manually specifying +all the relevant auth parameters like so:: + + bcfg2 -qv -S https://fqdn.of.bcfg2-server:6789 -u fqdn.of.client \ + -x SUPER_SECRET_PASSWORD + +If all goes well the client should recieve a freshly generated key and +cert and you should be able to run ``bcfg2`` again without specifying +the connection parameters. + +If you do run into problems you may want to review +:ref:`appendix-guides-authentication`. diff --git a/doc/appendix/guides/ubuntu.txt b/doc/appendix/guides/ubuntu.txt index f68c8b9ad..60f8e3a41 100644 --- a/doc/appendix/guides/ubuntu.txt +++ b/doc/appendix/guides/ubuntu.txt @@ -495,7 +495,7 @@ like this: .. code-block:: xml - <Bundle name='base-saucy'> + <Bundle> <!-- packages --> <Package name='bcfg2-server'/> <!-- or dependencies --> diff --git a/doc/appendix/guides/vcs.txt b/doc/appendix/guides/vcs.txt index 6c2879a65..fba61e722 100644 --- a/doc/appendix/guides/vcs.txt +++ b/doc/appendix/guides/vcs.txt @@ -30,7 +30,7 @@ While running ``bcfg2-info`` the following line will show up:: Initialized git plugin with git directory = /var/lib/bcfg2/.git -Mercurial +Mercurial ========= The :ref:`server-plugins-version-hg` plugin also allows you to store @@ -59,7 +59,7 @@ While running ``bcfg2-info`` the following line will show up:: Initialized hg plugin with hg directory = /var/lib/bcfg2/.hg -Darcs +Darcs ===== The :ref:`server-plugins-version-darcs` plugin also allows you to store @@ -70,8 +70,8 @@ be initialized:: darcs initialize -To commit to the darcs repository an author must be added to the -``_darcs/prefs/author`` file. If the ``author`` file is missing, +To commit to the darcs repository an author must be added to the +``_darcs/prefs/author`` file. If the ``author`` file is missing, darcs will ask you to enter your e-mail address. .. code-block:: sh @@ -99,7 +99,7 @@ Cvs The :ref:`server-plugins-version-cvs` plugin also allows you to store version information in the statistics database. - plugins = Base,Bundler,Cfg,...,Cvs + plugins = Bundler,Cfg,...,Cvs The CVS repository must be initialized:: diff --git a/doc/appendix/guides/web-reports-install.txt b/doc/appendix/guides/web-reports-install.txt index f03bad289..06932efc9 100644 --- a/doc/appendix/guides/web-reports-install.txt +++ b/doc/appendix/guides/web-reports-install.txt @@ -28,7 +28,7 @@ Add Reporting to the plugins line of ``bcfg2.conf``. The resulting [server] repository = /var/lib/bcfg2 - plugins = Base,Bundler,Cfg,...,Reporting + plugins = Bundler,Cfg,...,Reporting [reporting] transport = LocalFilesystem @@ -53,7 +53,7 @@ then have something like this:: [server] repository = /var/lib/bcfg2 - plugins = Base,Bundler,Cfg,...,Reporting + plugins = Bundler,Cfg,...,Reporting [database] engine = sqlite3 diff --git a/doc/client/tools.txt b/doc/client/tools.txt index 1dbb33b1a..09ea76230 100644 --- a/doc/client/tools.txt +++ b/doc/client/tools.txt @@ -133,8 +133,6 @@ RPM Executes RPM to manage packages on Redhat-based and similar systems. Consider using the :ref:`YUM <client-tools-yum>` tool instead if possible. -Formerly called ``RPMng``, but was renamed for the 1.3 release. - SMF --- @@ -166,13 +164,5 @@ Upstart service support. Uses `Upstart`_ to configure services. YUM --- -Handles RPMs using the YUM package manager. Renamed from ``YUMng`` for -the 1.3 release. See :ref:`client-tools-yum` for more details. - -YUM24 ------ - -.. warning:: Deprecated in favor of :ref:`YUM <client-tools-yum>` - -Handles RPMs using older versions of the YUM package manager. - +Handles RPMs using the YUM package manager. See +:ref:`client-tools-yum` for more details. diff --git a/doc/client/tools/actions.txt b/doc/client/tools/actions.txt index e5fdb1f39..52d07eb4f 100644 --- a/doc/client/tools/actions.txt +++ b/doc/client/tools/actions.txt @@ -28,18 +28,17 @@ so they can be centrally observed. Actions look like: Note that the status attribute tells the bcfg2 client to ignore return status, causing failures to still not be centrally reported. If central reporting of action failure is desired, set this attribute to -'check'. Also note that Action entries included in Base will not be -executed. +'check'. -Actions may be completely defined inside of a bundle with the use of -:ref:`server-configurationentries`, much like Packages, Services or Paths. -The Rules plugin can also bind these entries. For example to include the -above action in a bundle, first the Action entry must be included in the +Actions may be completely defined inside of a bundle with the use of +:ref:`server-configurationentries`, much like Packages, Services or Paths. +The Rules plugin can also bind these entries. For example to include the +above action in a bundle, first the Action entry must be included in the bundle: .. code-block:: xml - <Bundle name='bundle_name'> + <Bundle> ... <Action name='action_name'/> </Bundle> @@ -56,6 +55,16 @@ Then a corresponding entry must be included in the Rules directory, like: This allows different clients to get different actions as a part of the same bundle based on group membership. +It is also possible to do this in one step in the bundle itself with a +``BoundAction`` tag, e.g.: + +.. code-block:: xml + + <Bundle> + <BoundAction timing='post' when='modified' name='action_name' + command='/path/to/command arg1 arg2' status='ignore'/> + </Rules> + Example Action (add APT keys) ============================= diff --git a/doc/client/tools/yum.txt b/doc/client/tools/yum.txt index 47ef3d5e9..ed1a3d5fd 100644 --- a/doc/client/tools/yum.txt +++ b/doc/client/tools/yum.txt @@ -7,9 +7,7 @@ Bcfg2 RPM/YUM Client Drivers ============================ The RPM and YUM client drivers provide client support for RPMs -(installed directly from URLs) and Yum repositories. These drivers -were formerly called ``RPMng`` and ``YUMng``, respectively, but were -renamed for Bcfg2 1.3.0. +(installed directly from URLs) and Yum repositories. Features ======== diff --git a/doc/development/caching.txt b/doc/development/caching.txt new file mode 100644 index 000000000..47d627278 --- /dev/null +++ b/doc/development/caching.txt @@ -0,0 +1,73 @@ +.. -*- mode: rst -*- + +.. _development-cache: + +============================ + Server-side Caching System +============================ + +.. versionadded:: 1.4.0 + +Bcfg2 caches two kinds of data: + +* The contents of all files that it reads in, including (often) an + optimized representation. E.g., XML files are cached both in their + raw (text) format, and also as :class:`lxml.etree._Element` objects. +* Arbitrary data, in the server-side caching system documented on this + page. + +The caching system keeps a single unified cache with all cache data in +it. Each individual datum stored in the cache is associated with any +number of "tags" -- simple terms that uniquely identify the datum. +This lets you very easily expire related data from multiple caches at +once; for isntance, for expiring all data related to a host: + +.. code-block:: python + + Bcfg2.Server.Cache.expire("foo.example.com") + +This would expire *all* data related to ``foo.example.com``, +regardless of which plugin cached it, and so on. + +This permits a high level of interoperation between different plugins +and the cache, which is necessary due to the wide distribution of data +in Bcfg2 and the many different data sources that can be incorported. +More technical details about writing code that uses the caches is below. + +Currently known caches are: + +.. currentmodule:: Bcfg2.Server.Plugins.Packages.Collection + ++-------------+---------------------------------------+-------------------------------------------------+------------------------------------------------------+ +| Tags | Key(s) | Values | Use | ++=============+=======================================+=================================================+======================================================+ +| Metadata | Hostname | :class:`ClientMetadata | The :ref:`Metadata cache <server-caching>` | +| | | <Bcfg2.Server.Plugins.Metadata.ClientMetadata>` | | ++-------------+---------------------------------------+-------------------------------------------------+------------------------------------------------------+ +| Probes, | Hostname | ``list`` of group names | Groups set by :ref:`server-plugins-probes-index` | +| probegroups | | | | ++-------------+---------------------------------------+-------------------------------------------------+------------------------------------------------------+ +| Probes, | Hostname | ``dict`` of ``<probe name>``: | Other data set by :ref:`server-plugins-probes-index` | +| probedata | | :class:`ProbeData | | +| | | <Bcfg2.Server.Plugins.Probes.ProbeData>` | | ++-------------+---------------------------------------+-------------------------------------------------+------------------------------------------------------+ +| Packages, | :attr:`Packages Collection cache key | :class:`Collection` | Kept by :ref:`server-plugins-generators-packages` in | +| collections | <Collection.cachekey>` | | order to expire repository metadata cached on disk | ++-------------+---------------------------------------+-------------------------------------------------+------------------------------------------------------+ +| Packages, | Hostname | :attr:`Packages Collection cache key | Used by the Packages plugin to return Collection | +| clients | | <Collection.cachekey>` | objects for clients. This is cross-referenced with | +| | | | the ``Packages, collections`` cache | ++-------------+---------------------------------------+-------------------------------------------------+------------------------------------------------------+ +| Packages, | :attr:`Packages Collection cache key | ``set`` of package names | Cached results from looking up | +| pkg_groups | <Collection.cachekey>`, | | ``<Package group="..."/>`` entries | +| | hash of the selected package groups | | | ++-------------+---------------------------------------+-------------------------------------------------+------------------------------------------------------+ +| Packages, | :attr:`Packages Collection cache key | ``set`` of package names | Cached results from resolving complete package sets | +| pkg_sets | <Collection.cachekey>`, | | for clients | +| | hash of the initial package selection | | | ++-------------+---------------------------------------+-------------------------------------------------+------------------------------------------------------+ + +These are enumerated so that they can be expired as needed by other +plugins or other code points. + +.. automodule:: Bcfg2.Server.Cache diff --git a/doc/development/cfg.txt b/doc/development/cfg.txt index 6533e0d7a..f93bb42c7 100644 --- a/doc/development/cfg.txt +++ b/doc/development/cfg.txt @@ -55,12 +55,6 @@ exceptions: .. autoexception:: Bcfg2.Server.Plugin.exceptions.PluginInitError :noindex: -Global Variables -================ - -.. autodata:: Bcfg2.Server.Plugins.Cfg.SETUP -.. autodata:: Bcfg2.Server.Plugins.Cfg.CFG - Existing Cfg Handlers ===================== @@ -81,18 +75,11 @@ Creators .. autoclass:: Bcfg2.Server.Plugins.Cfg.CfgPrivateKeyCreator.CfgPrivateKeyCreator .. autoclass:: Bcfg2.Server.Plugins.Cfg.CfgPublicKeyCreator.CfgPublicKeyCreator -Filters -------- - -.. autoclass:: Bcfg2.Server.Plugins.Cfg.CfgCatFilter.CfgCatFilter -.. autoclass:: Bcfg2.Server.Plugins.Cfg.CfgDiffFilter.CfgDiffFilter - Info Handlers ------------- .. autoclass:: Bcfg2.Server.Plugins.Cfg.CfgDefaultInfo .. autoclass:: Bcfg2.Server.Plugins.Cfg.CfgInfoXML.CfgInfoXML -.. autoclass:: Bcfg2.Server.Plugins.Cfg.CfgLegacyInfo.CfgLegacyInfo Verifiers --------- @@ -105,6 +92,6 @@ Other Cfg Objects These other objects comprise the remainder of the Cfg plugin, and are included for completeness. -.. autoattribute:: Bcfg2.Server.Plugins.Cfg.DEFAULT_INFO .. autoclass:: Bcfg2.Server.Plugins.Cfg.CfgEntrySet .. autoclass:: Bcfg2.Server.Plugins.Cfg.Cfg +.. automethod:: Bcfg2.Server.Plugins.Cfg.get_cfg diff --git a/doc/development/core.txt b/doc/development/core.txt index 886a5538b..f5cc7de67 100644 --- a/doc/development/core.txt +++ b/doc/development/core.txt @@ -10,8 +10,10 @@ Bcfg2 1.3 added a pluggable server core system so that the server core itself can be easily swapped out to use different technologies. It -currently ships with two backends: a builtin core written from scratch -using the various server tools in the Python standard library; and an +currently ships with several backends: a builtin core written from +scratch using the various server tools in the Python standard library; +a variant on the builtin core that uses Python 2.6's +:mod:`multiprocessing` library to process requests in parallel; and an experimental `CherryPy <http://www.cherrypy.org/>`_ based core. This page documents the server core interface so that other cores can be written to take advantage of other technologies, e.g., `Tornado @@ -20,20 +22,25 @@ written to take advantage of other technologies, e.g., `Tornado A core implementation needs to: -* Override :func:`Bcfg2.Server.Core.BaseCore._daemonize` to handle - daemonization, writing the PID file, and dropping privileges. -* Override :func:`Bcfg2.Server.Core.BaseCore._run` to handle server +* Override :func:`Bcfg2.Server.Core.Core._run` to handle server startup. -* Override :func:`Bcfg2.Server.Core.BaseCore._block` to run the +* Override :func:`Bcfg2.Server.Core.Core._block` to run the blocking server loop. -* Call :func:`Bcfg2.Server.Core.BaseCore.shutdown` on orderly +* Call :func:`Bcfg2.Server.Core.Core.shutdown` on orderly shutdown. +A core that wants to use the network (i.e., a core that isn't used +entirely for introspection, as in :ref:`bcfg2-info +<server-bcfg2-info>`, or other local tasks) should inherit from +:class:`Bcfg2.Server.Core.NetworkCore`, and must also override +:func:`Bcfg2.Server.Core.NetworkCore._daemonize` to handle daemonization, +writing the PID file, and dropping privileges. + Nearly all XML-RPC handling is delegated entirely to the core implementation. It needs to: -* Call :func:`Bcfg2.Server.Core.BaseCore.authenticate` to authenticate - clients. +* Call :func:`Bcfg2.Server.Core.NetworkCore.authenticate` to + authenticate clients. * Handle :exc:`xmlrpclib.Fault` exceptions raised by the exposed XML-RPC methods as appropriate. * Dispatch XML-RPC method invocations to the appropriate method, @@ -59,7 +66,7 @@ Builtin Core The builtin server core consists of the core implementation (:class:`Bcfg2.Server.BuiltinCore.Core`) and the XML-RPC server -implementation (:mod:`Bcfg2.SSLServer`). +implementation (:mod:`Bcfg2.Server.SSLServer`). Core ~~~~ @@ -69,7 +76,7 @@ Core XML-RPC Server ~~~~~~~~~~~~~~ -.. automodule:: Bcfg2.SSLServer +.. automodule:: Bcfg2.Server.SSLServer Multiprocessing Core -------------------- @@ -79,4 +86,4 @@ Multiprocessing Core CherryPy Core ------------- -.. automodule:: Bcfg2.Server.CherryPyCore +.. automodule:: Bcfg2.Server.CherrypyCore diff --git a/doc/development/fam.txt b/doc/development/fam.txt index c2c3b14f5..e967aaf68 100644 --- a/doc/development/fam.txt +++ b/doc/development/fam.txt @@ -56,11 +56,6 @@ Pseudo .. automodule:: Bcfg2.Server.FileMonitor.Pseudo -Fam ---- - -.. automodule:: Bcfg2.Server.FileMonitor.Fam - Gamin ----- diff --git a/doc/development/lint.txt b/doc/development/lint.txt index 6c0be960d..56a3d8a66 100644 --- a/doc/development/lint.txt +++ b/doc/development/lint.txt @@ -10,14 +10,14 @@ lets you easily write your own plugins to verify various parts of your Bcfg2 specification. -Plugins are loaded in one of two ways: +Plugins are included in a module of the same name as the plugin class +in :mod:`Bcfg2.Server.Lint`, e.g., :mod:`Bcfg2.Server.Lint.Validate`. -* They may be included in a module of the same name as the plugin - class in :mod:`Bcfg2.Server.Lint`, e.g., - :mod:`Bcfg2.Server.Lint.Validate`. -* They may be included directly in a Bcfg2 server plugin, called - "<plugin>Lint", e.g., - :class:`Bcfg2.Server.Plugins.Metadata.MetadataLint`. +.. note:: + + It is no longer possible to include lint plugins directly in a + Bcfg2 server plugin, e.g., + :class:`Bcfg2.Server.Plugins.Metadata.MetadataLint`. Plugin Types ============ @@ -106,15 +106,15 @@ Basics Existing ``bcfg2-lint`` Plugins =============================== -AWSTagsLint ------------ +AWSTags +------- -.. autoclass:: Bcfg2.Server.Plugins.AWSTags.AWSTagsLint +.. automodule:: Bcfg2.Server.Lint.AWSTags -BundlerLint ------------ +Bundler +------- -.. autoclass:: Bcfg2.Server.Plugins.Bundler.BundlerLint +.. automodule:: Bcfg2.Server.Lint.Bundler Comments -------- @@ -131,10 +131,10 @@ GroupNames .. automodule:: Bcfg2.Server.Lint.GroupNames -GroupPatternsLint ------------------ +GroupPatterns +------------- -.. autoclass:: Bcfg2.Server.Plugins.GroupPatterns.GroupPatternsLint +.. automodule:: Bcfg2.Server.Lint.GroupPatterns InfoXML ------- @@ -146,25 +146,25 @@ MergeFiles .. automodule:: Bcfg2.Server.Lint.MergeFiles -MetadataLint ------------- +Metadata +-------- -.. autoclass:: Bcfg2.Server.Plugins.Metadata.MetadataLint +.. automodule:: Bcfg2.Server.Lint.Metadata -PkgmgrLint ----------- +Pkgmgr +------ -.. autoclass:: Bcfg2.Server.Plugins.Pkgmgr.PkgmgrLint +.. automodule:: Bcfg2.Server.Lint.Pkgmgr RequiredAttrs ------------- .. automodule:: Bcfg2.Server.Lint.RequiredAttrs -TemplateHelperLint ------------------- +TemplateHelper +-------------- -.. autoclass:: Bcfg2.Server.Plugins.TemplateHelper.TemplateHelperLint +.. automodule:: Bcfg2.Server.Lint.TemplateHelper Validate -------- diff --git a/doc/development/option_parsing.txt b/doc/development/option_parsing.txt new file mode 100644 index 000000000..091f43cdd --- /dev/null +++ b/doc/development/option_parsing.txt @@ -0,0 +1,246 @@ +.. -*- mode: rst -*- + +.. _development-option-parsing: + +==================== +Bcfg2 Option Parsing +==================== + +Bcfg2 uses an option parsing mechanism based on the Python +:mod:`argparse` module. It does several very useful things that +``argparse`` does not: + +* Collects options from various places, which lets us easily specify + per-plugin options, for example; +* Automatically loads components (such as plugins); +* Synthesizes option values from the command line, config files, and + environment variables; +* Can dynamically create commands with many subcommands (e.g., + bcfg2-info and bcfg2-admin); and +* Supports keeping documentation inline with the option declaration, + which will make it easier to generate man pages. + + +Collecting Options +================== + +One of the more important features of the option parser is its ability +to automatically collect options from loaded components (e.g., Bcfg2 +server plugins). Given the highly pluggable architecture of Bcfg2, +this helps ensure two things: + +#. We do not have to specify all options in all places, or even in + most places. Options are specified alongside the class(es) that use + them. +#. All options needed for a given script to run are guaranteed to be + loaded, without the need to specify all components that script uses + manually. + +For instance, assume a few plugins: + +* The ``Foo`` plugin takes one option, ``--foo`` +* The ``Bar`` plugin takes two options, ``--bar`` and ``--force`` + +The plugins are used by the ``bcfg2-quux`` command, which itself takes +two options: ``--plugins`` (which selects the plugins) and +``--test``. The options would be selected at runtime, so for instance +these would be valid: + +.. code-block:: bash + + bcfg2-quux --plugins Foo --foo --test + bcfg2-quux --plugins Foo,Bar --foo --bar --force + bcfg2-quux --plugins Bar --force + +But this would not: + + bcfg2-quux --plugins Foo --bar + +The help message would reflect the options that are available to the +default set of plugins. (For this reason, allowing component lists to +be set in the config file is very useful; that way, usage messages +reflect the components in the config file.) + +Components (in this example, the plugins) can be classes or modules. +There is no required interface for an option component. They may +*optionally* have: + +* An ``options`` attribute that is a list of + :class:`Bcfg2.Options.Options.Option` objects or option groups. +* A boolean ``parse_first`` attribute; if set to True, the options for + the component are parsed before all other options. This is useful + for, e.g., Django database settings, which must be parsed before + plugins that use Django can be loaded. +* A function or static method, ``options_parsed_hook``, that is called + when all options have been parsed. (This will be called again if + :func:`Bcfg2.Options.Parser.Parser.reparse` is called.) +* A function or static method, ``component_parsed_hook``, that is + called when early option parsing for a given component has + completed. This is *only* called for components with + ``parse_first`` set to True. It is passed a single argument: a + :class:`argparse.Namespace` object containing the complete set of + early options. + +Options are collected through two primary mechanisms: + +#. The :class:`Bcfg2.Options.Actions.ComponentAction` class. When a + ComponentAction subclass is used as the action of an option, then + options contained in the classes (or modules) given in the option + value will be added to the parser. +#. Modules that are not loaded via a + :class:`Bcfg2.Options.Actions.ComponentAction` option may load + options at runtime. + +Since it is preferred to add components instead of just options, +loading options at runtime is generally best accomplished by creating +a container object whose only purpose is to hold options. For +instance: + +.. code-block:: python + + def foo(): + # do stuff + + class _OptionContainer(object): + options = [ + Bcfg2.Options.BooleanOption("--foo", help="Enable foo")] + + @staticmethod + def options_parsed_hook(): + if Bcfg2.Options.setup.foo: + foo() + + Bcfg2.Options.get_parser().add_component(_OptionContainer) + +The Bcfg2.Options module +======================== + +.. currentmodule:: Bcfg2.Options + +.. autodata:: setup + +Options +------- + +The base :class:`Bcfg2.Options.Option` object represents an option. +Unlike options in :mod:`argparse`, an Option object does not need to +be associated with an option parser; it exists on its own. + +.. autoclass:: Option +.. autoclass:: PathOption +.. autoclass:: BooleanOption +.. autoclass:: PositionalArgument + +The Parser +---------- + +.. autoclass:: Parser +.. autofunction:: get_parser +.. autoexception:: OptionParserException + +Option Groups +------------- + +Options can be grouped in various meaningful ways. This uses a +variety of :mod:`argparse` functionality behind the scenes. + +In all cases, options can be added to groups in-line by simply +specifying them in the object group constructor: + +.. code-block:: python + + options = [ + Bcfg2.Options.ExclusiveOptionGroup( + Bcfg2.Options.Option(...), + Bcfg2.Options.Option(...), + required=True), + ....] + +Nesting object groups is supported in theory, but barely tested. + +.. autoclass:: OptionGroup +.. autoclass:: ExclusiveOptionGroup +.. autoclass:: Subparser +.. autoclass:: WildcardSectionGroup + +Subcommands +----------- + +This library makes it easier to work with programs that have a large +number of subcommands (e.g., :ref:`bcfg2-info <server-bcfg2-info>` and +:ref:`bcfg2-admin <server-admin-index>`). + +The normal implementation pattern is this: + +#. Define all of your subcommands as children of + :class:`Bcfg2.Options.Subcommand`. +#. Define a :class:`Bcfg2.Options.CommandRegistry` object that will be + used to register all of the commands. Registering a command + collect its options and adds it as a + :class:`Bcfg2.Options.Subparser` option group to the main option + parser. +#. Register your commands with + :func:`Bcfg2.Options.register_commands`, parse options, and run. + +:mod:`Bcfg2.Server.Admin` provides a fairly simple implementation, +where the CLI class is itself the command registry: + +.. code-block:: python + + class CLI(Bcfg2.Options.CommandRegistry): + def __init__(self): + Bcfg2.Options.CommandRegistry.__init__(self) + Bcfg2.Options.register_commands(self.__class__, + globals().values(), + parent=AdminCmd) + parser = Bcfg2.Options.get_parser( + description="Manage a running Bcfg2 server", + components=[self]) + parser.parse() + +In this case, commands are collected from amongst all global variables +(the most likely scenario), and they must be children of +:class:`Bcfg2.Server.Admin.AdminCmd`, which itself subclasses +:class:`Bcfg2.Options.Subcommand`. + +Commands are defined by subclassing :class:`Bcfg2.Options.Subcommand`. +At a minimum, the :func:`Bcfg2.Options.Subcommand.run` method must be +overridden, and a docstring written. + +.. autoclass:: Subcommand +.. autoclass:: HelpCommand +.. autoclass:: CommandRegistry +.. autofunction:: register_commands + +Actions +------- + +Several custom argparse `actions +<http://docs.python.org/dev/library/argparse.html#action>`_ provide +some of the option collection magic of :mod:`Bcfg2.Options`. + +.. autoclass:: ConfigFileAction +.. autoclass:: ComponentAction +.. autoclass:: PluginsAction + +Option Types +------------ + +:mod:`Bcfg2.Options` provides a number of useful types for use as the `type +<http://docs.python.org/dev/library/argparse.html#type>`_ keyword +argument to +the :class:`Bcfg2.Options.Option` constructor. + +.. autofunction:: Bcfg2.Options.Types.path +.. autofunction:: Bcfg2.Options.Types.comma_list +.. autofunction:: Bcfg2.Options.Types.colon_list +.. autofunction:: Bcfg2.Options.Types.octal +.. autofunction:: Bcfg2.Options.Types.username +.. autofunction:: Bcfg2.Options.Types.groupname +.. autofunction:: Bcfg2.Options.Types.timeout +.. autofunction:: Bcfg2.Options.Types.size + +Common Options +-------------- + +.. autoclass:: Common diff --git a/doc/development/plugins.txt b/doc/development/plugins.txt index 3f2a888ac..e4f16b84d 100644 --- a/doc/development/plugins.txt +++ b/doc/development/plugins.txt @@ -128,13 +128,15 @@ The two attributes you need to know about are: of the caching mode. See :ref:`server-caching` for a description of each mode. * :attr:`Bcfg2.Server.Core.metadata_cache`: A dict-like - :class:`Bcfg2.Cache.Cache` object that stores the cached data. + :class:`Bcfg2.Server.Cache.Cache` object that stores the cached + data. :class:`Bcfg2.Server.Plugin.base.Plugin` objects have access to the :class:`Bcfg2.Server.Core` object as ``self.core``. In general, -you'll be interested in the :func:`Bcfg2.Cache.Cache.expire` method; -if called with no arguments, it expires all cached data; if called -with one string argument, it expires cached data for the named client. +you'll be interested in the :func:`Bcfg2.Server.Cache.Cache.expire` +method; if called with no arguments, it expires all cached data; if +called with one string argument, it expires cached data for the named +client. It's important, therefore, that your Connector plugin can either track when changes are made to the group membership it reports, and expire @@ -163,7 +165,7 @@ Tracking Execution Time .. versionadded:: 1.3.0 Statistics can and should track execution time statistics using -:mod:`Bcfg2.Statistics`. This module tracks execution time for the +:mod:`Bcfg2.Server.Statistics`. This module tracks execution time for the server core and for plugins, and exposes that data via ``bcfg2-admin perf``. This data can be invaluable for locating bottlenecks or other performance issues. @@ -184,13 +186,13 @@ This will track the execution time of ``do_something``. More granular usage is possible by using :func:`time.time` to manually determine the execution time of a given event and calling -:func:`Bcfg2.Statistics.Statistics.add_value` with an appropriate +:func:`Bcfg2.Server.Statistics.Statistics.add_value` with an appropriate statistic name. -Bcfg2.Statistics -^^^^^^^^^^^^^^^^ +Bcfg2.Server.Statistics +^^^^^^^^^^^^^^^^^^^^^^^ -.. automodule:: Bcfg2.Statistics +.. automodule:: Bcfg2.Server.Statistics Plugin Helper Classes --------------------- diff --git a/doc/development/setup.txt b/doc/development/setup.txt index 05ad4157f..42aa0b023 100644 --- a/doc/development/setup.txt +++ b/doc/development/setup.txt @@ -1,4 +1,5 @@ .. -*- mode: rst -*- +.. vim: ft=rst .. _development-setup: @@ -12,6 +13,11 @@ Checking Out a Copy of the Code git clone https://github.com/Bcfg2/bcfg2.git +.. note:: + + The URL above is read-only. If you are planning on submitting patches + upstream, please see :ref:`development-submitting-patches`. + * Add :file:`bcfg2/src/sbin` to your :envvar:`PATH` environment variable * Add :file:`bcfg2/src/lib` to your :envvar:`PYTHONPATH` environment variable diff --git a/doc/development/submitting-patches.txt b/doc/development/submitting-patches.txt new file mode 100644 index 000000000..04492e6e1 --- /dev/null +++ b/doc/development/submitting-patches.txt @@ -0,0 +1,144 @@ +.. -*- mode: rst -*- +.. vim: ft=rst + +.. _development-submitting-patches: + +================== +Submitting Patches +================== + +The purpose of this document is to assist those who may be less familiar +with git in submitting patches upstream. While git is powerful, it can +be somewhat confusing to those who don't use it regularly (and even +those who do). + +.. note:: + + We prefer more in-depth commit messages than those + given below which are purely for brevity in this guide. See + http://tbaggery.com/2008/04/19/a-note-about-git-commit-messages.html + for more about creating proper git commit messages. + +.. _Github: https://github.com/ + +`Github`_ +========= + +These steps outline one way of submitting patches via `Github`_. First, +you will want to `fork <https://github.com/Bcfg2/bcfg2/fork>`_ the +upstream Bcfg2 repository. + +Create a local branch +--------------------- + +Once you have forked the upstream repository, you should clone a local +copy (where <YOUR USERNAME> is your github username). + +:: + + git clone git@github.com:<YOUR USERNAME>/bcfg2.git + +Create a local feature/bugfix branch off the appropriate upstream +branch. For example, let's say we want to submit a bugfix for +:program:`bcfg2-info` against the 1.2.x series. We can create a +``fix-bcfg2-info`` branch which is a copy of the ``maint-1.2`` branch. + +:: + + git branch fix-bcfg2-info maint-1.2 + git checkout fix-bcfg2-info + +Commit changes to your local branch +----------------------------------- + +Next make whatever changes need to be made and commit them to the +``fix-bcfg2-info`` branch. + +:: + + git add src/sbin/bcfg2-info + git commit -m "Fix bcfg2-info bug" + +Now you need to push your ``fix-bcfg2-info`` branch to github. + +:: + + git push origin fix-bcfg2-info + +Submit pull request +------------------- + +Next, submit a pull request against the proper branch (in this case, +https://github.com/username/bcfg2/pull/new/fix-bcfg2-info -- again, +username is your github username). At the top of the pull request, you can +edit the upstream branch you are targetting so that you create the pull +request against the proper upstream branch (in this case, ``maint-1.2``). + +All that's left to do is to write up a description of your pull request +and click **Send pull request**. Since your local branch is specific to +this fix, you can add additional commits if needed and push them. They +will automatically be added to the pull request. + +Remove local branch +------------------- + +Once we have merged your pull request, you can safely delete your local +feature/bugfix branch. To do so, you must first checkout a different branch. + +:: + + git checkout master # switch to a different branch + git branch -d fix-bcfg2-info # delete your local copy of fix-bcfg2-info + git push origin :fix-bcfg2-info # delete fix-bcfg2-info from github + +Mailing List +============ + +The following lists the steps needed to use git's facilities for +emailing patches to the mailing list. + +Commit changes to your local clone +---------------------------------- + +For example, let's say we want to fix a big in :program:`bcfg2-info`. +For the 1.2.x series. + +:: + + git clone https://github.com/Bcfg2/bcfg2.git + git checkout maint-1.2 + # make changes + git add src/sbin/bcfg2-info + git commit -m "Fix bcfg2-info bug" + +Setup git for gmail (optional) +------------------------------ + +If you would like to use the GMail SMTP server, you can add the following +to your ~/.gitconfig file as per the :manpage:`git-send-email(1)` manpage. + +:: + + [sendemail] + smtpencryption = tls + smtpserver = smtp.gmail.com + smtpuser = yourname@gmail.com + smtpserverport = 587 + +Format patches +-------------- + +Use git to create patches formatted for email with the following. + +:: + + git format-patch --cover-letter -M origin/maint-1.2 -o outgoing/ + + +Send emails to the mailing list +------------------------------- + +Edit ``outgoing/0000-*`` and then send your emails to the mailing list +(bcfg-dev@lists.mcs.anl.gov):: + + git send-email outgoing/* diff --git a/doc/development/unit-testing.txt b/doc/development/unit-testing.txt index 7311f49d7..8007e8c75 100644 --- a/doc/development/unit-testing.txt +++ b/doc/development/unit-testing.txt @@ -1,4 +1,5 @@ .. -*- mode: rst -*- +.. vim: ft=rst .. _development-unit-testing: @@ -13,7 +14,7 @@ You will first need to install the `Python Mock Module`_ and `Python Nose`_ modules. You can then run the existing tests with the following: -.. code-block: bash +.. code-block: sh cd testsuite nosetests @@ -123,7 +124,7 @@ writing tests for the base :class:`Bcfg2.Server.Plugin.base.Plugin` class, which all Bcfg2 :ref:`server-plugins-index` inherit from via the :mod:`Plugin interfaces <Bcfg2.Server.Plugin.interfaces>`, yielding several levels of often-multiple inheritance. To make this -easier, our unit tests adhere to several design considerations: +easier, our unit tests adhere to several design considerations. Inherit Tests ------------- diff --git a/doc/getting_started/index.txt b/doc/getting_started/index.txt index a9b1b847f..9b69bf65a 100644 --- a/doc/getting_started/index.txt +++ b/doc/getting_started/index.txt @@ -21,7 +21,7 @@ Get and Install Bcfg2 Server We recommend running the server on a Linux machine for ease of deployment due to the availability of packages for the dependencies. -First, you need to download and install Bcfg2. The section +First, you need to download and install Bcfg2. The section :ref:`installation-index` in this manual describes the steps to take. To start, you will need to install the server on one machine and the client on one or more machines. Yes, your server can also be a client @@ -71,7 +71,7 @@ That can be translated as "bcfg2 quick verbose no-op". The output should be something similar to:: Loaded tool drivers: - Chkconfig POSIX YUMng + Chkconfig POSIX YUM Phase: initial Correct entries: 0 @@ -108,7 +108,7 @@ After the above steps, you should have a toplevel repository structure that looks like:: bcfg-server:~ # ls /var/lib/bcfg2 - Base/ Bundler/ Cfg/ Metadata/ Pkgmgr/ Rules/ SSHbase/ etc/ + Bundler/ Cfg/ Metadata/ Pkgmgr/ Rules/ SSHbase/ etc/ The place to start is the Metadata directory, which contains two files: ``clients.xml`` and ``groups.xml``. Your current @@ -169,7 +169,7 @@ Next, we create a motd.xml file in the Bundler directory: .. code-block:: xml - <Bundle name='motd'> + <Bundle> <Path name='/etc/motd' /> </Bundle> diff --git a/doc/help/troubleshooting.txt b/doc/help/troubleshooting.txt index 72fec4c63..aa46bda2a 100644 --- a/doc/help/troubleshooting.txt +++ b/doc/help/troubleshooting.txt @@ -69,13 +69,13 @@ included with the source distribution and all packages. If the bcfg2 server is not reflecting recent changes, try restarting the bcfg2-server process ============================================================================================= -If this fixes the problem, it is either a bug in the -underlying file monitoring system (fam or gamin) or a bug in -Bcfg2's file monitoring code. In either case, file a `ticket +If this fixes the problem, it is either a bug in the underlying file +monitoring system (inotify or gamin) or a bug in Bcfg2's file +monitoring code. In either case, file a `ticket <https://trac.mcs.anl.gov/projects/bcfg2/newticket>`_ in the tracking system. In the ticket, include: -* filesystem monitoring system (fam or gamin) +* filesystem monitoring system (inotify or gamin) * kernel version (if on linux) * if any messages of the form "Handled N events in M seconds" appeared between the modification event and the client @@ -259,8 +259,7 @@ Server Errors :ref:`server-info` file for this entry. .. [s11] Verify that you have the proper prefix set in bcfg2.conf. .. [s12] Ensure that the client is a member of all the appropriate - :ref:`server-plugins-generators-packages-magic-groups` as - well as any additional groups you may have defined in your + groups you may have defined in your :ref:`server-plugins-generators-packages` configuration. FAQs diff --git a/doc/installation/prerequisites.txt b/doc/installation/prerequisites.txt index 0cb721bb9..81ac12632 100644 --- a/doc/installation/prerequisites.txt +++ b/doc/installation/prerequisites.txt @@ -50,12 +50,26 @@ Bcfg2 Server +-------------------------------+----------+--------------------------------+ | lxml | 0.9+ | lxml: libxml2, libxslt, python | +-------------------------------+----------+--------------------------------+ -| gamin or fam | Any | | +| gamin or inotify | Any | | +-------------------------------+----------+--------------------------------+ -| python-gamin or python-fam | Any | gamin or fam, python | +| python-gamin or pyinotify | Any | gamin or inotify, python | +-------------------------------+----------+--------------------------------+ | M2crypto or python-ssl (note | Any | python, openssl | | that the ssl module is | | | | included in python versions | | | | 2.6 and later | | | +-------------------------------+----------+--------------------------------+ + +Bcfg2 Reporting +--------------- + +A webserver capabable of running wsgi applications is required for web reporting, +such as Apache + mod_wsgi or nginx. + ++-------------------------------+----------+--------------------------------+ +| Software | Version | Requires | ++===============================+==========+================================+ +| django | 1.2.0+ | | ++-------------------------------+----------+--------------------------------+ +| south | 0.7.0+ | | ++-------------------------------+----------+--------------------------------+ diff --git a/doc/man/bcfg2-admin.txt b/doc/man/bcfg2-admin.txt index 6169ec537..7b19bd366 100644 --- a/doc/man/bcfg2-admin.txt +++ b/doc/man/bcfg2-admin.txt @@ -38,9 +38,6 @@ Modes backup Create an archive of the entire Bcfg2 repository. -bundle *action* - Display details about the available bundles (See BUNDLE OPTIONS - below). client *action* *client* [attribute=value] Add, edit, or remove clients entries in metadata (See CLIENT OPTIONS below). @@ -48,8 +45,12 @@ compare *old* *new* Compare two client configurations. Can be used to verify consistent behavior between releases. Determine differences between files or directories (See COMPARE OPTIONS below). +dbshell + Call the Django 'dbshell' command on the configured database. init Initialize a new repository (interactive). +initreports + Initialize the Reporting database. minestruct *client* [-f xml-file] [-g groups] Build structure entries based on client statistics extra entries (See MINESTRUCT OPTIONS below). @@ -58,34 +59,31 @@ perf pull *client* *entry-type* *entry-name* Install configuration information into repo based on client bad entries (See PULL OPTIONS below). -reports [init|load_stats|purge|scrub|update] - Interact with the dynamic reporting system (See REPORTS OPTIONS - below). -snapshots [init|dump|query|reports] - Interact with the Snapshots database (See SNAPSHOTS OPTIONS below). +purgereports + Purge historic and expired data from the Reporting database +reportssqlall + Call the Django 'shell' command on the Reporting database. +reportsstats + Print Reporting database statistics. +scrubreports + Scrub the Reporting database for duplicate reasons and orphaned + entries. +shell + Call the Django 'shell' command on the configured database. syncdb Sync the Django ORM with the configured database. tidy Remove unused files from repository. +updatereports + Apply database schema updates to the Reporting database. +validatedb + Call the Django 'validate' command on the configured database. viz [-H] [-b] [-k] [-o png-file] Create a graphviz diagram of client, group and bundle information (See VIZ OPTIONS below). xcmd Provides a XML-RPC Command Interface to the bcfg2-server. -BUNDLE OPTIONS -++++++++++++++ - -mode - One of the following. - - *list-xml* - List all available xml bundles - *list-genshi* - List all available genshi bundles - *show* - Interactive dialog to get details about the available bundles - CLIENT OPTIONS ++++++++++++++ @@ -110,11 +108,20 @@ attribute=value COMPARE OPTIONS +++++++++++++++ +-d *N*, --diff-lines *N* + Show only N lines of a diff + +-c, --color + Show colors even if not ryn from a TTY + +-q, --quiet + Only show that entries differ, not how they differ + old - Specify the location of the old configuration file. + Specify the location of the old configuration(s). new - Specify the location of the new configuration file. + Specify the location of the new configuration(s). MINESTRUCT OPTIONS ++++++++++++++++++ @@ -140,51 +147,24 @@ entry type entry name Specify the name of the entry to pull. -REPORTS OPTIONS -+++++++++++++++ - -load_stats [-s] [-c] [-03] - Load statistics data. - -purge [--client [n]] [--days [n]] [--expired] - Purge historic and expired data. - -scrub - Scrub the database for duplicate reasons and orphaned entries. - -update - Apply any updates to the reporting database. - -SNAPSHOTS OPTIONS -+++++++++++++++++ - -init - Initialize the snapshots database. - -query - Query the snapshots database. - -dump - Dump some of the contents of the snapshots database. - -reports [-a] [-b] [-e] [--date=MM-DD-YYYY] - Generate reports for clients in the snapshots database. - VIZ OPTIONS +++++++++++ --H +-H, --includehosts Include hosts in diagram. --b +-b, --includebundles Include bundles in diagram. --o <outfile> +-o *outfile*, --outfile *outfile* Write to outfile file instead of stdout. --k +-k, --includekey Add a shape/color key. +-c *hostname*, --only-client *hostname* + Only show groups and bundles for the named client + See Also -------- diff --git a/doc/man/bcfg2.conf.txt b/doc/man/bcfg2.conf.txt index 6faf48a1a..df49f3d4a 100644 --- a/doc/man/bcfg2.conf.txt +++ b/doc/man/bcfg2.conf.txt @@ -43,7 +43,6 @@ filemonitor inotify gamin - fam pseudo fam_blocking @@ -76,24 +75,22 @@ plugins A comma-delimited list of enabled server plugins. Currently available plugins are:: - Account - Base + ACL Bundler Bzr Cfg Cvs Darcs - DBStats Decisions + Defaults Deps - Editor FileProbes Fossil Git + GroupLogic GroupPatterns Guppy Hg - Hostbase Ldap Metadata NagiosGen @@ -108,14 +105,9 @@ plugins Rules SEModules ServiceCompat - Snapshots SSHbase - SSLCA - Statistics Svn - TCheetah TemplateHelper - TGenshi Trigger Descriptions of each plugin can be found in their respective @@ -158,25 +150,10 @@ Server Plugins This section has a listing of all the plugins currently provided with Bcfg2. -Account Plugin -++++++++++++++ - -The account plugin manages authentication data, including the following. - -* ``/etc/passwd`` -* ``/etc/group`` -* ``/etc/security/limits.conf`` -* ``/etc/sudoers`` -* ``/root/.ssh/authorized_keys`` - -Base Plugin -+++++++++++ +ACL Plugin +++++++++++ -The Base plugin is a structure plugin that provides the ability -to add lists of unrelated entries into client configuration entry -inventories. Base works much like Bundler in its file format. This -structure plugin is good for the pile of independent configs needed for -most actual systems. +The ACL plugin controls which hosts can make which XML-RPC calls. Bundler Plugin ++++++++++++++ @@ -203,25 +180,20 @@ contents for clients. In its simplest form, the Cfg repository is just a directory tree modeled off of the directory tree on your client machines. -Cvs Plugin (experimental) -+++++++++++++++++++++++++ +Cvs Plugin +++++++++++ The Cvs plugin allows you to track changes to your Bcfg2 repository using a Concurrent version control backend. Currently, it enables you to get revision information out of your repository for reporting purposes. -Darcs Plugin (experimental) -+++++++++++++++++++++++++++ +Darcs Plugin +++++++++++++ The Darcs plugin allows you to track changes to your Bcfg2 repository using a Darcs version control backend. Currently, it enables you to get revision information out of your repository for reporting purposes. -DBStats Plugin -++++++++++++++ - -Direct to database statistics plugin. - Decisions Plugin ++++++++++++++++ @@ -245,13 +217,6 @@ Deps Plugin The Deps plugin allows you to make a series of assertions like "Package X requires Package Y (and optionally also Package Z etc.)" -Editor Plugin -+++++++++++++ - -The Editor plugin attempts to allow you to partially manage -configuration for a file. Its use is not recommended and not well -documented. - FileProbes Plugin +++++++++++++++++ @@ -274,6 +239,12 @@ The Git plugin allows you to track changes to your Bcfg2 repository using a Git version control backend. Currently, it enables you to get revision information out of your repository for reporting purposes. +GroupLogic Plugin ++++++++++++++++++ + +The GroupLogic plugin lets you flexibly assign group membership with a +Genshi template. + GroupPatterns Plugin ++++++++++++++++++++ @@ -286,18 +257,13 @@ Guppy Plugin The Guppy plugin is used to trace memory leaks within the bcfg2-server process using Guppy. -Hg Plugin (experimental) -++++++++++++++++++++++++ +Hg Plugin ++++++++++ The Hg plugin allows you to track changes to your Bcfg2 repository using a Mercurial version control backend. Currently, it enables you to get revision information out of your repository for reporting purposes. -Hostbase Plugin -+++++++++++++++ - -The Hostbase plugin is an IP management system built on top of Bcfg2. - Ldap Plugin +++++++++++ @@ -316,8 +282,8 @@ NagiosGen Plugin The NagiosGen plugin dynamically generates Nagios configuration files based on Bcfg2 data. -Ohai Plugin (experimental) -++++++++++++++++++++++++++ +Ohai Plugin ++++++++++++ The Ohai plugin is used to detect information about the client operating system. The data is reported back to the server using JSON. @@ -373,10 +339,10 @@ dynamic reporting system. Rules Plugin ++++++++++++ -The Rules plugin provides literal configuration entries that resolve the -abstract configuration entries normally found in the Bundler and Base -plugins. The literal entries in Rules are suitable for consumption by -the appropriate client drivers. +The Rules plugin provides literal configuration entries that resolve +the abstract configuration entries normally found in Bundler. The +literal entries in Rules are suitable for consumption by the +appropriate client drivers. SEModules Plugin ++++++++++++++++ @@ -389,12 +355,6 @@ ServiceCompat Plugin The ServiceCompat plugin converts service entries for older clients. -Snapshots Plugin -++++++++++++++++ - -The Snapshots plugin stores various aspects of a client’s state when the -client checks in to the server. - SSHbase Plugin ++++++++++++++ @@ -402,17 +362,6 @@ The SSHbase generator plugin manages ssh host keys (both v1 and v2) for hosts. It also manages the ssh_known_hosts file. It can integrate host keys from other management domains and similarly export its keys. -SSLCA Plugin -++++++++++++ - -The SSLCA plugin is designed to handle creation of SSL privatekeys and -certificates on request. - -Statistics -++++++++++ - -The Statistics plugin is deprecated (see Reporting). - Svn Plugin ++++++++++ @@ -420,20 +369,6 @@ The Svn plugin allows you to track changes to your Bcfg2 repository using a Subversion backend. Currently, it enables you to get revision information out of your repository for reporting purposes. -TCheetah Plugin -+++++++++++++++ - -The TCheetah plugin allows you to use the cheetah templating system to -create files. It also allows you to include the results of probes -executed on the client in the created files. - -TGenshi Plugin -++++++++++++++ - -The TGenshi plugin allows you to use the Genshi templating system to -create files. It also allows you to include the results of probes -executed on the client in the created files. - Trigger Plugin ++++++++++++++ @@ -667,25 +602,12 @@ the configuration file. running in paranoid mode. Only the most recent versions of these copies will be kept. -Snapshots options ------------------ - -Specified in the **[snapshots]** section. These options control the -server snapshots functionality. - - driver - sqlite - - database - The name of the database to use for statistics data. - - e.g.: ``$REPOSITORY_DIR/etc/bcfg2.sqlite`` - -SSLCA options -------------- +SSL CA options +-------------- -These options are necessary to configure the SSLCA plugin and can be -found in the **[sslca_default]** section of the configuration file. +These options are necessary to configure the SSL CA feature of the Cfg +plugin and can be found in the **[sslca_default]** section of the +configuration file. config Specifies the location of the openssl configuration file for diff --git a/doc/reports/index.txt b/doc/reports/index.txt index 1360d5ffd..aaed29dfe 100644 --- a/doc/reports/index.txt +++ b/doc/reports/index.txt @@ -24,5 +24,4 @@ uses django and a database backend. .. toctree:: :maxdepth: 2 - static dynamic diff --git a/doc/reports/static.txt b/doc/reports/static.txt deleted file mode 100644 index 00c1867f8..000000000 --- a/doc/reports/static.txt +++ /dev/null @@ -1,100 +0,0 @@ -.. -*- mode: rst -*- - -.. _reports-static: - -============================= -Bcfg2 Static Reporting System -============================= - -The Bcfg2 reporting system collects and displays information about the -operation of the Bcfg2 client, and the configuration states of target -machines. - -Goals -===== - -The reporting system provides an interface to administrators describing -a few important tasks - -* Client configuration state, particularly aspects that do not match the configuration specification. - Information about bad and extra configuration elements is included. -* Client execution results (a list of configuration elements that were modified) -* Client execution performance data (including operation retry counts, and timings for several critical execution regions) - -This data can be used to understand the current configuration state -of the entire network, the operations performed by the client, how the -configuration changes propagate, and any reconfiguration operations that -have failed. - -Retention Model -=============== - -The current reporting system stores statistics in an XML data store, by -default to ``<repo>/etc/statistics.xml``. It retains either one or two -statistic sets per host. If the client has a clean configuration state, -the most recent (clean) record is retained. If the client has a dirty -configuration state, two records are retained. One record is the last -clean record. The other record is the most recent record collected, -detailing the incorrect state. - -This retention model, while non-optimal, does manage to persistently -record most of the data that users would like. - -Setup -===== - -In order to configure your Bcfg2 server for receiving reports, you -will need to list the Statistics plugin in the plugins line of your -``bcfg2.conf``. You will also need a [statistics] section -in your ``bcfg2.conf``. You can find out more about what goes there in the -``bcfg2.conf`` manpage. - -Output -====== - -Several output reports can be generated from the statistics store with -the command line tool ``bcfg2-build-reports``. - -* Nodes Digest -* Nodes Individual -* Overview Statistics -* Performance - -The data generated by these reports can be delivered by several -mechanisms: - -* HTML -* Email -* RSS - -Shortcomings and Planned Enhancements -===================================== - -When designing the current reporting system, we were overly concerned with -the potential explosion in data size over time. In order to address this, -we opted to use the retention scheme described above. This approach has -several shortcomings: - -* A comprehensive list of reconfiguration operations (with associated - timestamps) isn't retained -* Client results for any given day (except the last one) aren't uniformly - retained. This means that inter-client analysis is difficult, if - not impossible - -We plan to move to a database backend to address the dataset size -problem and start retaining all information. The move to a SQL backend -will allow many more types of queries to be efficiently processed. It -will also make on-demand reports simpler. - -Other sorts of information would also be useful to track. We plan to -add the ability to tag a particular configuration element as security -related, and include this in reports. This will aid in the effective -prioritization of manual and failed reconfiguration tasks. - -Capability Goals (posed as questions) -------------------------------------- - -* What machines have not yet applied critical updates? -* How long did critical updates take to be applied? -* What configuration did machine X have on a particular date? -* When did machine X perform configuration update Y? diff --git a/doc/server/acl.txt b/doc/server/acl.txt new file mode 100644 index 000000000..3cb9d59a1 --- /dev/null +++ b/doc/server/acl.txt @@ -0,0 +1,41 @@ +.. -*- mode: rst -*- + +.. _server-access-control: + +================ + Access Control +================ + +.. versionadded:: 1.4.0 + +Bcfg2 exposes various functions via XML-RPC calls. Some of these are +relatively benign (e.g., the calls necessary to generate a client +configuration) while others can be used to inspect potentially private +data on the server or very easily mount a denial of service attack. +As a result, access control lists to limit exposure of these calls is +built in. There are two possible ACL methods: built-in, and the +:ref:`server-plugins-misc-acl` plugin. + +The built-in approach simply applies a restrictive default ACL that +lets ``localhost`` perform all XML-RPC calls, and restricts all other +machines to only the calls necessary to run the Bcfg2 client. +Specifically: + +* If the remote client is ``127.0.0.1``, the call is allowed. Note + that, depending on where your Bcfg2 server listens and how it + communicates with itself, it likely will not identify to itself as + ``localhost``. +* If the remote client is not ``127.0.0.1`` and the call is any of the + ``set_debug`` or ``toggle_debug`` methods (including + ``[toggle|set]_core_debug``), it is rejected. +* If the remote client is not ``127.0.0.1`` and the call is + ``get_statistics`` (used by ``bcfg2-admin perf``), it is rejected. +* If the remote client is not ``127.0.0.1`` and the call includes a + ``.`` -- i.e., it is dispatched to any plugin, such as + ``Packages.Refresh`` -- then it is rejected. +* Otherwise, the call is allowed. + +The built-in ACL is *only* intended to ensure that Bcfg2 is secure by +default; it will not be sufficient in many (or even most) cases. In +these cases, it's recommended that you use the +:ref:`server-plugins-misc-acl` plugin. diff --git a/doc/server/admin/bundle.txt b/doc/server/admin/bundle.txt deleted file mode 100644 index e9cb79781..000000000 --- a/doc/server/admin/bundle.txt +++ /dev/null @@ -1,34 +0,0 @@ -.. -*- mode: rst -*- - -.. _server-admin-bundle: - -bundle -====== - -For a list of all available xml bundles use ``list-xml``. ``list-genshi`` -will list all available genshi bundles.:: - -.. code-block:: sh - - # bcfg2-admin bundles list-xml - # bcfg2-admin bundles list-genshi - -``show`` provides an interactive dialog to get details about the available -bundles.:: - -.. code-block:: sh - - # bcfg2-admin bundles show - Available bundles (Number of bundles: 4) - ---------------------------------------- - [0] motd.xml - [1] snmpd.xml - [2] bcfg2.xml - [3] ntp.xml - Enter the line number of a bundle for details: 3 - Details for the "ntp" bundle: - Package: xntp - Path: /etc/sysconfig/xntp - Path: /etc/sysconfig/clock - Path: /etc/ntp.conf - Service: xntpd diff --git a/doc/server/admin/compare.txt b/doc/server/admin/compare.txt index 6a770055e..ffe19efdf 100644 --- a/doc/server/admin/compare.txt +++ b/doc/server/admin/compare.txt @@ -6,11 +6,10 @@ compare ======= Determine differences between files or directories of client -specification instances.:: +specification instances:: bcfg2-admin compare <file1> <file2> -If you want to compare two directories recursively then use ``-r`` as an -option. :: +Or:: - bcfg2-admin compare -r <dir1> <dir2> + bcfg2-admin compare <dir1> <dir2> diff --git a/doc/server/admin/index.txt b/doc/server/admin/index.txt index c563ead9c..707f7c724 100644 --- a/doc/server/admin/index.txt +++ b/doc/server/admin/index.txt @@ -16,14 +16,11 @@ functionality. Available modes are listed below. :maxdepth: 1 backup - bundle client compare init minestruct perf pull - snapshots - tidy viz xcmd diff --git a/doc/server/admin/init.txt b/doc/server/admin/init.txt index 0e8b3afd3..db42c8222 100644 --- a/doc/server/admin/init.txt +++ b/doc/server/admin/init.txt @@ -36,7 +36,6 @@ detected or a default value is provided. :: A toplevel repository structure was created under the provided path. :: /var/lib/bcfg2 - |-- Base |-- Bundler |-- Cfg |-- etc diff --git a/doc/server/admin/snapshots.txt b/doc/server/admin/snapshots.txt deleted file mode 100644 index 25a7286c2..000000000 --- a/doc/server/admin/snapshots.txt +++ /dev/null @@ -1,8 +0,0 @@ -.. -*- mode: rst -*- - -.. _server-admin-snapshots: - -snapshots -========= - -Interact with the Snapshots system. diff --git a/doc/server/admin/tidy.txt b/doc/server/admin/tidy.txt deleted file mode 100644 index 816d6cdb3..000000000 --- a/doc/server/admin/tidy.txt +++ /dev/null @@ -1,8 +0,0 @@ -.. -*- mode: rst -*- - -.. _server-admin-tidy: - -tidy -==== - -Clean up useless files in the repo. diff --git a/doc/server/caching.txt b/doc/server/caching.txt index 51245bd08..32be684db 100644 --- a/doc/server/caching.txt +++ b/doc/server/caching.txt @@ -13,7 +13,7 @@ Metadata Caching Caching (or, rather, cache expiration) is always a difficult problem, but it's particularly vexing in Bcfg2 due to the number of different -data sources incorporated. In 1.3.0, we introduce some limited +data sources incorporated. In 1.3.0, we introduced some limited caching of client metadata objects. Since a client metadata object can be generated anywhere from 7 to dozens of times per client run (depending on your templates), and since client metadata generation diff --git a/doc/server/configuration.txt b/doc/server/configuration.txt index bb4983367..d3fa42601 100644 --- a/doc/server/configuration.txt +++ b/doc/server/configuration.txt @@ -55,9 +55,7 @@ itself, which would prevent the ``bcfg2`` user from enabling a new plugin. If you depend on this capability (e.g., if your specification is stored in a VCS and checked out onto the Bcfg2 server by a script running as the ``bcfg2`` user), then you would want to ``chown`` and -``chmod`` ``/var/lib/bcfg2`` rather than ``/var/lib/bcfg2/*``. Note -also that the recursive ``chmod`` will change permissions on any files -that are using ``mode="inherit"`` in :ref:`server-info`. +``chmod`` ``/var/lib/bcfg2`` rather than ``/var/lib/bcfg2/*``. The Bcfg2 server also needs to be able to read its SSL certificate, key and the SSL CA certificate: diff --git a/doc/server/configurationentries.txt b/doc/server/configurationentries.txt index 66ff617c0..446257d62 100644 --- a/doc/server/configurationentries.txt +++ b/doc/server/configurationentries.txt @@ -28,7 +28,7 @@ Example: .. code-block:: xml - <Bundle name='ntp'> + <Bundle> <BoundPackage name='ntp' type='deb' version='1:4.2.4p4+dfsg-3ubuntu2.1'/> </Bundle> diff --git a/doc/server/encryption.txt b/doc/server/encryption.txt index e31124d4b..b657deb8c 100644 --- a/doc/server/encryption.txt +++ b/doc/server/encryption.txt @@ -24,7 +24,7 @@ feature requires M2Crypto 0.18 or newer. single Bcfg2 repository with multiple admins who should not necessarily have access to each other's sensitive data. -Two types of data can be encrypted: +Two basic types of data can be encrypted: * :ref:`server-plugins-generators-cfg` files can be encrypted as whole files. See :ref:`server-plugins-generators-cfg-encryption` @@ -51,6 +51,13 @@ In general, Properties encryption is preferred for a few reasons: amongst different teams, this lets teams collaborate more closely on files and other data. +Other types of data that can be encrypted are: + +* Text content of Path tags in + :ref:`server-plugins-structures-bundler-index` +* Passphrases in XML description files for generated + :ref:`server-plugins-generators-cfg-sshkeys` + .. _bcfg2-crypt: bcfg2-crypt @@ -203,6 +210,8 @@ get a list of valid algorithms, you can run:: openssl list-cipher-algorithms | grep -v ' => ' | \ tr 'A-Z-' 'a-z_' | sort -u +.. _server-encryption-lax-strict: + Lax vs. Strict decryption ------------------------- @@ -223,7 +232,10 @@ This can be overridden by individual XML files by setting ``decrypt="strict"`` on the top-level tag (or, vice-versa; if strict is the default an XML file can specify ``decrypt="lax"``. +Note that you could, for instance, set lax decryption by default, and +then set strict decryption on individual files. + Encryption API ============== -.. automodule:: Bcfg2.Encryption +.. automodule:: Bcfg2.Server.Encryption diff --git a/doc/server/genshi-xml.txt b/doc/server/genshi-xml.txt deleted file mode 100644 index 3216cc00d..000000000 --- a/doc/server/genshi-xml.txt +++ /dev/null @@ -1,24 +0,0 @@ -.. -*- mode: rst -*- - -.. _xml-genshi-reference: - -=============================== - Genshi XML Template Reference -=============================== - -Genshi's XML templating language is used in -:ref:`server-plugins-structures-bundler-index` for templated bundles. -The language is described in depth at `Genshi -<http://genshi.edgewall.org>`_. The XML schema reference follows. - -Genshi Tags -=========== - -.. xml:group:: genshiElements - :namespace: py - -Genshi Attributes -================= - -.. xml:attributegroup:: genshiAttrs - :namespace: py diff --git a/doc/server/index.txt b/doc/server/index.txt index 2ccc9c923..ce751f2ae 100644 --- a/doc/server/index.txt +++ b/doc/server/index.txt @@ -26,11 +26,11 @@ clients. admin/index configurationentries info - snapshots/index bcfg2-info selinux configuration database caching encryption - genshi-xml + xml-common + acl diff --git a/doc/server/info.txt b/doc/server/info.txt index b4d1f7113..8342e1cee 100644 --- a/doc/server/info.txt +++ b/doc/server/info.txt @@ -7,8 +7,7 @@ info.xml ======== Various file properties for entries served by most generator plugins, -including :ref:`server-plugins-generators-cfg`, -:ref:`server-plugins-generators-sslca`, and +including :ref:`server-plugins-generators-cfg` and :ref:`server-plugins-generators-sshbase`, are controlled through the use of ``info.xml`` files. @@ -53,25 +52,3 @@ A more complex example for a template that generates both See :ref:`server-selinux` for more information on the ``secontext`` attribute and managing SELinux in general. - -:info and info files -==================== - -.. deprecated:: 1.3.0 - -Historically, Bcfg2 also accepted the use of ``:info`` and ``info`` -files, which function the same as ``info.xml``, but are not XML. They -lack the ability to specify different permissions based on client, -group, or path, and cannot be used to specify ACLs, either. - -An example ``:info`` or ``info`` file would look like:: - - owner: www - group: www - mode: 0755 - -All attributes allowed on the ``<Info>`` tag of an ``info.xml`` file -can be used in an ``:info`` or ``info`` file. - -You should not use more than one ``:info``, ``info``, or ``info.xml`` -file for a single entry. diff --git a/doc/server/plugins/connectors/properties.txt b/doc/server/plugins/connectors/properties.txt index 47e82fdbf..a6f6af741 100644 --- a/doc/server/plugins/connectors/properties.txt +++ b/doc/server/plugins/connectors/properties.txt @@ -120,6 +120,8 @@ in ``bcfg2.conf``:: [properties] writes_enabled = false +.. _server-plugins-connectors-properties-xml: + XML Property Files ------------------ @@ -266,47 +268,13 @@ Encrypted Properties data .. versionadded:: 1.3.0 You can encrypt selected data in XML Properties files to protect that -data from other people who need access to the repository. See -:ref:`server-encryption-configuration` for details on configuring -encryption passphrases. The data is decrypted transparently -on-the-fly by the server; you never need to decrypt the data in your -templates. Encryption is only supported on XML properties files. - -.. note:: - - This feature is *not* intended to secure the files against a - malicious attacker who has gained access to your Bcfg2 server, as - the encryption passphrases are held in plaintext in - ``bcfg2.conf``. This is only intended to make it easier to use a - single Bcfg2 repository with multiple admins who should not - necessarily have access to each other's sensitive data. - -Properties files are encrypted on a per-element basis; that is, rather -than encrypting the whole file, only the character content of -individual elements is encrypted. This makes it easier to track -changes to the file in a VCS, and also lets unprivileged users work -with the other data in the file. Only character content of an element -can be encrypted; attribute content and XML elements themselves cannot -be encrypted. - -By default, decryption is *strict*; that is, if any element cannot be -decrypted, parsing of the file is aborted. If you wish for parsing to -continue, with unencryptable elements simply skipped, then you can set -decryption to *lax* in one of two ways: - -* Set ``decrypt=lax`` in the ``[encryption]`` section of - ``bcfg2.conf`` to set lax decryption on all files by default; or -* Set the ``decrypt="lax"`` attribute on the top-level ``Properties`` - tag of a Properties file to set lax decryption for a single file. - -Note that you could, for instance, set lax decryption by default, and -then set strict decryption on individual files. - -To encrypt or decrypt a file, use :ref:`bcfg2-crypt`. - -See :ref:`server-encryption` for more details on encryption in Bcfg2 -in general. +data from other people who need access to the repository. The +data is decrypted transparently on-the-fly by the server; you never +need to decrypt the data in your templates. Encryption is only +supported on XML properties files. +See :ref:`server-encryption` for details on encryption in general, and +:ref:`xml-encryption` for details on encryption in XML files. Accessing Properties contents from Genshi Templates =================================================== diff --git a/doc/server/plugins/connectors/templatehelper.txt b/doc/server/plugins/connectors/templatehelper.txt index 374aeb171..c719f93cb 100644 --- a/doc/server/plugins/connectors/templatehelper.txt +++ b/doc/server/plugins/connectors/templatehelper.txt @@ -31,7 +31,7 @@ helpers will be available to all clients. Writing Helpers =============== -A helper module is just a Python module with three special conditions: +A helper module is just a Python module with several special conditions: * The filename must end with ``.py`` * The module must have an attribute, ``__export__``, that lists all of @@ -43,6 +43,12 @@ A helper module is just a Python module with three special conditions: an underscore or double underscore is bad form, and may also produce errors. +Additionally, the module *may* have an attribute, ``__default__``, +that lists all of the symbols that you wish to include by default in +the template namespace. ``name``, ``metadata``, ``source_path``, +``repo``, and ``path`` are reserved names, and should not be included +in ``__default__``. + See ``examples/TemplateHelper`` for examples of helper modules. Usage @@ -54,17 +60,23 @@ a HelperModule object will has, as attributes, all symbols listed in ``__export__``. For example, consider this helper module:: __export__ = ["hello"] - + __default__ = ["pining"] + def hello(metadata): return "Hello, %s!" % metadata.hostname + def pining(text): + return "It's pinin' for the %s!" % text + To use this in a Genshi template, we could do:: ${metadata.TemplateHelper['hello'].hello(metadata)} + ${pining("fjords")} The template would produce:: Hello, foo.example.com! + It's pinin' for the fjords! Note that the client metadata object is not passed to a helper module in any magical way; if you want to access the client metadata object diff --git a/doc/server/plugins/generators/account.txt b/doc/server/plugins/generators/account.txt deleted file mode 100644 index 99c35c814..000000000 --- a/doc/server/plugins/generators/account.txt +++ /dev/null @@ -1,115 +0,0 @@ -.. -*- mode: rst -*- - -.. _server-plugins-generators-account: - -======= -Account -======= - -The account plugin manages authentication data, including - -* ``/etc/passwd`` -* ``/etc/group`` -* ``/etc/security/limits.conf`` -* ``/etc/sudoers`` -* ``/root/.ssh/authorized_keys`` - -User access data is stored in three files in the Account directory: - -* superusers (a list of users who always have root privs) -* rootlist (a list of user:host pairs for scoped root privs) -* useraccess (a list of user:host pairs for login access) - -SSH keys are stored in files named $username.key; these are installed -into root's authorized keys for users in the superusers list as well as -for the pertitent users in the rootlike file (for the current system). - -Authentication data is read in from (static|dyn).(passwd|group) The static -ones are for system local ones, while the dyn. versions are for external -synchronization (from ldap/nis/etc). There is also a static.limits.conf -that provides the limits.conf header and any static entries. - -Files in the Account directory: - -``<username>.key`` - - **Format**: The SSH public key for user <username>. - - If the user is in the "rootlike" or "superusers" group, these - keys will be appended to ``/root/.ssh/auth`` - -``useraccess`` - - **Format**: "user:hostname" on each line. - - Describes who may login where (via PAMs - ``/etc/security/limits.conf``). Everybody else will be denied - access.(?) - - **Example**: - - If Alice should be able to access host "foo", Bob should access - "foo" and "bar":: - - alice:foo.example.com - bob:foo.example.com - bob:bar.example.com - -``rootlike`` - - **Format**: "user:hostname" on each line. - - Describes who will be allowed root access where. The user may - login via public key and use sudo. - - **Example**: - - If Chris should be root only on host "foo":: - - chris:foo.example.com - -``superusers`` - - **Format**: usernames, separated by spaces or newlines. (Any whitespace that makes pythons split() happy.) - - Describes who will be allowed root access on all hosts. The user - may login via public key and use sudo. - - **Example**: - - Daniel, Eve and Faith are global admins:: - - daniel eve - faith - -``static.passwd``, ``static.group`` - - **Format**: Lines from ``/etc/passwd`` or ``/etc/group`` - - These entries are appended to the passwd and group files - (in addition to the auto-generated entries from "useraccess", - "rootlike" and "superusers" above) without doing anything else. - -``dyn.passwd``, ``dyn.group`` - - **Format**: Lines from ``/etc/passwd`` or ``/etc/group`` - - Similar to "static.*" above, but for entries that are managed "on - the network" (yp, LDAP, ...), so it is most likely periodically - (re)filled. - -``static.limits.conf`` - - **Format**: Lines from ``/etc/security/limit.conf`` - - These limits will be appended to limits.conf (in addition to - the auto-generated entries from "useraccess", "rootlike" and - "superusers" above). - -``static.sudoers`` - - **Format**: Lines from ``/etc/sudoers`` - - These lines will be appended to to sudoers file (in addition - to the auto-generated entries from "useraccess", "rootlike" and - "superusers" above). diff --git a/doc/server/plugins/generators/cfg.txt b/doc/server/plugins/generators/cfg.txt index 0f0601105..7a404c824 100644 --- a/doc/server/plugins/generators/cfg.txt +++ b/doc/server/plugins/generators/cfg.txt @@ -102,9 +102,8 @@ Genshi Templates ---------------- Genshi templates allow you to use the `Genshi -<http://genshi.edgewall.org>`_ templating system. This is similar to -the deprecated :ref:`server-plugins-generators-tgenshi-index` plugin. -Genshi templates should be named with a ``.genshi`` extension, e.g.:: +<http://genshi.edgewall.org>`_ templating system. Genshi templates +should be named with a ``.genshi`` extension, e.g.:: % ls Cfg/etc/motd info.xml motd.genshi @@ -214,9 +213,8 @@ Cheetah Templates ----------------- Cheetah templates allow you to use the `cheetah templating system -<http://www.cheetahtemplate.org/>`_. This is similar to -the deprecated :ref:`server-plugins-generators-tcheetah` plugin. -Cheetah templates should be named with a ``.cheetah`` extension, e.g.:: +<http://www.cheetahtemplate.org/>`_. Cheetah templates should be +named with a ``.cheetah`` extension, e.g.:: % ls Cfg/etc/motd info.xml motd.cheetah @@ -415,7 +413,7 @@ See :ref:`server-encryption` for more details on encryption in Bcfg2 in general. ``pubkey.xml`` -~~~~~~~~~~~~~~~ +~~~~~~~~~~~~~~ ``pubkey.xml`` only ever contains a single line: @@ -563,110 +561,162 @@ Example Hopefully, the performance concerns can be resolved in a future release and these features can be added. +.. _server-plugins-generators-cfg-ssl-certificates: + +SSL Keys and Certificates +========================= + +Cfg can also create SSL keys and certs on the fly, and store the +generated data in the repo so that subsequent requests do not result +in repeated key/cert recreation. In the event that a new key or cert +is needed, the old file can simply be removed from the +repository, and the next time that host checks in, a new file will be +created. If that file happens to be the key, any dependent +certificates will also be regenerated. + +See also :ref:`appendix-guides-sslca_howto` for a detailed example +that uses the SSL key management feature to automate Bcfg2 certificate +authentication. + +Getting started +--------------- + +In order to use the SSL certificate generation feature, you must first +have at least one CA configured on your system. For details on +setting up your own OpenSSL based CA, please see +http://www.openssl.org/docs/apps/ca.html for details of the suggested +directory layout and configuration directives. + +For SSL cert generation to work, the openssl.cnf (or other +configuration file) for that CA must contain full (not relative) +paths. + +#. Add a section to your ``/etc/bcfg2.conf`` called ``sslca_foo``, + replacing foo with the name you wish to give your CA so you can + reference it in certificate definitions. (If you only have one CA, + you can name it ``sslca_default``, and it will be the default CA + for all other operations.) + +#. Under that section, add a ``config`` option that gives the location + of the ``openssl.cnf`` file for your CA. + +#. If necessary, add a ``passphrase`` option containing the passphrase + for the CA's private key. If no passphrase is entry exists, it is + assumed that the private key is stored unencrypted. + +#. Optionally, add a ``chaincert`` option that points to the location + of your ssl chaining certificate. This is used when preexisting + certificate hostfiles are found, so that they can be validated and + only regenerated if they no longer meet the specification. If + you're using a self signing CA this would be the CA cert that you + generated. If the chain cert is a root CA cert (e.g., if it is a + self-signing CA), also add an entry ``root_ca = true``. If + ``chaincert`` is omitted, certificate verification will not be + performed. + +#. Once all this is done, you should have a section in your + ``/etc/bcfg2.conf`` that looks similar to the following:: + + [sslca_default] + config = /etc/pki/CA/openssl.cnf + passphrase = youReallyThinkIdShareThis? + chaincert = /etc/pki/CA/chaincert.crt + root_ca = true + +#. You are now ready to create key and certificate definitions. For + this example we'll assume you've added Path entries for the key, + ``/etc/pki/tls/private/localhost.key``, and the certificate, + ``/etc/pki/tls/certs/localhost.crt`` to a bundle. + +#. Within the ``Cfg/etc/pki/tls/private/localhost.key`` directory, + create a `sslkey.xml`_ file containing the following: + + .. code-block:: xml + + <KeyInfo/> + +#. This will cause the generation of an SSL key when a client requests + that Path. (By default, it will be a 2048-bit RSA key; see + `sslkey.xml`_ for details on how to change the key type and size.) + +#. Similarly, create `sslcert.xml`_ in + ``Cfg/etc/pki/tls/certs/localhost.cfg/``, containing the following: + + .. code-block:: xml + + <CertInfo> + <Cert key="/etc/pki/tls/private/localhost.key" ca="foo"/> + </CertInfo> + +#. When a client requests the cert path, a certificate will be + generated using the key hostfile at the specified key location, + using the CA matching the ``ca`` attribute. ie. ``ca="foo"`` will + match ``[sslca_default]`` in your ``/etc/bcfg2.conf`` + +The :ref:`Bcfg2 bundle example +<server-plugins-structures-bundler-bcfg2-server>` contains entries to +automate the process of setting up a CA. + Configuration ------------- -In addition to ``privkey.xml`` and ``authorized_keys.xml``, described -above, the behavior of the SSH key generation feature can be -influenced by several options in the ``[sshkeys]`` section of -``bcfg2.conf``: +``bcfg2.conf`` +~~~~~~~~~~~~~~ -+----------------+---------------------------------------------------------+-----------------------+------------+ -| Option | Description | Values | Default | -+================+=========================================================+=======================+============+ -| ``passphrase`` | Use the named passphrase to encrypt private keys on the | String | None | -| | filesystem. The passphrase must be defined in the | | | -| | ``[encryption]`` section. See :ref:`server-encryption` | | | -| | for more details on encryption in Bcfg2 in general. | | | -+----------------+---------------------------------------------------------+-----------------------+------------+ -| ``category`` | Generate keys specific to groups in the given category. | String | None | -| | It is best to pick a category that all clients have a | | | -| | group from. | | | -+----------------+---------------------------------------------------------+-----------------------+------------+ - -Deltas -====== +In ``bcfg2.conf``, you must declare your CA(s) in ``[sslca_<name>]`` +sections. At least one is required. Valid options are detailed +below, in `Cfg Configuration`_. -.. note:: +Only the ``config`` option is required; i.e., the simplest possible CA +section is:: + + [sslca_default] + config = /etc/pki/CA/openssl.cnf + +``sslcert.xml`` +~~~~~~~~~~~~~~~ + +.. xml:schema:: sslca-cert.xsd + :linktotype: + :inlinetypes: CertType + +Example +^^^^^^^ + +.. code-block:: xml + + <CertInfo> + <subjectAltName>test.example.com</subjectAltName> + <Group name="apache"> + <Cert key="/etc/pki/tls/private/foo.key" days="730"/> + </Group> + <Group name="nginx"> + <Cert key="/etc/pki/tls/private/foo.key" days="730" + append_chain="true"/> + </Group> + </CertInfo> + +``sslkey.xml`` +~~~~~~~~~~~~~~ + +.. xml:schema:: sslca-key.xsd + :linktotype: + :inlinetypes: KeyType + +Example +^^^^^^^ + +.. code-block:: xml + + <KeyInfo> + <Group name="fast"> + <Key type="rsa" bits="1024"/> + </Group> + <Group name="secure"> + <Key type="rsa" bits="4096"/> + </Group> + </KeyInfo> - In Bcfg2 1.3 and newer, deltas are deprecated. It is recommended - that you use templates instead. The - :ref:`TemplateHelper plugin - <server-plugins-connectors-templatehelper>` comes with an example - helper that can be used to include other files easily, a subset of - cat file functionality. ``bcfg2-lint`` checks for deltas and - warns about them. - -.. warning:: - - In Bcfg2 1.3, deltas **do not** work with `SSH key or - authorized_keys generation <SSH Keys>`_. - -Bcfg2 has finer grained control over how to deliver configuration -files to a host. Let's say we have a Group named file-server. Members -of this group need the exact same ``/etc/motd`` as all other hosts except -they need one line added. We could copy motd to ``motd.G01_file-server``, -add the one line to the Group specific version and be done with it, -but we're duplicating data in both files. What happens if we need to -update the motd? We'll need to remember to update both files then. Here's -where deltas come in. A delta is a small change to the base file. There -are two types of deltas: cats and diffs. The cat delta simply adds or -removes lines from the base file. The diff delta is more powerful since -it can take a unified diff and apply it to the base configuration file -to create the specialized file. Diff deltas should be used very sparingly. - -Cat Files ---------- - -Continuing our example for cat files, we would first create a file named -``motd.G01_file-server.cat``. The .cat suffix designates that the file is -a diff. We would then edit that file and add the following line:: - - +This is a file server - -The **+** at the begining of the file tells Bcfg2 that the line should be -appended to end of the file. You can also start a line with **-** to tell -Bcfg2 to remove that exact line wherever it might be in the file. How do -we know what base file Bcfg2 will choose to use to apply a delta? The -same rules apply as before: Bcfg2 will choose the highest priority, -most specific file as the base and then apply deltas in the order of -most specific and then increasing in priority. What does this mean in -real life. Let's say our machine is a web server, mail server, and file -server and we have the following configuration files:: - - motd - motd.G01_web-server - motd.G01_mail-server.cat - motd.G02_file-server.cat - motd.H_bar.example.com - motd.H_foo.example.com.cat - -If our machine isn't *foo.example.com* or *bar.example.com*, but -is a web server, then Bcfg2 would choose ``motd.G01_web-server`` as -the base file. It is the most specific base file for this host. Bcfg2 -would apply the ``motd.G01_mail-server.cat`` delta to the -``motd.G01_web-server`` base file. It is the least specific -delta. Bcfg2 would then apply the ``motd.G02_file-server.cat`` delta -to the result of the delta before it. - -If our machine is *foo.example.com* and a web server, then Bcfg2 would -choose ``motd.G01_web-server`` as the base file. It is the most -specific base file for this host. Bcfg2 would apply the -``motd.H_foo.example.com.cat`` delta to the ``motd.G01_web-server`` -base file. The reason the other deltas aren't applied to -*foo.example.com* is because a **.H_** delta is more specific than a -**.G##_** delta. Bcfg2 applies all the deltas at the most specific -level. - -If our machine is *bar.example.com*, then Bcfg2 would chose -``motd.H_foo.example.com`` as the base file because it is the most -specific base file for this host. Regardless of the groups -*bar.example.com* is a member of, **no cat files** would be applied, -because only cat files as specific or more specific than the base file -are applied. (In other words, if a group-specific base file is -selected, only group- or host-specific cat files can be applied; if a -host-specific base file is selected, only host-specific cat files can -be applied.) .. _server-plugins-generators-cfg-validation: @@ -721,3 +771,56 @@ File permissions File permissions for entries handled by Cfg are controlled via the use of :ref:`server-info` files. Note that you **cannot** use both a Permissions entry and a Path entry to handle the same file. + +.. _server-plugins-generators-cfg-configuration: + +Cfg Configuration +================= + +The behavior of many bits of the Cfg plugin can be configured in +``bcfg2.conf`` with the following options. + +In addition to ``privkey.xml`` and ``authorized_keys.xml``, described +above, the behavior of the SSH key generation feature can be +influenced by several options in the ``[sshkeys]`` section of +``bcfg2.conf``: + ++-------------+----------------+---------------------------------------------------------+-----------------------+------------+ +| Section | Option | Description | Values | Default | ++=============+================+=========================================================+=======================+============+ +| ``cfg`` | ``passphrase`` | Use the named passphrase to encrypt created data on the | String | None | +| | | filesystem. (E.g., SSH and SSL keys.) The passphrase | | | +| | | must be defined in the ``[encryption]`` section. | | | ++-------------+----------------+---------------------------------------------------------+-----------------------+------------+ +| ``cfg`` | ``category`` | Generate data (e.g., SSH keys, SSL keys and certs) | String | None | +| | | specific to groups in the given category. It is best to | | | +| | | pick a category that all clients have a group from. | | | ++-------------+----------------+---------------------------------------------------------+-----------------------+------------+ +| ``cfg`` | ``validation`` | Whether or not to perform `Content Validation`_ | Boolean | True | +| | | specific to groups in the given category. It is best to | | | +| | | pick a category that all clients have a group from. | | | ++-------------+----------------+---------------------------------------------------------+-----------------------+------------+ +| ``sshkeys`` | ``passphrase`` | Override the global Cfg passphrase with a specific | String | None | +| | | passphrase for encrypting created SSH private keys. | | | ++-------------+----------------+---------------------------------------------------------+-----------------------+------------+ +| ``sshkeys`` | ``category`` | Override the global Cfg category with a specific | String | None | +| | | category for created SSH keys. | | | ++-------------+----------------+---------------------------------------------------------+-----------------------+------------+ +| ``sslca`` | ``passphrase`` | Override the global Cfg passphrase with a specific | String | None | +| | | passphrase for encrypting created SSL keys. | | | ++-------------+----------------+---------------------------------------------------------+-----------------------+------------+ +| ``sslca`` | ``category`` | Override the global Cfg category with a specific | String | None | +| | | category for created SSL keys and certs. | | | ++-------------+----------------+---------------------------------------------------------+-----------------------+------------+ +| ``sslca_*`` | ``config`` | Path to the openssl config for the CA | String | None | ++-------------+----------------+---------------------------------------------------------+-----------------------+------------+ +| ``sslca_*`` | ``passphrase`` | Passphrase for the CA private key | String | None | ++-------------+----------------+---------------------------------------------------------+-----------------------+------------+ +| ``sslca_*`` | ``chaincert`` | Path to the SSL chaining certificate for verification | String | None | ++-------------+----------------+---------------------------------------------------------+-----------------------+------------+ +| ``sslca_*`` | ``root_ca`` | Whether or not ``<chaincert>`` is a root CA (as | Boolean | False | +| | | opposed to an intermediate cert) | | | ++-------------+----------------+---------------------------------------------------------+-----------------------+------------+ + +See :ref:`server-encryption` for more details on encryption in Bcfg2 +in general. diff --git a/doc/server/plugins/generators/decisions.txt b/doc/server/plugins/generators/decisions.txt index 9a40ab8fd..f0afeba0a 100644 --- a/doc/server/plugins/generators/decisions.txt +++ b/doc/server/plugins/generators/decisions.txt @@ -29,18 +29,23 @@ client's whitelists or blacklists. is not used. See `Decision Mode`_ below. The Decisions plugin uses a directory in the Bcfg2 repository called -Decisions. Files in the Decisions subdirectory are named similarly to -files managed by Cfg and Probes, so you can use host- and -group-specific files and the like after their basename. File basenames -are either ``whitelist`` or ``blacklist``. These files have a simple -format; the following is an example. +Decisions, which may contain two files: ``whitelist.xml`` and +``blacklist.xml``. These files have a simple format: + +.. xml:type:: DecisionsType + :linktotype: + :noautodep: py:genshiElements + +For example: .. code-block:: xml - $ cat Decisions/whitelist + $ cat Decisions/whitelist.xml <Decisions> <Decision type='Service' name='*'/> - <Decision type='Path' name='/etc/apt/apt.conf'/> + <Group name="debian"> + <Decision type='Path' name='/etc/apt/apt.conf'/> + </Group> </Decisions> This example, included as a whitelist due to its name, enables all services, @@ -60,12 +65,6 @@ list. This list is sent to the client. control these via their respective options (``-I`` or ``-n``, for example). -To add syntax highlighting to Decisions files in vim and emacs, you -can add comments such as this:: - - <Decisions><!--*- mode: xml; -*--> - <!-- vim: set ft=xml : --> - Decision Mode ============= diff --git a/doc/server/plugins/generators/examples/genshi/ganglia.txt b/doc/server/plugins/generators/examples/genshi/ganglia.txt index 3a20fde92..d7030e990 100644 --- a/doc/server/plugins/generators/examples/genshi/ganglia.txt +++ b/doc/server/plugins/generators/examples/genshi/ganglia.txt @@ -33,7 +33,7 @@ Bundler/ganglia.xml .. code-block:: xml - <Bundle name='ganglia'> + <Bundle> <Package name='ganglia-gmond' /> <Package name='ganglia-gmond-modules-python' /> <Path name='/etc/ganglia/gmond.conf' /> diff --git a/doc/server/plugins/generators/hostbase.txt b/doc/server/plugins/generators/hostbase.txt deleted file mode 100644 index c6007f70e..000000000 --- a/doc/server/plugins/generators/hostbase.txt +++ /dev/null @@ -1,228 +0,0 @@ -.. -*- mode: rst -*- - -.. _server-plugins-generators-hostbase: - -======== -Hostbase -======== - -IP management system built on top of Bcfg2. It has four main parts: a -django data model, a web frontend, command-line utilities, and a Bcfg2 -plugin that generates dhcp, dns, and yp configuration files. - -Installation -============ - -Installation of Hostbase requires installation of a python module, -configuration of database (mysql or postgres), and configuration of an -Apache webserver with mod_python. Hostbase was developed using MySQL, -so this document is aimed at MySQL users. - -Prerequisites -------------- - -* `mysql`_ -* `python-mysqldb`_ -* `Django`_ - -.. _Django: http://www.djangoproject.com -.. _python-mysqldb: http://mysql-python.sourceforge.net/MySQLdb.html -.. _mysql: http://www.mysql.com/ - -Configure the database ----------------------- - -Create the hostbase database and a user. For MySQL users:: - - mysql> CREATE DATABASE hostbase - mysql> quit - - systemprompt#: mysql -u root hostbase - mysql> GRANT ALL PRIVILEGES ON *.* TO hostbaseuser@mycomputer.private.net IDENTIFIED - BY 'password' WITH GRANT OPTION; - mysql> quit - -As of Bcfg2 v0.8.7 configuration options for Hostbase have moved to -``/etc/bcfg2.conf``. There is an example bcfg2.conf with Hostbase -options located at ``bcfg2-tarball/examples/bcfg2.confHostbase``. -Edit the hostbase options to correspond to the database you've -initialized and copy the configuration to ``/etc/bcfg2.conf``. To -finish creating the database, from your ``path to -python/Bcfg2/Server/Hostbase`` directory, run ``python manage.py -syncdb`` to do all table creation. - -Configure the web interface ---------------------------- - -Now it's possible to explore the Hostbase web interface. For -curiosity, you can run Django's built-in development server to take a -peek. Do this by running ``python manage.py runserver -[servername:port]`` from your Hostbase directory. Django will -default to ``localhost:8000`` if no server or port is entered. Now -you can explore the web interface. Try adding a host and a zone. -You'll see that a ".rev" zone already exists. This is where -information for reverse files will go. - -For production, you'll want to have this configured for Apache with -mod_python. Here is an example of how to configure Hostbase as a -virtual host. - -.. code-block:: html - - <VirtualHost hostbase.mcs.anl.gov:80> - ServerAdmin systems@mcs.anl.gov - - DocumentRoot /var/www/hostbase/ - <Directory /> - AllowOverride None - </Directory> - - # Possible values include: debug, info, notice, warn, error, crit, - # alert, emerg. - LogLevel warn - - ServerSignature Off - - # Stop TRACE/TRACK vulnerability - <IfModule mod_rewrite.c> - RewriteEngine on - RewriteCond %{REQUEST_METHOD} ^(TRACE|TRACK) - RewriteRule .* - [F] - </IfModule> - - Redirect / https://hostbase.mcs.anl.gov/ - </VirtualHost> - - <VirtualHost hostbase.mcs.anl.gov:443> - ServerAdmin systems@mcs.anl.gov - - DocumentRoot /var/www/hostbase/ - <Directory /> - AllowOverride None - </Directory> - - # Possible values include: debug, info, notice, warn, error, crit, - # alert, emerg. - LogLevel warn - - ServerSignature Off - - # Stop TRACE/TRACK vulnerability - <IfModule mod_rewrite.c> - RewriteEngine on - RewriteCond %{REQUEST_METHOD} ^(TRACE|TRACK) - RewriteRule .* - [F] - </IfModule> - - SSLEngine On - SSLCertificateFile /etc/apache2/ssl/hostbase_server.crt - SSLCertificateKeyfile /etc/apache2/ssl/hostbase_server.key - - <Location "/"> - SetHandler python-program - PythonHandler django.core.handlers.modpython - SetEnv DJANGO_SETTINGS_MODULE Bcfg2.Server.Hostbase.settings - PythonDebug On - </Location> - <Location "/site_media/"> - SetHandler None - </Location> - </VirtualHost> - - -You'll need to copy the contents of ``Hostbase/media`` into -``/var/www/hostbase/site_media`` in this configuration to serve the -correct css files. - -Enable the Hostbase plugin --------------------------- - -Now that the database is accessible and there is some data in it, you can -enable the Hostbase plugin on your Bcfg2 server to start generating some -configuration files. All that needs to be done is to add ``Hostbase`` -to the end of the list of generators in your bcfg2.conf file. To see -what's being generated by Hostbase, fire up a Bcfg2 development server: -``bcfg2-info``. For more information on how to use the Bcfg2 development -server, type help at the prompt. For our purposes, type ``debug``. -This will bring you to an interactive python prompt where you can access -bcfg's core data. - -.. code-block:: python - - for each in bcore.plugins['Hostbase'].filedata: - print each - - -The above loop will print out the name of each file that was generated -by Hostbase. You can see the contents of any of these by typing ``print -bcore.plugins['Hostbase'].filedata[filename]``. - -Create a bundle ---------------- - -Bcfg2 needs a way to distribute the files generated by Hostbase. -We'll do this with a bundle. In bcfg's ``Bundler`` directory, touch -``hostbase.xml``. - -.. code-block:: xml - - <Bundle name='hostbase' version='0.1'> - <Package name='dhcp3-server'/> - <Package name='bind9'/> - <Service name='dhcp3-server'/> - <Service name='bind9'/> - <Path name='/etc/dhcp3/dhcpd.conf'/> - <Path name='/etc/bind/[your domain]'/> - <Path name='/etc/bind/xxx.xxx.xxx.rev'/> - </Bundle> - -The above example is a bundle that will deliver both dhcp and dns files. -This can be trivially split into separate bundles. It is planned that -Hostbase will eventually be able to generate the list of ``Paths`` -in its bundles automatically. - -Do a Hostbase push ------------------- - -You'll want to be able to trigger the Hostbase plugin to rebuild -it's config files and push them out when data has been modified -in the database. This can be done through and XMLRPC function -available from the Bcfg2 server. From a client that is configured -to receive one or more hostbase bundles, you'll need to first -edit your ``python/site-packages/Bcfg2/Client/Proxy.py`` file. -Add ``'Hostbase.rebuildState'`` to the list of methods in the Bcfg2 -client proxy object. The modified list is shown below: - -.. code-block:: python - - class bcfg2(ComponentProxy): - '''bcfg2 client code''' - name = 'bcfg2' - methods = ['AssertProfile', 'GetConfig', 'GetProbes', 'RecvProbeData', 'RecvStats', 'Hostbase.rebuildState'] - -Now copy the file ``hostbasepush.py`` from ``bcfg2/tools`` in the Bcfg2 -source to your machine. When this command is run as root, it triggers -the Hostbase to rebuild it's files, then runs the Bcfg2 client on your -local machine to grab the new configs. - -NIS Authentication -================== - -Django allows for custom authentication backends to its login procedure. -Hostbase has an NIS authentication backend that verifies a user to be -in the unix group allowed to modify Hostbase. - -To enable this feature: - -* first edit your ``Hostbase/settings.py`` file and uncomment - the line **Hostbase.backends.NISBackend** in the list of - *AUTHENTICATION_BACKENDS* -* enter the name of the unix group you want to give access to Hostbase - in the *AUTHORIZED_GROUP* variable -* in your ``Hostbase/hostbase/views.py`` file at the very bottom, - uncomment the block(s) of lines that give you the desired level - of access - -Hostbase will now direct the user to a login page if he or she is not -authorized to view a certain page. Users should log in with their -regular Unix username and password. diff --git a/doc/server/plugins/generators/nagiosgen.txt b/doc/server/plugins/generators/nagiosgen.txt index 0ae922fa3..1ccdd66c1 100644 --- a/doc/server/plugins/generators/nagiosgen.txt +++ b/doc/server/plugins/generators/nagiosgen.txt @@ -12,7 +12,7 @@ This page describes the installation and use of the `NagiosGen`_ plugin. Update ``/etc/bcfg2.conf``, adding NagiosGen to plugins:: - plugins = Base,Bundler,Cfg,...,NagiosGen + plugins = Bundler,Cfg,...,NagiosGen Create the NagiosGen directory:: @@ -124,21 +124,21 @@ Create a nagios Bcfg2 bundle ``/var/lib/bcfg2/Bundler/nagios.xml`` .. code-block:: xml - <Bundle name='nagios' version='2.0'> + <Bundle> <Path name='/etc/nagiosgen.status'/> - <Group name='rh'> + <Group name='redhat'> <Group name='nagios-server'> - <Path name='/etc/nagios/nagiosgen.cfg'/> + <Path name='/etc/nagios/conf.d/bcfg2.cfg'/> <Package name='libtool-libs'/> <Package name='nagios'/> <Package name='nagios-www'/> <Service name='nagios'/> </Group> </Group> - <Group name='debian-lenny'> + <Group name='debian-wheezy'> <Group name='nagios-server'> - <Path name='/etc/nagios3/nagiosgen.cfg' - altsrc='/etc/nagios/nagiosgen.cfg'/> + <Path name='/etc/nagios3/conf.d/bcfg2.cfg' + altsrc='/etc/nagios/conf.d/bcfg2.cfg'/> <Package name='nagios3'/> <Package name='nagios3-common'/> <Package name='nagios3-doc'/> @@ -161,10 +161,6 @@ Assign clients to nagios groups in <Bundle name='nagios'/> </Group> -Update nagios configuration file to use ``nagiosgen.cfg``:: - - cfg_file=/etc/nagios/nagiosgen.cfg - Note that some of these files are built on demand, each time a client in group "nagios-server" checks in with the Bcfg2 server. Local nagios instances can be configured to use the NagiosGen directory in the Bcfg2 diff --git a/doc/server/plugins/generators/packages.txt b/doc/server/plugins/generators/packages.txt index 31f3ccf22..8b317552f 100644 --- a/doc/server/plugins/generators/packages.txt +++ b/doc/server/plugins/generators/packages.txt @@ -18,14 +18,10 @@ through those channels. Limiting sources to groups ========================== -`sources.xml`_ processes ``<Group>`` and ``<Client>`` tags just like -Bundles. In addition to any groups or clients specified that way, -clients must be a member of the appropriate architecture group as -specified in a Source stanza. In total, in order for a source to be -associated with a client, the client must be in any explicit groups or -clients specified in `sources.xml`_, and any specified architecture -groups. If `"Magic Groups"`_ are enabled, then the client must be a -member of a matching magic group as well. +``Packages/sources.xml`` processes ``<Group>`` and ``<Client>`` tags +just like Bundles. In addition to any groups or clients specified that +way, clients must be a member of the appropriate architecture group as +specified in a Source stanza. Memberships in architecture groups is needed so that Packages can map software sources to clients. There is no other way to handle this than @@ -36,62 +32,6 @@ source to which they apply (based on group memberships, as described above). Packages and dependencies are resolved from all applicable sources. -.. note:: - - To recap, a client needs to be a member of the **Architecture** - group and any other groups defined in your - `sources.xml`_ file in order for the client to be - associated to the proper sources. If you are using - :ref:`server-plugins-generators-packages-magic-groups`, then a - client must also be a member of the appropriate OS group. - -.. _server-plugins-generators-packages-magic-groups: - -"Magic Groups" -============== - -.. deprecated:: 1.3.0 - -Packages has the ability to use a feature known as "magic groups"; it -is the only plugin to use that feature. Most plugins operate based on -client group memberships, without any concern for the particular names -chosen for groups by the user. The Packages plugin is the sole -exception to this rule. Packages needs to "know" two different sorts -of facts about clients. The first is the basic OS/distro of the -client, enabling classes of sources. The second is the architecture of -the client, enabling sources for a given architecture. In addition to -these magic groups, each source may also specify non-magic groups to -limit the source's applicability to group member clients. - -+--------+----------+--------------+ -| Source | OS Group | Architecture | -+========+==========+==============+ -| Apt | debian | i386 | -+--------+----------+--------------+ -| Apt | ubuntu | amd64 | -+--------+----------+--------------+ -| Apt | nexenta | | -+--------+----------+--------------+ -| Apt | apt | | -+--------+----------+--------------+ -| Yum | redhat | i386 | -+--------+----------+--------------+ -| Yum | centos | x86_64 | -+--------+----------+--------------+ -| Yum | fedora | | -+--------+----------+--------------+ -| Yum | yum | | -+--------+----------+--------------+ - -Magic OS groups are disabled by default in Bcfg2 1.3 and greater. If -you require magic groups, you can enable them by setting -``magic_groups`` to ``1`` in the ``[packages]`` section of -``bcfg2.conf``. - -Magic groups will be removed in a future release. - -Magic architecture groups cannot be disabled. - Setup ===== @@ -102,14 +42,13 @@ Three basic steps are required for Packages to work properly. software repositories should be used, and which clients are eligible to use each one. #. Ensure that clients are members of the proper groups. Each client - should be a member of all of the groups listed in the `sources.xml` - (like ubuntu-intrepid or centos-5.2 in the following examples), one - of the architecture groups listed in the source configuration - (i386, amd64 or x86_64 in the following examples), and one of the - magic groups listed above, if magic groups are enabled. '''Failure - to do this will result in the source either not applying to the - client, or only architecture independent packages being made - available to the client.''' + should be a member of all of the groups listed in the + ``sources.xml`` (like ubuntu-intrepid or centos-5.2 in the + following examples), and one of the architecture groups listed in + the source configuration (i386, amd64 or x86_64 in the following + examples). '''Failure to do this will result in the source either + not applying to the client, or only architecture independent + packages being made available to the client.''' #. Add Package entries to bundles. #. Sit back and relax, as dependencies are resolved, and automatically added to client configurations. @@ -122,6 +61,7 @@ Packages plugin. It processes ``<Group>`` and ``<Client>`` tags just like Bundles. The primary element in ``sources.xml`` is the Source tag: .. xml:element:: Source + :noautodep: py:genshiElements Handling GPG Keys ----------------- @@ -198,9 +138,7 @@ processed. After this phase, but before entry binding, a list of packages and the client metadata instance is passed into Packages' resolver. This process determines a superset of packages that will fully satisfy dependencies of all package entries included in structures, and reports -any prerequisites that cannot be satisfied. This facility should largely -remove the need to use the :ref:`Base <server-plugins-structures-base>` -plugin. +any prerequisites that cannot be satisfied. Disabling dependency resolution ------------------------------- @@ -451,7 +389,7 @@ attribute, e.g.: .. code-block:: xml - <Bundle name="yum"> + <Bundle> <Group name="sles"> <Path name="/etc/yum/yum.repos.d/bcfg2.repo" altsrc="/etc/yum.repos.d/bcfg2.repo"/> diff --git a/doc/server/plugins/generators/pkgmgr.txt b/doc/server/plugins/generators/pkgmgr.txt index ace7c16ef..8d9979ba0 100644 --- a/doc/server/plugins/generators/pkgmgr.txt +++ b/doc/server/plugins/generators/pkgmgr.txt @@ -10,10 +10,10 @@ The Pkgmgr plugin resolves the Abstract Configuration Entity "Package" to a package specification that the client can use to detect, verify and install the specified package. -For a package specification to be included in the Literal configuration -the name attribute from an Abstract Package Tag (from Base or Bundler) -must match the name attribute of a Package tag in Pkgmgr, along with -the appropriate group associations of course. +For a package specification to be included in the Literal +configuration the name attribute from an abstract Package tag (from +Bundler) must match the name attribute of a Package tag in Pkgmgr, +along with the appropriate group associations of course. Each file in the Pkgmgr directory has a priority. This allows the same package to be served by multiple files. The priorities can be diff --git a/doc/server/plugins/generators/rules.txt b/doc/server/plugins/generators/rules.txt index 9ba70238d..64dbc8597 100644 --- a/doc/server/plugins/generators/rules.txt +++ b/doc/server/plugins/generators/rules.txt @@ -20,32 +20,14 @@ The Rules plugin resolves the following Abstract Configuration Entities: to literal configuration entries suitable for the client drivers to consume. -For an entity specification to be included in the Literal configuration -the name attribute from an Abstract Entity Tag (from Base or Bundler) -must match the name attribute of an Entity tag in Rules, along with the -appropriate group associations of course. +For an entity specification to be included in the Literal +configuration the name attribute from an abstract entity tag (from +Bundler) must match the name attribute of an entity tag in Rules, +along with the appropriate group associations of course. Each file in the Rules directory has a priority. This allows the same Entities to be served by multiple files. The priorities can be used to -break ties in the case that multiple files serve data for the same Entity. - - -Usage of Groups in Rules -======================== - -Groups are used by the Rules plugin, along with host metadata, for -selecting the Configuration Entity entries to include in the clients -literal configuration. They can be thought of as:: - - if client is a member of group1 then - assign to literal config - -Nested groups are conjunctive (logical and).:: - - if client is a member of group1 and group2 then - assign to literal config - -Group membership may be negated. +break ties in the case that multiple files serve data for the same entity. Tag Attributes in Rules ======================= @@ -497,8 +479,8 @@ If you wish, you can configure the Rules plugin to support regular expressions. This entails a small performance and memory usage penalty. To do so, add the following setting to ``bcfg2.conf``:: - [rules] - regex = yes + [rules] + regex = yes With regular expressions enabled, you can use a regex in the ``name`` attribute to match multiple abstract configuration entries. diff --git a/doc/server/plugins/generators/semodules.txt b/doc/server/plugins/generators/semodules.txt index 04d72e139..d75160cdf 100644 --- a/doc/server/plugins/generators/semodules.txt +++ b/doc/server/plugins/generators/semodules.txt @@ -41,7 +41,7 @@ SEModules handles ``<SEModule>`` entries. For instance: .. code-block:: xml - <Bundle name="foo"> + <Bundle> <SEModule name="foo.pp"/> </Bundle> @@ -50,7 +50,7 @@ The ``.pp`` extension is optional. .. note:: If you use a ``BoundSEModule`` tag, you must *not* include the - ``.pp`` extension. This is not recommend, though. + ``.pp`` extension. This is not recommended, though. You can also install a disabled module: diff --git a/doc/server/plugins/generators/sshbase.txt b/doc/server/plugins/generators/sshbase.txt index 2b6c8640b..641b9c598 100644 --- a/doc/server/plugins/generators/sshbase.txt +++ b/doc/server/plugins/generators/sshbase.txt @@ -160,6 +160,17 @@ in order to permit :ref:`pulling with bcfg2-admin <server-admin-pull>`. You should almost certainly set ``sensitive`` to "true" in ``info.xml``. +Encryption +========== + +SSHbase can optionally encrypt the private keys that it generates. To +enable this feature, set the ``passphrase`` option in the +``[sshbase]`` section of ``bcfg2.conf`` to the name of the passphrase +that should be used to encrypt all SSH keys. (The passphrases are +enumerated in the ``[encryption]`` section.) See +:ref:`server-encryption` for more details on Bcfg2 encryption in +general. + Blog post ========= diff --git a/doc/server/plugins/generators/sslca.txt b/doc/server/plugins/generators/sslca.txt deleted file mode 100644 index 7ef358a31..000000000 --- a/doc/server/plugins/generators/sslca.txt +++ /dev/null @@ -1,361 +0,0 @@ -.. -*- mode: rst -*- - -.. _server-plugins-generators-sslca: - -===== -SSLCA -===== - -SSLCA is a generator plugin designed to handle creation of SSL private -keys and certificates on request. - -Borrowing ideas from :ref:`server-plugins-generators-cfg-genshi` and -the :ref:`server-plugins-generators-sshbase` plugin, SSLCA automates -the generation of SSL certificates by allowing you to specify key and -certificate definitions. Then, when a client requests a Path that -contains such a definition within the SSLCA repository, the matching -key/cert is generated, and stored in a hostfile in the repo so that -subsequent requests do not result in repeated key/cert recreation. In -the event that a new key or cert is needed, the offending hostfile can -simply be removed from the repository, and the next time that host -checks in, a new file will be created. If that file happens to be the -key, any dependent certificates will also be regenerated. - -.. _getting-started: - -Getting started -=============== - -In order to use SSLCA, you must first have at least one CA configured -on your system. For details on setting up your own OpenSSL based CA, -please see http://www.openssl.org/docs/apps/ca.html for details of the -suggested directory layout and configuration directives. - -For SSLCA to work, the openssl.cnf (or other configuration file) for -that CA must contain full (not relative) paths. - -#. Add SSLCA to the **plugins** line in ``/etc/bcfg2.conf`` and - restart the server -- This enabled the SSLCA plugin on the Bcfg2 - server. - -#. Add a section to your ``/etc/bcfg2.conf`` called ``sslca_foo``, - replacing foo with the name you wish to give your CA so you can - reference it in certificate definitions. - -#. Under that section, add an entry for ``config`` that gives the - location of the openssl configuration file for your CA. - -#. If necessary, add an entry for ``passphrase`` containing the - passphrase for the CA's private key. We store this in - ``/etc/bcfg2.conf`` as the permissions on that file should have it - only readable by the bcfg2 user. If no passphrase is entry exists, - it is assumed that the private key is stored unencrypted. - -#. Optionally, Add an entry ``chaincert`` that points to the location - of your ssl chaining certificate. This is used when preexisting - certifcate hostfiles are found, so that they can be validated and - only regenerated if they no longer meet the specification. If - you're using a self signing CA this would be the CA cert that you - generated. If the chain cert is a root CA cert (e.g., if it is a - self-signing CA), also add an entry ``root_ca = true``. If - ``chaincert`` is omitted, certificate verification will not be - performed. - -#. Once all this is done, you should have a section in your - ``/etc/bcfg2.conf`` that looks similar to the following:: - - [sslca_default] - config = /etc/pki/CA/openssl.cnf - passphrase = youReallyThinkIdShareThis? - chaincert = /etc/pki/CA/chaincert.crt - root_ca = true - -#. You are now ready to create key and certificate definitions. For - this example we'll assume you've added Path entries for the key, - ``/etc/pki/tls/private/localhost.key``, and the certificate, - ``/etc/pki/tls/certs/localhost.crt`` to a bundle or base. - -#. Defining a key or certificate is similar to defining a Cfg file. - Under your Bcfg2's ``SSLCA/`` directory, create the directory - structure to match the path to your key. In this case this would be - something like - ``/var/lib/bcfg2/SSLCA/etc/pki/tls/private/localhost.key``. - -#. Within that directory, create a `key.xml`_ file containing the - following: - - .. code-block:: xml - - <KeyInfo> - <Key type="rsa" bits="2048" /> - </KeyInfo> - -#. This will cause the generation of an 2048 bit RSA key when a client - requests that Path. Alternatively you can specify ``dsa`` as the - keytype, or a different number of bits. - -#. Similarly, create the matching directory structure for the - certificate path, and a `cert.xml`_ containing the following: - - .. code-block:: xml - - <CertInfo> - <Cert format="pem" key="/etc/pki/tls/private/localhost.key" - ca="default" days="365" c="US" l="New York" st="New York" - o="Your Company Name" /> - </CertInfo> - -#. When a client requests the cert path, a certificate will be - generated using the key hostfile at the specified key location, - using the CA matching the ca attribute. ie. ca="default" will match - [sslca_default] in your ``/etc/bcfg2.conf`` - -.. _sslca-configuration: - -Configuration -============= - -bcfg2.conf ----------- - -``bcfg2.conf`` contains miscellaneous configuration options for the -SSLCA plugin. These are described in some detail above in -`getting-started`, but are also enumerated here as a reference. Any -booleans in the config file accept the values "1", "yes", "true", and -"on" for True, and "0", "no", "false", and "off" for False. - -Each directive below should appear at most once in each -``[sslca_<name>]`` section. The following directives are understood: - -+--------------+------------------------------------------+---------+---------+ -| Name | Description | Values | Default | -+==============+==========================================+=========+=========+ -| config | Path to the openssl config for the CA | String | None | -+--------------+------------------------------------------+---------+---------+ -| passphrase | Passphrase for the CA private key | String | None | -+--------------+------------------------------------------+---------+---------+ -| chaincert | Path to the SSL chaining certificate for | String | None | -| | verification | | | -+--------------+------------------------------------------+---------+---------+ -| root_ca | Whether or not ``<chaincert>`` is a root | Boolean | false | -| | CA (as opposed to an intermediate cert) | | | -+--------------+------------------------------------------+---------+---------+ - -Only ``config`` is required. - -cert.xml --------- - -.. xml:schema:: sslca-cert.xsd - :linktotype: - :inlinetypes: CertType - -Example -^^^^^^^ - -.. code-block:: xml - - <CertInfo> - <subjectAltName>test.example.com</subjectAltName> - <Group name="apache"> - <Cert key="/etc/pki/tls/private/foo.key" days="730"/> - </Group> - <Group name="nginx"> - <Cert key="/etc/pki/tls/private/foo.key" days="730" - append_chain="true"/> - </Group> - </CertInfo> - -key.xml -------- - -.. xml:schema:: sslca-key.xsd - :linktotype: - :inlinetypes: KeyType - -Example -^^^^^^^ - -.. code-block:: xml - - <KeyInfo> - <Group name="fast"> - <Key type="rsa" bits="1024"/> - </Group> - <Group name="secure"> - <Key type="rsa" bits="4096"/> - </Group> - </KeyInfo> - -Automated Bcfg2 SSL Authentication -================================== - -This section describes one possible scenario for automating ssl -certificate generation and distribution for bcfg2 client/server -communication using SSLCA. The process involves configuring a -certificate authority (CA), generating the CA cert and key pair, -configuring the bcfg2 SSLCA plugin and a Bundle to use the SSLCA -generated certs to authenticate the bcfg2 client and server. - -OpenSSL CA ----------- - -If you already have a SSL CA available you can skip this section, -otherwise you can easily build one on the server using openssl. The -paths should be adjusted to suite your preferences. - -#. Prepare the directories and files:: - - mkdir -p /etc/pki/CA/newcerts - mkdir /etc/pki/CA/crl - echo '01' > /etc/pki/CA/serial - touch /etc/pki/CA/index.txt - touch /etc/pki/CA/crlnumber - -#. Edit the ``openssl.cnf`` config file, and in the **[ CA_default ]** - section adjust the following parameters:: - - dir = /etc/pki # Where everything is kept - certs = /etc/pki/CA/certs # Where the issued certs are kept - database = /etc/pki/CA/index.txt # database index file. - new_certs_dir = /etc/pki/CA/newcerts # default place for new certs. - certificate = /etc/pki/CA/certs/bcfg2ca.crt # The CA certificate - serial = /etc/pki/CA/serial # The current serial number - crl_dir = /etc/pki/CA/crl # Where the issued crl are kept - crlnumber = /etc/pki/CA/crlnumber # the current crl number - crl = /etc/pki/CA/crl.pem # The current CRL - private_key = /etc/pki/CA/private/bcfg2ca.key # The private key - -#. Create the CA root certificate and key pair. You'll be asked to - supply a passphrase, and some organizational info. The most - important bit is **Common Name** which you should set to be the - hostname of your bcfg2 server that your clients will see when doing - a reverse DNS query on it's ip address.:: - - openssl req -new -x509 -extensions v3_ca -keyout bcfg2ca.key \ - -out bcfg2ca.crt -days 3650 - -#. Move the generated cert and key to the locations specified in - ``openssl.cnf``:: - - mv bcfg2ca.key /etc/pki/CA/private/ - mv bcfg2ca.crt /etc/pki/CA/certs/ - -Your self-signing CA is now ready to use. - -Bcfg2 ------ - -SSLCA -^^^^^ - -The SSLCA plugin was not designed specifically to manage bcfg2 -client/server communication though it is certainly able to provide -certificate generation and management services for that -purpose. You'll need to configure the **SSLCA** plugin to serve the -key, and certificate paths that we will define later in our client's -``bcfg2.conf`` file. - -The rest of these instructions will assume that you've configured the -**SSLCA** plugin as described above and that the files -``SSLCA/etc/pki/tls/certs/bcfg2client.crt/cert.xml`` and -``SSLCA/etc/pki/tls/private/bcfg2client.key/key.xml`` represent the -cert and key paths you want generated for SSL auth. - -Client Bundle -^^^^^^^^^^^^^ - -To automate the process of generating and distributing certs to the -clients we need define at least the Cert and Key paths served by the -SSLCA plugin, as well as the ca certificate path in a Bundle. For -example: - -.. code-block:: xml - - <Path name='/etc/pki/tls/certs/bcfg2ca.crt'/> - <Path name='/etc/pki/tls/bcfg2client.crt'/> - <Path name='/etc/pki/tls/private/bcfg2client.key'/> - -Here's a more complete example bcfg2-client bundle: - -.. code-block:: xml - - <Bundle name='bcfg2-client'> - <Path name='/etc/bcfg2.conf'/> - <Path name='/etc/cron.d/bcfg2-client'/> - <Package name='bcfg2'/> - <Service name='bcfg2'/> - <Group name='rpm'> - <Path name='/etc/sysconfig/bcfg2'/> - <Path name='/etc/pki/tls/certs/bcfg2ca.crt'/> - <Path name='/etc/pki/tls/certs/bcfg2client.crt'/> - <Path name='/etc/pki/tls/private/bcfg2client.key'/> - </Group> - <Group name='deb'> - <Path name='/etc/default/bcfg2' altsrc='/etc/sysconfig/bcfg2'/> - <Path name='/etc/ssl/certs/bcfg2ca.crt' altsrc='/etc/pki/tls/certs/bcfg2ca.crt'/> - <Path name='/etc/ssl/certs/bcfg2client.crt' altsrc='/etc/pki/tls/certs/bcfg2client.crt'/> - <Path name='/etc/ssl/private/bcfg2client.key' altsrc='/etc/pki/tls/private/bcfg2client.key'/> - </Group> - </Bundle> - -In the above example we told Bcfg2 that it also needs to serve -``/etc/bcfg2.conf``. This is optional but convenient. - -The ``bcfg2.conf`` client config needs at least 5 parameters set for -SSL auth. - -#. ``key`` : This is the host specific key that SSLCA will generate. -#. ``certificate`` : This is the host specific cert that SSLCA will - generate. -#. ``ca`` : This is a copy of your CA certificate. Not generated by - SSLCA. -#. ``user`` : Usually set to fqdn of client. This *shouldn't* be - required but is as of 1.3.0. See: - http://trac.mcs.anl.gov/projects/bcfg2/ticket/1019 -#. ``password`` : Set to arbitrary string when using certificate - auth. This also *shouldn't* be required. See: - http://trac.mcs.anl.gov/projects/bcfg2/ticket/1019 - -Here's what a functional **[communication]** section in a -``bcfg2.conf`` genshi template for clients might look like.:: - - [communication] - protocol = xmlrpc/ssl - {% if metadata.uuid != None %}\ - user = ${metadata.uuid} - {% end %}\ - password = DUMMYPASSWORDFORCERTAUTH - {% choose %}\ - {% when 'rpm' in metadata.groups %}\ - certificate = /etc/pki/tls/certs/bcfg2client.crt - key = /etc/pki/tls/private/bcfg2client.key - ca = /etc/pki/tls/certs/bcfg2ca.crt - {% end %}\ - {% when 'deb' in metadata.groups %}\ - certificate = /etc/ssl/certs/bcfg2client.crt - key = /etc/ssl/private/bcfg2client.key - ca = /etc/ssl/certs/bcfg2ca.crt - {% end %}\ - {% end %}\ - -As a client will not be able to authenticate with certificates it does -not yet posses we need to overcome the chicken and egg scenario the -first time we try to connect such a client to the server. We can do so -using password based auth to boot strap the client manually specifying -all the relevant auth parameters like so:: - - bcfg2 -qv -S https://fqdn.of.bcfg2-server:6789 -u fqdn.of.client \ - -x SUPER_SECRET_PASSWORD - -If all goes well the client should recieve a freshly generated key and -cert and you should be able to run ``bcfg2`` again without specifying -the connection parameters. - -If you do run into problems you may want to review -:ref:`appendix-guides-authentication`. - -TODO -==== - -#. Add generation of pkcs12 format certs diff --git a/doc/server/plugins/generators/tcheetah.txt b/doc/server/plugins/generators/tcheetah.txt deleted file mode 100644 index c79a8ced5..000000000 --- a/doc/server/plugins/generators/tcheetah.txt +++ /dev/null @@ -1,197 +0,0 @@ -.. -*- mode: rst -*- - -.. _server-plugins-generators-tcheetah: - -======== -TCheetah -======== - -.. warning:: - - TCheetah is deprecated. You should instead use - :ref:`server-plugins-generators-cfg-cheetah` in the Cfg plugin. - -This document reflects the ``TCheetah`` plugin. - -The ``TCheetah`` plugin allows you to use the `cheetah templating system -<http://www.cheetahtemplate.org/>`_ to create files, instead of the -various diff-based methods offered by the ``Cfg`` plugin. It also allows -you to include the results of probes executed on the client in the -created files. - -To begin, you will need to download and install the Cheetah templating -engine from http://www.cheetahtemplate.org/. Once it is installed, -you can enable it by adding ``TCheetah`` to the ``plugins`` line in -``/etc/bcfg2.conf`` on your Bcfg server. For example:: - - plugins = Base,Bundler,Cfg,...,TCheetah - -The ``TCheetah`` plugin makes use of a ``Cfg``-like directory structure -located in in a ``TCheetah`` subdirectory of your repository, usually -``/var/lib/bcfg2/TCheetah``. Each file has a directory containing two -files, ``template`` and ``info``. The template is a standard Cheetah -template with two additions: - -* `self.metadata` is the client's :ref:`metadata <server-plugins-grouping-metadata-clientmetadata>` -* `self.metadata.Properties.xdata` is an xml document of unstructured data - -The ``info`` file is formatted like ``:info`` files from Cfg. - -Mostly, people will want to use client metadata. - -File permissions -================ - -File permissions for entries handled by TCheetah are controlled via the -use of :ref:`server-info` files. Note that you **cannot** use both a -Permissions entry and a Path entry to handle the same file. - -self.metadata variables -======================= - -self.metadata is an instance of the class ClientMetadata and documented -:ref:`here <server-plugins-grouping-metadata-clientmetadata>`. - -self.metadata.Properties.xdata -============================== - -.. note:: - - If you want to use Properties, you will need to enable the - :ref:`server-plugins-connectors-properties` plugin in - ``/etc/bcfg2.conf``. - -Properties.xdata is a python `ElementTree <http://codespeak.net/lxml/>`_ -object, loaded from the data in ``/var/lib/bcfg2/Properties/<properties -file>.xml``. That file should have a ``Properties`` node at its root. - -Example ``Properties/example.xml``: - -.. code-block:: xml - - <Properties> - <host> - <www.example.com> - <rootdev>/dev/sda</rootdev> - </www.example.com> - </host> - </Properties> - -You may use any of the ElementTree methods to access data in your -template. Several examples follow, each producing an identical result -on the host 'www.example.com':: - - $self.metadata.Properties['example.xml'].xdata.find('host').find('www.example.com').find('rootdev').text - $self.metadata.Properties['example.xml'].xdata.find('host').find($self.metadata.hostname).find('rootdev').text - ${self.metadata.Properties['example.xml'].xdata.xpath('host/www.example.com/rootdev')[0].text} - ${self.metadata.Properties['example.xml'].xdata.xpath('host/' + self.metadata.hostname + '/rootdev')[0].text} - #set $path = 'host/' + $self.metadata.hostname + '/rootdev' - ${self.metadata.Properties['example.xml'].xdata.xpath($path)[0].text} - ${self.metadata.Properties['example.xml'].xdata.xpath(path)[0].text} - -Other Variables -=============== - -* **Template.searchList(self)[1]['path']** is the Path name specified in a Bundle -* **Template.searchList(self)[1]['source_path']** is the path to the TCheetah template on the Bcfg2 server - -Simple Example -============== - -TCheetah works similar to Cfg in that you define all literal information -about a particular file in a directory rooted at TCheetah/path_to_file. -The actual file contents are placed in a file named `template` in that -directory. Below is a simple example a file ``/foo``. - -``/var/lib/bcfg2/TCheetah/foo/template`` - -.. code-block:: none - - > buildfile /foo <clientname> - Hostname is $self.metadata.hostname - Filename is $Template.searchList(self)[1]['path'] - Template is $Template.searchList(self)[1]['source_path'] - Groups: - #for $group in $self.metadata.groups: - * $group - #end for - Categories: - #for $category in $self.metadata.categories: - * $category -- $self.metadata.categories[$category] - #end for - - Probes: - #for $probe in $self.metadata.Probes: - * $probe -- $self.metadata.Probes[$probe] - #end for - -``/var/lib/bcfg2/TCheetah/foo/info`` - -.. code-block:: none - - mode: 624 - -Output ------- - -The following output can be generated with bcfg2-info. Note that probe -information is not persistent, hence, it only works when clients directly -query the server. For this reason, bcfg2-info output doesn't reflect -current client probe state. - -.. code-block:: xml - - <Path type="file" name="/foo" owner="root" mode="0624" group="root"> - Hostname is topaz.mcs.anl.gov - Filename is /foo - Template is /var/lib/bcfg2/TCheetah/foo/template - Groups: - * desktop - * mcs-base - * ypbound - * workstation - * xserver - * debian-sarge - * debian - * a - Categories: - * test -- a - - Probes: - </Path> - -Example: Replace the crontab plugin -=================================== - -In many cases you can use the TCheetah plugin to avoid writing custom -plugins in Python. This example randomizes the time of cron.daily -execution with a stable result. Cron.daily is run at a consistent, -randomized time between midnight and 7am.:: - - #import random - #silent random.seed($self.metadata.hostname) - - # /etc/crontab: system-wide crontab - # Unlike any other crontab you don't have to run the `crontab` - # command to install the new version when you edit this file. - # This file also has a username field, that none of the other crontabs do. - - SHELL=/bin/sh - PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin://bin - - # m h dom mon dow user command - 17 * * * * root run-parts --report /etc/cron.hourly - $random.randrange(0,59) $random.randrange(0,6) * * * root test -x /usr/sbin/anacron || run-parts --report /etc/cron.daily - 47 6 * * 7 root test -x /usr/sbin/anacron || run-parts --report /etc/cron.weekly - 52 6 1 * * root test -x /usr/sbin/anacron || run-parts --report /etc/cron.monthly. - -.. note:: Comments and Cheetah - As Cheetah processes your templates it will consider hash "#" style - comments to be actual comments in the template and will strip them - from the final config file. If you would like to preserve the comment - in the final config file you need to escape the hash character '\#' - which will tell Cheetah (and Python) that you do in fact want the - comment to appear in the final config file.:: - - # This is a comment in my template which will be stripped when it's processed through Cheetah - \# This comment will appear in the generated config file. diff --git a/doc/server/plugins/generators/tgenshi.txt b/doc/server/plugins/generators/tgenshi.txt deleted file mode 100644 index 43a02f253..000000000 --- a/doc/server/plugins/generators/tgenshi.txt +++ /dev/null @@ -1,213 +0,0 @@ -.. -*- mode: rst -*- - -.. _server-plugins-generators-tgenshi-index: - -======= -TGenshi -======= - -.. warning:: - - The TGenshi plugin is deprecated. You should instead use - :ref:`server-plugins-generators-cfg-genshi` in the Cfg plugin. - -This page documents the TGenshi plugin. This plugin works with version -0.4 and newer of the genshi library. - -The TGenshi plugin allows you to use the `Genshi -<http://genshi.edgewall.org>`_ templating system to create files, -instead of the various diff-based methods offered by the Cfg -plugin. It also allows you to include the results of probes executed -on the client in the created files. - -To begin, you will need to download and install the Genshi templating engine. - -To install on CentOS or RHEL, run:: - - sudo yum install python-genshi - -Once it is installed, you can enable it by adding ``TGenshi`` to the -generators line in ``/etc/bcfg2.conf`` on your Bcfg server. For example:: - - plugins = Base,Bundler,Cfg,...,TGenshi - -The TGenshi plugin makes use of a Cfg-like directory structure -located in in a TGenshi subdirectory of your repository, usually -``/var/lib/bcfg2/TGenshi``. Each file has a directory containing two file -types, template and info. Templates are named according to the genshi -format used; template.txt uses the genshi text format, and template.xml -uses the XML format. - -If used with Genshi 0.5 or later the plugin also supports the `new -style -<http://genshi.edgewall.org/wiki/Documentation/0.5.x/text-templates.html>`_ -text template format for files named template.newtxt. One of the -advantages of the new format is that it does not use # as a command -delimiter, making it easier to utilize for configuration files that -use # as a comment character. - -Only one template format may be used per file served. Info files are -identical to those used in ``Cfg``, and ``info.xml`` files are -supported. - -Inside of templates -=================== - -* **metadata** is the client's :ref:`metadata - <server-plugins-grouping-metadata-clientmetadata>` -* **metadata.Properties** is an xml document of unstructured data (only - available when used in conjunction with the - :ref:`server-plugins-connectors-properties` plugin) -* **name** is the path name specified in bcfg -* **path** is the path to the TGenshi template. It starts with a - leading slash, and is relative to the Bcfg2 specification root. - E.g., ``/Cfg/etc/foo.conf/foo.conf.genshi`` or - ``/TGenshi/etc/foo.conf/template.newtxt.H_foo.example.com`` - -See the genshi `documentation -<http://genshi.edgewall.org/wiki/Documentation>`_ for examples of -Genshi syntax. - -Examples: Old Genshi Syntax ---------------------------- - -Genshi's web pages recommend against using this syntax, as it may -disappear from future releases. - -Group Negation -^^^^^^^^^^^^^^ - -Templates are also useful for cases where more sophisticated boolean -operations than those supported by Cfg are needed. For example, the -template:: - - #if "ypbound" in metadata.groups and "workstation" in metadata.groups - client is ypbound workstation - #end - #if "ubuntu" not in metadata.groups and "desktop" in metadata.groups - client is a desktop, but not an ubuntu desktop - #end - -Produces: - -.. code-block:: xml - - <Path type="file" name="/bar.conf" owner="root" mode="0644" group="root">client is ypbound workstation - client is a desktop, but not an ubuntu desktop - </Path> - -This flexibility provides the ability to build much more compact and -succinct definitions of configuration contents than Cfg can. - -Troubleshooting -=============== - -When developing a template, you can see what the template would -generate on a client with :ref:`bcfg2-info <server-bcfg2-info>`:: - - bcfg2-info buildfile <path> <hostname> - -E.g.:: - - bcfg2-info buildfile /etc/foo.conf foo.example.com - -To generate a file with an altsrc attribute, you can run:: - - bcfg2-info buildfile /etc/foo/foo.conf --altsrc=/etc/foo.conf \ - foo.example.com - -Sometimes, it's useful to be able to do more in-depth troubleshooting -by running the template manually. To do this, run ``bcfg2-info -debug``, and, once in the Python interpreter, run:: - - metadata = self.build_metadata("<hostname>") - path = "<relative path to template (see note below)>" - -``path`` should be set to the path to the template file with a leading -slash, relative to the Bcfg2 specification root. See `Inside of -Templates`_ for examples. - -Then, run:: - - import os, Bcfg2.Options - from genshi.template import TemplateLoader, NewTextTemplate - name = os.path.dirname(path[path.find('/', 1):]) - setup = Bcfg2.Options.OptionParser({'repo': - Bcfg2.Options.SERVER_REPOSITORY}) - setup.parse('--') - template = TemplateLoader().load(setup['repo'] + path, cls=NewTextTemplate) - print template.generate(metadata=metadata, path=path, name=name).render() - -This gives you more fine-grained control over how your template is -rendered. - -You can also use this approach to render templates that depend on -:ref:`altsrc <server-plugins-structures-altsrc>` tags by setting -``path`` to the path to the template, and setting ``name`` to the path -to the file to be generated, e.g.:: - - metadata = self.build_metadata("foo.example.com") - path = "/Cfg/etc/sysconfig/network-scripts/ifcfg-template/ifcfg-template.genshi" - name = "/etc/sysconfig/network-scripts/ifcfg-bond0" - -File permissions -================ - -File permissions for entries handled by TGenshi are controlled via the -use of :ref:`server-info` files. Note that you **cannot** use both a -Permissions entry and a Path entry to handle the same file. - -Error handling -================ - -Situations may arise where a templated file cannot be generated due to -missing or incomplete information. A TemplateError can be raised to -force a bind failure and prevent sending an incomplete file to the -client. For example, this template:: - - {% python - from genshi.template import TemplateError - grp = None - for g in metadata.groups: - if g.startswith('ganglia-gmond-'): - grp = g - break - else: - raise TemplateError, "Missing group" - %}\ - -will fail to bind if the client is not a member of a group starting with -"ganglia-gmond-". The syslogs on the server will contain this message:: - - bcfg2-server[5957]: Genshi template error: Missing group - bcfg2-server[5957]: Failed to bind entry: Path /etc/ganglia/gmond.conf - -indicating the bind failure and message raised with the TemplateError. - -FAQs -==== - -**Question** - -How do I escape the $ (dollar sign) in a TGenshi text template? For -example, if I want to include SVN (subversion) keywords like $Id$ or -$HeadURL$ in TGenshi-generated files, or am templating a bourne shell -(sh/bash) script or Makefile (make). - -**Answer** - -Use $$ (double dollar sign) to output a literal $ (dollarsign) -in a TGenshi text template. So instead of $Id$, you'd use -$$Id$$. See also Genshi tickets `#282: Document $$ escape -convention <http://genshi.edgewall.org/ticket/282>`_ and -`#283: Allow for redefinition of template syntax per-file -<http://genshi.edgewall.org/ticket/283>`_. - -Examples -======== - -.. toctree:: - :glob: - :maxdepth: 1 - - examples/genshi/* diff --git a/doc/server/plugins/grouping/metadata.txt b/doc/server/plugins/grouping/metadata.txt index ceac5dc24..832b1a13f 100644 --- a/doc/server/plugins/grouping/metadata.txt +++ b/doc/server/plugins/grouping/metadata.txt @@ -90,6 +90,8 @@ Database Settings <server-database>`. The `clients.xml`_-based model remains the default. +.. _server-plugins-grouping-metadata-groups-xml: + groups.xml ========== @@ -180,76 +182,6 @@ groups: .. xml:schema:: metadata.xsd - -XInclude -======== - -.. versionadded:: 0.9.0 - -`XInclude <http://www.w3.org/TR/xinclude/>`_ is a W3C specification -for the inclusion of external XML documents into XML source files, -allowing complex definitions to be split into smaller, more manageable -pieces. The `Metadata`_ plugin supports the use of XInclude -specifications to split the `clients.xml`_ and `groups.xml`_ -files. This mechanism allows the following specification to produce -useful results: - -.. code-block:: xml - - <Groups xmlns:xi="http://www.w3.org/2001/XInclude"> - <xi:include href="my-groups.xml" /> - <xi:include href="their-groups.xml" /> - </Groups> - -Each of the included groups files has the same format. These files are -properly validated by ``bcfg2-lint``. This mechanism is useful for -composing group definitions from multiple sources, or setting -different permissions in an svn repository. - -You can also optionally include a file that may or may not exist with -the ``fallback`` tag: - -.. code-block:: xml - - <Groups xmlns:xi="http://www.w3.org/2001/XInclude"> - <xi:include href="my-groups.xml"/> - <xi:include href="their-groups.xml"><xi:fallback/></xi:include> - </Groups> - -In this case, if ``their-groups.xml`` does not exist, no error will be -raised and everything will work fine. (You can also use ``fallback`` -to include a different file, or explicit content in the case that the -parent include does not exist.) - -Wildcard XInclude ------------------ - -.. versionadded:: 1.3.1 - -Bcfg2 supports an extension to XInclude that allows you to use shell -globbing in the hrefs. (Stock XInclude doesn't support this, since -the href is supposed to be a URL.) - -For instance: - -.. code-block:: xml - - <Groups xmlns:xi="http://www.w3.org/2001/XInclude"> - <xi:include href="groups/*.xml"/> - </Groups> - -This would include all ``*.xml`` files in the ``groups`` subdirectory. - -Note that if a glob finds no files, that is treated the same as if a -single included file does not exist. You should use the ``fallback`` -tag, described above, if a glob may potentially find no files. - -Probes -====== - -The metadata plugin includes client-side probing functionality. This -is fully documented :ref:`here <server-plugins-probes-index>`. - Metadata Caching ================ diff --git a/doc/server/plugins/misc/acl.txt b/doc/server/plugins/misc/acl.txt new file mode 100644 index 000000000..226b56a44 --- /dev/null +++ b/doc/server/plugins/misc/acl.txt @@ -0,0 +1,235 @@ +.. -*- mode: rst -*- + +.. _server-plugins-misc-acl: + +=== +ACL +=== + +The ACL plugin lets you set client communication ACLs to prevent +clients from accessing the full range of exposed XML-RPC methods. + +You can get a list of all exposed methods by running:: + + bcfg2-admin xcmd listMethods + +Note that this will only list methods that are available to the client +this is run from; that is, if the ACL plugin is in place, +``listMethods`` will reflect the ACLs. + +ACLs can be set in two different ways: + +* IP-based ACLs allow you to set ACLs based on client IP address or + CIDR range. +* Metadata-based ACLs allow you to set ACLs based on client hostname, + group membership, or complex combinations thereof. + +IP-based ACLs are much faster, but metadata-based ACLs are often +easier and better. + +If you are not going to use any ACLs, it is recommended that you +disable this plugin because using it can incur a slight performance +hit. If you are using IP-based ACLs but *not* metadata-based ACLs, it +is similarly recommended that you ensure that your IP-based ACL file +ends with an explicit Deny for all clients; this will ensure that +metadata-based ACLs are never checked. If you are using +metadata-based ACLs, :ref:`server-caching` can alleviate most of the +performance penalty. + +Enabling the ACL plugin +======================= + +First, create ``/var/lib/bcfg2/ACL/``. Then, add ``ACL`` to your +``plugins`` list in ``bcfg2.conf``:: + + plugins = Bundler, Cfg, ..., Packages, ACL + +Finally, create ``/var/lib/bcfg2/ACL/ip.xml`` (for `IP-based ACLs`_), +``/var/lib/bcfg2/ACL/metadata.xml`` (for `Metadata-based ACLs`_), or +both. + +IP-based ACLs +============= + +IP-based ACLs allow you to set ACLs based on client IP address or CIDR +range. IP-based ACLs are very fast. If you are using IP-based ACLs +but *not* metadata-based ACLs, it is recommended that you ensure that +your IP-based ACL file ends with an explicit Deny for all clients; +this will ensure that metadata-based ACLs are never checked. + +IP-based ACLs are defined in ``ACL/ip.xml``. The file is parsed +sequentially; the first matching rule applies. Each rule is either +Allow (to allow the client access), Deny (to deny the client access), +or Defer (to defer to `Metadata-based ACLs`_). The last rule in +``ip.xml`` is an implicit default allow for 127.0.0.1, and an implicit +default defer for all other machines. + +If no ``ip.xml`` file exists, then ACL checking will be deferred to +metadata-based ACLs. + +Example +------- + +.. code-block:: xml + + <ACL> + <Allow address="192.168.1.10" method="*.*"/> + <Deny address="192.168.2.0" netmask="255.255.255.0" + method="AssertProfile"/> + <Allow address="192.168.1.12" method="Git.Update"/> + <Allow method="*"/> + </ACL> + +In this example: + +* The machine at 192.168.1.10 (perhaps the Bcfg2 server) can call all + plugin XML-RPC methods; +* Machines in the 192.168.2.0/24 network cannot assert their own + profiles; +* The machine at 192.168.1.12 (perhaps the Git server) can call the + Git.Update method; +* All machines can call core methods (except 192.168.2.0/24, which can + call all core methods except AssertProfile). + +Implicitly, all machines (except localhost) except 192.168.1.10 are +disallowed access to the plugin methods. + +You can also provide a minimal configuration to try to weed out some +obvious bad requests before doing the more expensive `Metadata-based +ACLs`_. For instance: + +.. code-block:: xml + + <ACL> + <Allow method="*"/> + <Defer address="192.168.1.0" netmask="24" method="*.*"/> + <Deny method="*.*"/> + </ACL> + +In this example: + +* All machines can call all core methods without checking metadata + ACLs; +* Plugin method calls from machines in 192.168.1.0/24 are deferred to + metadata ACLs; and +* All other plugin method calls are denied. + +The only time metadata ACLs would be checked in this example would be +plugin method calls by machines in 192.168.1.0/24. + +Reference +--------- + +.. xml:type: IPACLContainerType + +Metadata-based ACLs +=================== + +Metadata-based ACLs let you set ACLs based on client hostname or group +membership, which is much more flexible and maintainable than +`IP-based ACLs`_. The downside is that it is slower, because it +requires generating client metadata for each machine that tries to +authenticate. Without :ref:`server-caching`, using metadata-based +ACLs will double the number of client metadata builds per client run, +which could be a sizeable performance penalty. + +In order to limit the performance penalty, it's highly recommended +to: + +* Enable :ref:`server-caching` in ``cautious`` or ``aggressive`` mode; + and +* Deny as many clients as possible with `IP-based ACLs`_. + +Metadata-based ACLs are defined in ``ACL/metadata.xml``. Only Allow +and Deny rules are supported, not Defer rules. The file is parsed +sequentially; the first matching rule applies. The last rule in +``metadata.xml`` is an implicit default allow for machines called +``localhost`` or ``localhost.localdomain``, and an implicit default +deny for all other machines. + +If no ``metadata.xml`` file exists, then all requests are implicitly +allowed. + +Example +------- + +This example is functionally identical to the `IP-based ACLs` example +above, but more maintainable in several ways: + +.. code-block:: xml + + <ACL> + <Group name="bcfg2-server"> + <Allow method="*.*"/> + </Group> + <Group name="user-workstations"> + <Deny method="AssertProfile"/> + </Group> + <Group name="git-server"> + <Allow method="Git.Update"/> + </Group> + <Allow method="*"/> + </ACL> + +In this case, if you add a Bcfg2 server or Git server, or one of those +servers changes IP address, you don't need to rewrite your ACLs. +Similarly, you could add a new subnet of user workstations. + +Reference +--------- + +.. xml:type: MetadataACLContainerType + +.. _server-plugins-misc-acl-wildcards: + +Wildcards +========= + +The ACL descriptions allow you to use '*' as a wildcard for any number +of characters *other than* ``.``. That is: + +* ``*`` would match ``DeclareVersion`` and ``GetProbes``, but would + *not* match ``Git.Update``. +* ``*.*`` would match ``Git.Update``, but not ``DeclareVersion`` or + ``GetProbes``. + +Since all plugin methods are scoped to their plugin (i.e., they are +all ``<plugin name>.<method name>``), and all core methods have no +scope, this lets you easily allow or deny core or plugin methods. You +could also do something like ``*.toggle_debug`` to allow a host to +enable or disable debugging for all plugins. + +No other bash globbing is supported. + +Examples +======== + +The :ref:`default ACL list <server-access-control>` can be described +in ``ip.xml`` fairly simply: + +.. code-block:: xml + + <ACL> + <Allow address="127.0.0.1" method="*.*"/> + <Allow address="127.0.0.1" method="*"/> + <Deny method="*.*"/> + <Deny method="*_debug"/> + <Deny method="get_statistics"/> + <Allow method="*"/> + </ACL> + +A basic configuration that is still very secure but perhaps more +functional could be given in ``metadata.xml``: + +.. code-block:: xml + + <ACL> + <Group name="bcfg2-server"> + <Allow method="*.*"/> + <Allow method="*"/> + </Group> + <Deny method="*.*"/> + <Deny method="*_debug"/> + <Deny method="get_statistics"/> + <Allow method="*"/> + </ACL> diff --git a/doc/server/plugins/probes/fileprobes.txt b/doc/server/plugins/probes/fileprobes.txt index 0baec2c59..1bee38c5a 100644 --- a/doc/server/plugins/probes/fileprobes.txt +++ b/doc/server/plugins/probes/fileprobes.txt @@ -1,3 +1,5 @@ +.. -*- mode: rst -*- + .. _server-plugins-probes-fileprobes: ========== diff --git a/doc/server/plugins/statistics/reporting.txt b/doc/server/plugins/statistics/reporting.txt index c3c51cd29..74ea61e62 100644 --- a/doc/server/plugins/statistics/reporting.txt +++ b/doc/server/plugins/statistics/reporting.txt @@ -9,7 +9,7 @@ Reporting Reporting can be enabled by adding Reporting to the plugins line in ``/etc/bcfg2.conf``: - plugins = Base,Bundler,Cfg,...,Reporting + plugins = Bundler,Cfg,...,Reporting For more information on how to use Reporting to setup reporting, see :ref:`reports-dynamic`. diff --git a/doc/server/plugins/statistics/statistics.txt b/doc/server/plugins/statistics/statistics.txt deleted file mode 100644 index d16f5a828..000000000 --- a/doc/server/plugins/statistics/statistics.txt +++ /dev/null @@ -1,7 +0,0 @@ -.. -*- mode: rst -*- - -.. _server-plugins-statistics-statistics: - -========== -Statistics -========== diff --git a/doc/server/plugins/structures/altsrc.txt b/doc/server/plugins/structures/altsrc.txt index 1268a8584..f3911e33e 100644 --- a/doc/server/plugins/structures/altsrc.txt +++ b/doc/server/plugins/structures/altsrc.txt @@ -11,7 +11,7 @@ altsrc Altsrc is a generic, Bcfg2 server-side mechanism for performing configuration entry name remapping for the purpose of data binding. Altsrc can be used as a parameter for any entry type, and can be used -in any structure, including Bundler and Base. +in any structure. Use Cases ========= @@ -36,7 +36,7 @@ Examples .. code-block:: xml - <Bundle name='netinfo'> + <Bundle> <Group name='solaris'> <Path name='/etc/inet/hosts' altsrc='/etc/hosts'/> </Group> @@ -58,7 +58,7 @@ Examples .. code-block:: xml - <Bundle name='openssl'> + <Bundle> <Package name='openssl' altsrc='openssl-encap'/> <Package name='openssl' altsrc='openssl-rpm'/> </Bundle> @@ -76,7 +76,7 @@ Examples .. code-block:: xml - <Bundle name='firewall'> + <Bundle> ... <Group name='conduit'> <Path name='/etc/firewall-rules' altsrc='/etc/firewall-rules-external'/> @@ -97,7 +97,7 @@ Examples .. code-block:: xml - <Bundle name='netconfig'> + <Bundle> <Path name='/etc/sysconfig/network-scripts/ifcfg-eth0' altsrc='/etc/ifcfg-template'/> <Path name='/etc/sysconfig/network-scripts/ifcfg-eth1' altsrc='/etc/ifcfg-template'/> <Path name='/etc/sysconfig/network-scripts/ifcfg-eth2' altsrc='/etc/ifcfg-template'/> diff --git a/doc/server/plugins/structures/base.txt b/doc/server/plugins/structures/base.txt deleted file mode 100644 index 03eae0573..000000000 --- a/doc/server/plugins/structures/base.txt +++ /dev/null @@ -1,83 +0,0 @@ -.. -*- mode: rst -*- - -.. _server-plugins-structures-base: - -==== -Base -==== - -.. deprecated:: 1.2.0 - -.. warning:: - - The Base plugin no longer receives new features/functionality. - Please use :ref:`server-plugins-structures-bundler-index` instead. - -The Base plugin is a structure plugin that provides the ability to add -lists of unrelated entries into client configuration entry inventories. - -Base works much like Bundler in its file format. The main difference -between Base and Bundler is that Base files are included in all clients' -configuration whereas bundles must be included explicitly in your -Metadata. See the :ref:`server-plugins-structures-bundler-index` page -for details. - -If you have lots of unconnected items (for instance: software packages -whose configuration wasn't modified, and that are also not depended -on by other packages; or single directories or files not belonging -to a package), using Bundles in Metadata would clutter or enlarge -your ``Metadata/groups.xml`` file, because they all would need to be -explicitly specified. ``Base/`` on the other hand is the perfect place -to put these items. - -Without using Base, you would be forced to put them directly -into your group definitions in ``groups.xml``, either as many -small bundles (substantially enlarging it) or into something like -``Bundler/unrelated-entries.xml``. Using the latter is especially bad -if you mix packages and services in your Bundle, since for any updated -package in that bundle, the now-related services would be restarted. - -The Base entries can still be assigned based on group membership, but when -they aren't part of a group, each and every client gets the entry. So Base is -also a great place to put entries that a large number of your clients will -get. - -For example, you could have a file ``Base/packages.xml`` - -.. code-block:: xml - - <Base> - <Package name='acpid'/> - <Package name='auditd'/> - [...] - <Group name='openSUSE11.2'> - <Package name='syslog-ng'/> - </Group> - <Group name='openSUSE11.3'> - <Package name='rsyslog'/> - </Group> - [...] - <Package name='zlib'/> - </Base> - -.. note:: - - You don't have to reference to the files in Base from anywhere. As long - as you include ``Base`` in your ``plugins = ...`` line in ``bcfg2.conf``, - these are included automatically. - -.. note:: - - Your Base files have to match the pattern ``Base/*.xml`` to be included. - - -The decision when to use Base and when to use Bundler depends on the -configuration entry in question, and what you are trying to achieve. - -Base is mainly used for cases where you don't want/need to explicitly -include particular configuration items. Let's say all your machines are -various linux distributions. In this case, you may want to manage the -``/etc/hosts`` file using Base instead of Bundler since you will not have -to include any Bundles in your Metadata. However, you could alternatively -have a base 'linux' group that all the clients inherit which includes a -*linux* Bundle with the ``/etc/hosts`` configuration entry. diff --git a/doc/server/plugins/structures/bundler/bcfg2.txt b/doc/server/plugins/structures/bundler/bcfg2.txt new file mode 100644 index 000000000..0fd0a3fdf --- /dev/null +++ b/doc/server/plugins/structures/bundler/bcfg2.txt @@ -0,0 +1,87 @@ +.. -*- mode: rst -*- + +.. _server-plugins-structures-bundler-bcfg2-server: + +Bcfg2 Server +============ + +These two bundles split out the entries that do require a restart of +``bcfg2-server`` from those that don't. + +These bundles also demonstrate use of bound entries to avoid splitting +entries between Bundler and Rules. + +``Bundler/bcfg2-server.xml``: + +.. code-block:: xml + + <Bundle> + <Bundle name="bcfg2-server-base.xml"/> + + <Path name="/etc/pki/tls/private/bcfg2.key"/> + <Path name="/etc/sysconfig/bcfg2-server"/> + <Path name="/etc/bcfg2.conf"/> + + <BoundPath name="/var/lib/bcfg2/Packages/cache" type="directory" + owner="bcfg2" group="bcfg2" mode="0755"/> + <BoundPath name="/var/lib/bcfg2" type="symlink" + to="/var/lib/bcfg2-vcs/bcfg2/public"/> + <BoundPath name="/var/lib/bcfg2/etc/bcfg2.sqlite" type="permissions" + owner="bcfg2" group="apache" mode="0660"/> + + <BoundService name="bcfg2-server" type="chkconfig" status="on"/> + + <Package name="bcfg2-server"/> + <Package name="python-genshi"/> + <Package name="python-inotify"/> + <Package name="Django"/> + <Package name="Django-south"/> + <Package name="m2crypto"/> + <Package name="GitPython"/> + </Bundle> + +``Bundler/bcfg2-server-base.xml``: + +.. code-block:: xml + + <Bundle> + <Path name="/etc/bcfg2-web.conf"/> + <Path name="/etc/cron.daily/bcfg2_cleanup_db"/> + + <BoundPOSIXGroup name='bcfg2'/> + <BoundPOSIXUser name='bcfg2' shell='/sbin/nologin' gecos='Bcfg2 User'/> + <Path name="/home/bcfg2/.ssh/id_rsa"/> + + <!-- SSL CA setup --> + <BoundPath name="/etc/pki/CA" type="directory" important="true" + owner="bcfg2" group="bcfg2" mode="755"/> + <BoundPath name="/etc/pki/CA/crl" type="directory" owner="bcfg2" + group="bcfg2" mode="755"/> + <BoundPath name="/etc/pki/CA/certs" type="directory" owner="bcfg2" + group="bcfg2" mode="755"/> + <BoundPath name="/etc/pki/CA/newcerts" type="directory" owner="bcfg2" + group="bcfg2" mode="755"/> + <BoundPath name="/etc/pki/CA/private" type="directory" owner="bcfg2" + group="bcfg2" mode="755"/> + <Path name="/etc/pki/CA/openssl.cnf" altsrc="/etc/pki/CA/openssl.cnf"/> + <Path name="/etc/pki/CA/index.txt.attr"/> + <Path name="/etc/pki/CA/CA.crt"/> + <Path name="/etc/pki/CA/CA.key"/> + <Path name="/etc/pki/CA/CA.pem"/> + <Path name="/etc/pki/tls/certs/server-chain.crt"/> + <BoundPath name="/etc/pki/CA/serial" type="permissions" owner="bcfg2" + group="bcfg2" mode="0600"/> + <BoundPath name="/etc/pki/CA/index.txt" type="permissions" owner="bcfg2" + group="bcfg2" mode="0600"/> + <BoundPath name="/etc/pki/CA/crlnumber" type="permissions" owner="bcfg2" + group="bcfg2" mode="0644"/> + <BoundAction + name="create-CA-serial" timing="post" when="always" status="check" + command="[ -e /etc/pki/CA/serial ] || echo '01' > /etc/pki/CA/serial"/> + <BoundAction + name="create-CA-index" timing="post" when="always" status="check" + command="[ -e /etc/pki/CA/index.txt ] || touch /etc/pki/CA/index.txt"/> + <BoundAction + name="create-CA-crlnumber" timing="post" when="always" status="check" + command="[ -e /etc/pki/CA/crlnumber ] || touch /etc/pki/CA/crlnumber"/> + </Bundle> diff --git a/doc/server/plugins/structures/bundler/index.txt b/doc/server/plugins/structures/bundler/index.txt index a19959e66..0b6b8eb50 100644 --- a/doc/server/plugins/structures/bundler/index.txt +++ b/doc/server/plugins/structures/bundler/index.txt @@ -19,142 +19,112 @@ will receive. Group and Client tags can be used inside of bundles to differentiate which entries particular clients will recieve; this is useful for the case where entries are named differently across systems; for example, -one linux distro may have a package called openssh while another uses -the name ssh. Configuration entries nested inside of Group elements -only apply to clients who are a member of those groups; multiple -nested groups must all apply. Also, groups may be negated; entries -included in such groups will only apply to clients who are not a -member of said group. The same applies to Client elements. +one Linux distro may have a package called ``openssh`` while another +uses the name ``ssh``. See :ref:`xml-group-client-tags` for details +and a longer example. -The following is an annotated copy of a bundle: +A brief example: .. code-block:: xml - <Bundle name='ssh' version='2.0'> - <Path name='/etc/ssh/ssh_host_dsa_key'/> - <Path name='/etc/ssh/ssh_host_rsa_key'/> - <Path name='/etc/ssh/ssh_host_dsa_key.pub'/> - <Path name='/etc/ssh/ssh_host_rsa_key.pub'/> - <Path name='/etc/ssh/ssh_host_key'/> - <Path name='/etc/ssh/ssh_host_key.pub'/> - <Path name='/etc/ssh/sshd_config'/> + <Bundle> <Path name='/etc/ssh/ssh_config'/> - <Path name='/etc/ssh/ssh_known_hosts'/> <Group name='rpm'> - <Package name='openssh'/> - <Package name='openssh-askpass'/> <Service name='sshd'/> - <Group name='fedora' > - <Group name='fc14' negate='true'> - <Package name='openssh-clients'/> - </Group> - <Package name='openssh-server'/> - </Group> + <Package name='openssh-server'/> </Group> <Group name='deb'> <Package name='ssh'/> <Service name='ssh'/> </Group> - <Client name='trust.example.com'> - <Path name='/etc/ssh/shosts.equiv'/> - </Client> </Bundle> -In this bundle, most of the entries are common to all systems. Clients -in group **deb** get one extra package and service, while clients in -group **rpm** get two extra packages and an extra service. In -addition, clients in group **fedora** *and* group **rpm** get one -extra package entries, unless they are not in the **fc14** group, in -which case, they get an extra package. The client -**trust.example.com** gets one extra file that is not distributed to -any other clients. Notice that this file doesn't describe which -versions of these entries that clients should get, only that they -should get them. (Admittedly, this example is slightly contrived, but -demonstrates how group entries can be used in bundles) - -+----------------------------+-------------------------------+ -| Group/Hostname | Entry | -+============================+===============================+ -| all | /etc/ssh/ssh_host_dsa_key | -+----------------------------+-------------------------------+ -| all | /etc/ssh/ssh_host_rsa_key | -+----------------------------+-------------------------------+ -| all | /etc/ssh/ssh_host_dsa_key.pub | -+----------------------------+-------------------------------+ -| all | /etc/ssh/ssh_host_rsa_key.pub | -+----------------------------+-------------------------------+ -| all | /etc/ssh/ssh_host_key | -+----------------------------+-------------------------------+ -| all | /etc/ssh/ssh_host_key.pub | -+----------------------------+-------------------------------+ -| all | /etc/ssh/sshd_config | -+----------------------------+-------------------------------+ -| all | /etc/ssh/ssh_config | -+----------------------------+-------------------------------+ -| all | /etc/ssh/ssh_known_hosts | -+----------------------------+-------------------------------+ -| rpm | Package openssh | -+----------------------------+-------------------------------+ -| rpm | Package openssh-askpass | -+----------------------------+-------------------------------+ -| rpm | Service sshd | -+----------------------------+-------------------------------+ -| rpm and fedora | Package openssh-server | -+----------------------------+-------------------------------+ -| rpm and fedora and not fc4 | Package openssh-clients | -+----------------------------+-------------------------------+ -| deb | Package ssh | -+----------------------------+-------------------------------+ -| deb | Service ssh | -+----------------------------+-------------------------------+ -| trust.example.com | /etc/ssh/shosts.equiv | -+----------------------------+-------------------------------+ +Note that we do not specify *how* a given entry should be managed, +only that it should be. The concrete specification of each entry will +be provided by a different plugin such as +:ref:`server-plugins-generators-cfg`, +:ref:`server-plugins-generators-rules`, or +:ref:`server-plugins-generators-packages`. -Genshi templates -================ +Alternatively, you can use fully-bound entries in Bundler, which has +various uses. For instance: -Genshi XML templates allow you to use the `Genshi -<http://genshi.edgewall.org>`_ templating system to dynamically -generate a bundle. Genshi templates can be specified one of two ways: +.. code-block:: xml -1. Add an XML-style genshi template to the Bundler directory with a - ``.genshi`` and the associated namespace attribute. -2. Simply add the appropriate namespace attribute to your existing XML - bundle. + <Bundle> + <Path name='/etc/ssh/ssh_config'/> + <Group name='rpm'> + <BoundService name='sshd' type="chkconfig" status="on"/> + <BoundPackage name='openssh-server' version='5.8p2' type="yum" /> + </Group> + <Group name='deb'> + <Package name='ssh'/> + <BoundService name='ssh' type="chkconfig" status="on"/> + </Group> + </Bundle> -The top-level Bundle tag should look like the following:: +In this example, both Service tags and one Package tag are fully bound +-- i.e., all information required by the client to manage those +entries is provided in the bundle itself. - <Bundle name="foo" xmlns:py="http://genshi.edgewall.org/"> +.. _server-plugins-structures-bundler-magic: -Several variables are pre-defined inside templates: +Bundle "Magic" +============== -+-------------+--------------------------------------------------------+ -| Name | Description | -+=============+========================================================+ -| metadata | :ref:`Client metadata | -| | <server-plugins-grouping-metadata-clientmetadata>` | -+-------------+--------------------------------------------------------+ -| repo | The path to the Bcfg2 repository on the filesystem | -+-------------+--------------------------------------------------------+ +Bundles are collections of *related* entries. That point is very, +very important, because a bundle performs certain "magic" actions when +one or more entries in it are modified: + +* :xml:type:`Service <ServiceType>` entries whose ``restart`` + attribute is ``true`` (the default) will be restarted. +* :xml:type:`Action <ActionType>` entries whose ``when`` attribute is + ``modified`` will be run. + +Because of these two magic actions, it's extremely important to +structure your bundles around Service and Action entries, rather than +around some loose idea of which entries are related. For instance, in +order to manage a Bcfg2 server, a number of packages, paths, services, +etc. must be managed. But not all of these entries would require +``bcfg2-server`` to be restarted, so to limit restarts it's wise to +split these entries into two bundles. See +:ref:`server-plugins-structures-bundler-bcfg2-server` for an example +of this. + +Disabling Magic +--------------- -.. note:: +Disabling magic bundler actions can be done in one of two ways: - ``<Group>`` and ``<Client>`` tags are allowed inside of Genshi - templates as of Bcfg2 1.2. However, they do not behave the same - as using a Genshi conditional, e.g.:: +* On a per-entry basis. Set ``restart="false"`` on a Service to + prevent it from being restarted when the bundle is modified. Set + ``when="always"`` on an Action to cause it to run every time, + regardless of whether or not the bundle was modified. +* On a per-bundle basis. Set ``independent="true"`` on the top-level + ``Bundle`` tag to signify that the bundle is a collection of + independent (i.e., unrelated) entries, and to prevent any magic + actions from being performed. (This is similar to the ``Base`` + plugin in older versions of Bcfg2.) This was added in Bcfg2 1.4. - <py:if test="'groupname' in metadata.groups"> - </py:if> +Service entries in independent bundles are never restarted, and Action +entries in independent bundles are only executed if ``when="always"``. +(I.e., an Action entry in an independent bundle with +``when="modified"`` is useless.) + +Genshi templates +================ + +Genshi XML templates allow you to use the `Genshi +<http://genshi.edgewall.org>`_ templating system to dynamically +generate a bundle. Genshi templates can be specified one of two ways: - The conditional is evaluated when the template is rendered, so - code inside the conditional is not executed if the conditional - fails. A ``<Group>`` tag is evaluated *after* the template is - rendered, so code inside the tag is always executed. This is an - important distinction: if you have code that will fail on some - groups, you *must* use a Genshi conditional, not a ``<Group>`` - tag. The same caveats apply to ``<Client>`` tags. +1. Add an XML-style genshi template to the Bundler directory with a + ``.genshi`` and the associated namespace attribute. *This is + deprecated as of Bcfg2 1.4.* +2. Add the Genshi namespace to your existing XML + bundle. -See also the :ref:`xml-genshi-reference`. +See :ref:`xml-genshi-templating` for details. Troubleshooting --------------- @@ -168,6 +138,55 @@ entries in the bundle. See :ref:`bcfg2-info <server-bcfg2-info>` for more details. +Dependencies +============ + +Dependencies on other bundles can be specified by adding an empty +bundle tag that adds another bundle by name, e.g.: + +.. code-block:: xml + + <Bundle> + <Bundle name="nfs-client"/> + ... + </Bundle> + +The dependent bundle is added to the list of bundles sent to the +client, *not* to the parent bundle itself. In other words, if an +entry in the dependent bundle changes, Services are restarted and +Actions are run in the dependent bundle *only*. An example: + +``nfs-client.xml``: + +.. code-block:: xml + + <Bundle> + <Package name="nfs-utils"/> + <Service name="nfslock"/> + <Service name="rpcbind"/> + <Service name="nfs"/> + </Bundle> + +``automount.xml``: + +.. code-block:: xml + + <Bundle> + <Bundle name="nfs-client"/> + + <Path name="/mnt/home"/> + <Path name="/etc/auto.master"/> + <Path name="/etc/auto.misc"/> + <Service name="autofs"/> + <Package name="automount"/> + </Bundle> + +If a new ``nfs-utils`` package was installed, the ``nfslock``, +``rpcbind``, and ``nfs`` services would be restarted, but *not* the +``autofs`` service. Similarly, if a new ``/etc/auto.misc`` file was +sent out, the ``autofs`` service would be restarted, but the +``nfslock``, ``rpcbind``, and ``nfs`` services would not be restarted. + Altsrc ====== @@ -184,8 +203,8 @@ in their name. The following template produces such a config file entry. .. code-block:: xml - <Bundle name='foo' xmlns:py="http://genshi.edgewall.org/"> - <Path name='/etc/package-${metadata.hostname}'/> + <Bundle xmlns:py="http://genshi.edgewall.org/"> + <Path name='/etc/package-${metadata.hostname}'/> </Bundle> Depending on the circumstance, these configuration files can either be @@ -199,7 +218,7 @@ and returns them in a newline delimited string. .. code-block:: xml - <Bundle name="networkinterfaces" xmlns:py="http://genshi.edgewall.org/"> + <Bundle xmlns:py="http://genshi.edgewall.org/"> <?python files = metadata.Probes["getmacs"].split("\n") ?> @@ -219,7 +238,7 @@ if declaration. .. code-block:: xml - <Bundle name='bacula' xmlns:py="http://genshi.edgewall.org/"> + <Bundle xmlns:py="http://genshi.edgewall.org/"> <Path name="/etc/bacula/bconsole.conf"/> <Path name="/etc/bacula/bacula-fd.conf"/> <Path name="/etc/bacula/bacula-sd.conf"/> @@ -231,7 +250,7 @@ or alternately .. code-block:: xml - <Bundle name='bacula' xmlns:py="http://genshi.edgewall.org/"> + <Bundle xmlns:py="http://genshi.edgewall.org/"> <Path name="/etc/bacula/bconsole.conf"/> <Path name="/etc/bacula/bacula-fd.conf"/> <Path name="/etc/bacula/bacula-sd.conf"/> @@ -244,7 +263,7 @@ or yet another way .. code-block:: xml - <Bundle name='bacula' xmlns:py="http://genshi.edgewall.org/"> + <Bundle xmlns:py="http://genshi.edgewall.org/"> <Path name="/etc/bacula/bconsole.conf"/> <Path name="/etc/bacula/bacula-fd.conf"/> <Path name="/etc/bacula/bacula-sd.conf"/> @@ -274,6 +293,7 @@ more complex example Bundles. .. toctree:: :maxdepth: 1 + bcfg2 kernel moab nagios diff --git a/doc/server/plugins/structures/bundler/kernel.txt b/doc/server/plugins/structures/bundler/kernel.txt index d83679683..54f70606f 100644 --- a/doc/server/plugins/structures/bundler/kernel.txt +++ b/doc/server/plugins/structures/bundler/kernel.txt @@ -22,7 +22,7 @@ some of which might be better than this one. Feel free to hack as needed. .. code-block:: xml - <Bundle name='kernel'> + <Bundle> <Group name='sles8'> <!-- =================== ia32 ==================== --> <Group name='ia32'> diff --git a/doc/server/plugins/structures/bundler/moab.txt b/doc/server/plugins/structures/bundler/moab.txt index e0d96be74..8f747376a 100644 --- a/doc/server/plugins/structures/bundler/moab.txt +++ b/doc/server/plugins/structures/bundler/moab.txt @@ -9,7 +9,7 @@ This is a fairly simple Bundle for the Moab workload manager. .. code-block:: xml - <Bundle name='moab' version='2.0'> + <Bundle> <Path name='/var/spool/moab'/> <Path name='/var/spool/moab/moab.cfg'/> <Group name='moab-server'> diff --git a/doc/server/plugins/structures/bundler/nagios.txt b/doc/server/plugins/structures/bundler/nagios.txt index fa5b67f30..d25e1cf0a 100644 --- a/doc/server/plugins/structures/bundler/nagios.txt +++ b/doc/server/plugins/structures/bundler/nagios.txt @@ -12,7 +12,7 @@ the clients. .. code-block:: xml - <Bundle name='nagios-client' version='2.0'> + <Bundle> <Group name='sles8'> <Package name='ucdsnmp'/> </Group> diff --git a/doc/server/plugins/structures/bundler/ntp.txt b/doc/server/plugins/structures/bundler/ntp.txt index b1264b5ee..31bc8a97a 100644 --- a/doc/server/plugins/structures/bundler/ntp.txt +++ b/doc/server/plugins/structures/bundler/ntp.txt @@ -12,7 +12,7 @@ better through use of groups. .. code-block:: xml - <Bundle name='ntp'> + <Bundle> <Package name='xntp'/> <Path name='/etc/sysconfig/xntp'/> <Path name='/etc/sysconfig/clock'/> diff --git a/doc/server/plugins/structures/bundler/snmpd.txt b/doc/server/plugins/structures/bundler/snmpd.txt index 2318f8ca1..859e07f7f 100644 --- a/doc/server/plugins/structures/bundler/snmpd.txt +++ b/doc/server/plugins/structures/bundler/snmpd.txt @@ -10,7 +10,7 @@ configuration file. .. code-block:: xml - <Bundle name="snmpd" version="3.0"> + <Bundle> <Package name="snmpd"/> <Service name="snmpd"/> <Path name="/etc/snmp/snmpd.conf"/> diff --git a/doc/server/plugins/structures/bundler/torque.txt b/doc/server/plugins/structures/bundler/torque.txt index 32e6d4c30..f6349df6e 100644 --- a/doc/server/plugins/structures/bundler/torque.txt +++ b/doc/server/plugins/structures/bundler/torque.txt @@ -11,7 +11,7 @@ A longer Bundle that includes many group-specific entries. .. code-block:: xml - <Bundle name='torque' version='1.0'> + <Bundle> <Service name='nfs'/> <Service name='nfslock'/> <BoundPath type='directory' owner='root' group='root' mode='0755' name='/var/spool/torque'/> diff --git a/doc/server/plugins/structures/bundler/yp.txt b/doc/server/plugins/structures/bundler/yp.txt index 6eecb3304..9990fbc2c 100644 --- a/doc/server/plugins/structures/bundler/yp.txt +++ b/doc/server/plugins/structures/bundler/yp.txt @@ -14,7 +14,7 @@ treatment too. .. code-block:: xml - <Bundle name='yp' version='2.0'> + <Bundle> <Package name='yp-tools'/> <Path name='/etc/nsswitch.conf'/> <Path name='/etc/yp.conf'/> diff --git a/doc/server/plugins/version/bzr.txt b/doc/server/plugins/version/bzr.txt index 0755bf80c..ae247985f 100644 --- a/doc/server/plugins/version/bzr.txt +++ b/doc/server/plugins/version/bzr.txt @@ -21,7 +21,7 @@ How to enable the Bazaar plugin Simply add "Bzr" to your plugins line in ``/etc/bcfg2.conf``:: [server] - plugins = Base,Bundler,Cfg,...,Bzr + plugins = Bundler,Cfg,...,Bzr Usage notes =========== diff --git a/doc/server/plugins/version/cvs.txt b/doc/server/plugins/version/cvs.txt index a80b1edbc..f969302d0 100644 --- a/doc/server/plugins/version/cvs.txt +++ b/doc/server/plugins/version/cvs.txt @@ -21,4 +21,4 @@ How to enable the CVS plugin Simply add "Cvs" to your plugins line in ``/etc/bcfg2.conf``:: [server] - plugins = Base,Bundler,Cfg,...,Cvs + plugins = Bundler,Cfg,...,Cvs diff --git a/doc/server/plugins/version/darcs.txt b/doc/server/plugins/version/darcs.txt index 30ac0176a..6fa384679 100644 --- a/doc/server/plugins/version/darcs.txt +++ b/doc/server/plugins/version/darcs.txt @@ -6,7 +6,7 @@ Darcs ===== -This page describes the new Darcs plugin which is experimental. +This page describes the new Darcs plugin which is experimental. Why use the Darcs plugin ======================== @@ -25,4 +25,4 @@ You will need to install Darcs on the Bcfg2 server first. Once installed, simply add Darcs to your plugins line in ``/etc/bcfg2.conf``:: [server] - plugins = Base,Bundler,Cfg,...,Darcs + plugins = Bundler,Cfg,...,Darcs diff --git a/doc/server/plugins/version/fossil.txt b/doc/server/plugins/version/fossil.txt index 7bf523a9e..a19c21760 100644 --- a/doc/server/plugins/version/fossil.txt +++ b/doc/server/plugins/version/fossil.txt @@ -21,4 +21,4 @@ How to enable the Fossil plugin Simply add "Fossil" to your plugins line in ``/etc/bcfg2.conf``:: [server] - plugins = Base,Bundler,Cfg,...,Fossil + plugins = Bundler,Cfg,...,Fossil diff --git a/doc/server/plugins/version/hg.txt b/doc/server/plugins/version/hg.txt index 747699f0e..a11623836 100644 --- a/doc/server/plugins/version/hg.txt +++ b/doc/server/plugins/version/hg.txt @@ -22,4 +22,4 @@ You will need to install Mercurial on the Bcfg2 server first. Simply add Hg to your plugins line in ``/etc/bcfg2.conf``:: [server] - plugins = Base,Bundler,Cfg,...,Hg + plugins = Bundler,Cfg,...,Hg diff --git a/doc/server/snapshots/index.txt b/doc/server/snapshots/index.txt deleted file mode 100644 index a7e5940ed..000000000 --- a/doc/server/snapshots/index.txt +++ /dev/null @@ -1,155 +0,0 @@ -.. -*- mode: rst -*- - -.. _server-snapshots-index: - -=============== -Bcfg2 Snapshots -=============== - -.. versionadded:: 1.0.0 - -This page describes the Snapshots plugin. Snapshots is deprecated, and -will be removed in a future release. - -Before you begin -================ - -Make sure you have version 0.5 or greater of sqlalchemy. - -On CentOS/RHEL 5 ----------------- - -* Download a tarball of SQLAlchemy. -* Extract and build the RPM:: - - tar xzf SQLAlchemy-0.5.6.tar.gz - cd SQLAlchemy-0.5.6 - python setup.py bdist_rpm - -* Copy the RPM in ``SQLAlchemy-0.5.6/dist/`` to your Yum repository, - and rebuild the repository using ``createrepo``. -* Clear the Yum cache:: - - sudo yum clean all - -* Install SQLAlchemy:: - - sudo yum install SQLAlchemy - -* Manage the package in Bcfg2 as you would any other package. - -Configuration -============= - -* A database location needs to be added to ``bcfg2.conf``. Three drivers - are currently supported; mysql, postgres, and sqlite. When using the - sqlite driver, only the driver and database lines are required. - - * For MySQL:: - - [snapshots] - driver = mysql - database = snapshots - user = snapshots - password = snapshots - host = dbserver - - * For SQLite:: - - [snapshots] - driver = sqlite - database = /var/lib/bcfg2/var/snapshots.sqlite - -* The database needs to be initialized.:: - - $ bcfg2-admin snapshots init - 2009-03-22 21:40:24,683 INFO sqlalchemy.engine.base.Engine.0x...3e2c PRAGMA table_info("connkeyval") - PRAGMA table_info("connkeyval") - 2009-03-22 21:40:24,684 INFO sqlalchemy.engine.base.Engine.0x...3e2c () - () - 2009-03-22 21:40:24,686 INFO sqlalchemy.engine.base.Engine.0x...3e2c PRAGMA table_info("package") - PRAGMA table_info("package") - 2009-03-22 21:40:24,687 INFO sqlalchemy.engine.base.Engine.0x...3e2c () - () - ..... - COMMIT - -* The Snapshots plugin needs to be enabled for the bcfg2-server (by adding - Snapshots to the plugins line in ``/etc/bcfg2.conf``). Once done, - this will cause the the server to store statistics information when - clients run. - -Using the reports interface -=========================== - -All hosts:: - - $ bcfg2-admin snapshots reports -a - - ============= ========= ========================================== ============================ - Client Correct Revision Time - ============= ========= ========================================== ============================ - bcfg2client True f46ac7773712bd3c3cfb765ae5d2a3b2a37ac9b7 2009-04-23 11:27:54.378941 - ============= ========= ========================================== ============================ - -List bad entries for a single host:: - - $ bcfg2-admin snapshots reports -b bcfg2client - Bad entries: - Package:nscd - Package:cupsys - File:/etc/ldap.conf - -List extra entries for a single host:: - - $ bcfg2-admin snapshots reports -e bcfg2client - Extra entries: - Package:python-pyxattr - Package:librsync1 - Package:python-pylibacl - Package:gcc-4.2-multilib - Package:nxlibs - Package:freenx-session-launcher - Package:dx-doc - Package:dirdiff - Package:libhdf4g - Package:nxclient - Package:freenx-rdp - Package:freenx-vnc - Package:libxml2-dev - Package:mysql-client - Package:mysql-client-5.0 - Package:libxcompext3 - Package:lib32gomp1 - Package:dx - Package:freenx-media - Package:dxsamples - Package:gcc-multilib - Package:rdiff-backup - Package:libdbd-mysql-perl - Package:libxcomp3 - Package:freenx-server - Package:smbfs - Package:planner - Package:nxagent - Package:libc6-dev-i386 - Package:libfltk1.1-dev - Package:freenx - Package:libdx4 - Package:libxcompshad3 - Service:freenx-server - -Detailed view of hosts for a particular date:: - - $ bcfg2-admin snapshots reports --date 2009 5 30 - ============= ========= ========================================== ============================ - Client Correct Revision Time - ============= ========= ========================================== ============================ - bcfg2client False 10c1a12c62c57c0861cc453b8d2640c4839a7357 2009-05-29 10:52:34.701056 - -TODO/Wishlist -============= - -* Identify per-client changes in correctness over time -* Detailed view for a particular date -* Track entry changes over time (glibc updated on these dates to these versions) diff --git a/doc/server/xml-common.txt b/doc/server/xml-common.txt new file mode 100644 index 000000000..3aacfd468 --- /dev/null +++ b/doc/server/xml-common.txt @@ -0,0 +1,399 @@ +.. -*- mode: rst -*- + +.. _xml-features: + +===================== + Common XML Features +===================== + +Most of the XML files in Bcfg2 have a common set of features that are +supported. These are described in some detail below, and a precise +rundown of which features are supported by which files is provided. + +.. _xml-group-client-tags: + +Group and Client tags +===================== + +These allow the portions of an XML document inside a Client or Group +tag to only apply to the given client group. That is, they can be +thought of as conditionals, where the following are roughly equivalent: + +.. code-block:: xml + + <Group name="group1"> + <Path name="/etc/foo.conf"/> + </Group> + +And:: + + If client is a member of group1 then + Manage the abstract path "/etc/foo.conf" + +Nested Group and Client tags are conjunctive (logical ``AND``). For +instance, the following are roughly equivalent: + +.. code-block:: xml + + <Group name="group1"> + <Client name="foo.example.com"> + <Package name="bar"/> + </Client> + <Package name="baz"/> + </Group> + +And:: + + If client is a member of group1 and has hostname "foo.example.com" then + Manage the abstract package "bar" + If client is a member of group1 then + Manage the abstract package "baz" + +There is no convenient ``else``; you must specify all conditions +explicitly. To do this, Group and Client tags may be negated, as in: + +.. code-block:: xml + + <Group name="group1"> + <Service name="foo"/> + </Group> + <Group name="group1" negate="true"> + <Service name="bar"/> + </Group> + +This is roughly equivalent to:: + + If client is a member of group1 then + Manage the abstract service "foo" + If client is not a member of group 1 then + Manage the abstract service "bar" + +Or, more compactly: + + If client is a member of group1 then + Manage the abstract service "foo" + Else + Manage the abstract service "bar" + +As an example, consider the following :ref:`bundle +<server-plugins-structures-bundler-index>`: + +.. code-block:: xml + + <Bundle> + <Path name='/etc/ssh/ssh_host_dsa_key'/> + <Path name='/etc/ssh/ssh_host_rsa_key'/> + <Path name='/etc/ssh/ssh_host_dsa_key.pub'/> + <Path name='/etc/ssh/ssh_host_rsa_key.pub'/> + <Path name='/etc/ssh/ssh_host_key'/> + <Path name='/etc/ssh/ssh_host_key.pub'/> + <Path name='/etc/ssh/sshd_config'/> + <Path name='/etc/ssh/ssh_config'/> + <Path name='/etc/ssh/ssh_known_hosts'/> + <Group name='rpm'> + <Package name='openssh'/> + <Package name='openssh-askpass'/> + <Service name='sshd'/> + <Group name='fedora' > + <Group name='fedora14' negate='true'> + <Package name='openssh-clients'/> + </Group> + <Package name='openssh-server'/> + </Group> + </Group> + <Group name='deb'> + <Package name='ssh'/> + <Service name='ssh'/> + </Group> + <Client name='trust.example.com'> + <Path name='/etc/ssh/shosts.equiv'/> + </Client> + </Bundle> + +In this bundle, most of the entries are common to all systems. Clients +in group ``deb`` get one extra package and service, while clients in +group ``rpm`` get two extra packages and an extra service. In +addition, clients in group ``fedora`` *and* group ``rpm`` get one +extra package entries, unless they are not in the ``fedora14`` group, +in which case, they get an extra package. The client +``trust.example.com`` gets one extra file that is not distributed to +any other clients. + ++------------------------+-----------------------------------+ +| Group/Hostname | Entry | ++========================+===================================+ +| all | ``/etc/ssh/ssh_host_dsa_key`` | ++------------------------+-----------------------------------+ +| all | ``/etc/ssh/ssh_host_rsa_key`` | ++------------------------+-----------------------------------+ +| all | ``/etc/ssh/ssh_host_dsa_key.pub`` | ++------------------------+-----------------------------------+ +| all | ``/etc/ssh/ssh_host_rsa_key.pub`` | ++------------------------+-----------------------------------+ +| all | ``/etc/ssh/ssh_host_key`` | ++------------------------+-----------------------------------+ +| all | ``/etc/ssh/ssh_host_key.pub`` | ++------------------------+-----------------------------------+ +| all | ``/etc/ssh/sshd_config`` | ++------------------------+-----------------------------------+ +| all | ``/etc/ssh/ssh_config`` | ++------------------------+-----------------------------------+ +| all | ``/etc/ssh/ssh_known_hosts`` | ++------------------------+-----------------------------------+ +| ``rpm`` | Package ``openssh`` | ++------------------------+-----------------------------------+ +| ``rpm`` | Package ``openssh-askpass`` | ++------------------------+-----------------------------------+ +| ``rpm`` | Service ``sshd`` | ++------------------------+-----------------------------------+ +| ``rpm`` AND ``fedora`` | Package ``openssh-server`` | ++------------------------+-----------------------------------+ +| ``rpm`` AND ``fedora`` | Package ``openssh-clients`` | +| AND NOT ``fedora14`` | | ++------------------------+-----------------------------------+ +| ``deb`` | Package ``ssh`` | ++------------------------+-----------------------------------+ +| ``deb`` | Service ``ssh`` | ++------------------------+-----------------------------------+ +| ``trust.example.com`` | ``/etc/ssh/shosts.equiv`` | ++------------------------+-----------------------------------+ + +.. _xml-genshi-templating: + +Genshi templating +================= + +Genshi XML templates allow you to use the `Genshi +<http://genshi.edgewall.org>`_ templating system to dynamically +generate XML file content for a given client. Genshi templating can +be enabled on a file by adding the Genshi namespace to the top-level +tag, e.g.: + +.. code-block:: xml + + <Bundle xmlns:py="http://genshi.edgewall.org/"> + +Several variables are pre-defined inside Genshi XML templates: + ++-------------+--------------------------------------------------------+ +| Name | Description | ++=============+========================================================+ +| metadata | :ref:`Client metadata | +| | <server-plugins-grouping-metadata-clientmetadata>` | ++-------------+--------------------------------------------------------+ +| repo | The path to the Bcfg2 repository on the filesystem | ++-------------+--------------------------------------------------------+ + +.. note:: + + ``<Group>`` and ``<Client>`` tags can be used inside templates as + of Bcfg2 1.2, but they do not behave the same as using a Genshi + conditional, e.g.:: + + <py:if test="'groupname' in metadata.groups"> + </py:if> + + The conditional is evaluated when the template is rendered, so + code inside the conditional is not executed if the conditional + fails. A ``<Group>`` tag is evaluated *after* the template is + rendered, so code inside the tag is always executed. This is an + important distinction: if you have code that will fail on some + groups, you *must* use a Genshi conditional, not a ``<Group>`` + tag. The same caveats apply to ``<Client>`` tags. + +.. _xml-genshi-reference: + +Genshi XML Template Reference +----------------------------- + +The Genshi XML templating language is described in depth at `Genshi +<http://genshi.edgewall.org>`_. The XML schema reference follows. + +Genshi Tags +~~~~~~~~~~~ + +.. xml:group:: genshiElements + :namespace: py + +Genshi Attributes +~~~~~~~~~~~~~~~~~ + +.. xml:attributegroup:: genshiAttrs + :namespace: py + +.. _xml-encryption: + +Encryption +========== + +You can encrypt data in XML files to protect that data from other +people who need access to the repository. The data is decrypted +transparently on-the-fly by the server. + +.. note:: + + This feature is *not* intended to secure the files against a + malicious attacker who has gained access to your Bcfg2 server, as + the encryption passphrases are held in plaintext in + ``bcfg2.conf``. This is only intended to make it easier to use a + single Bcfg2 repository with multiple admins who should not + necessarily have access to each other's sensitive data. + +XML files are encrypted on a per-element basis; that is, rather than +encrypting the whole file, only the character content of individual +elements is encrypted. This makes it easier to track changes to the +file in a VCS, and also lets unprivileged users work with the other +data in the file. Only character content of an element can be +encrypted; attribute content and XML elements themselves cannot be +encrypted. + +By default, decryption is *strict*; that is, if any element cannot be +decrypted, parsing of the file is aborted. See +:ref:`server-encryption-lax-strict` for information on changing this +on a global or per-file basis. + +To encrypt or decrypt a file, use :ref:`bcfg2-crypt`. + +See :ref:`server-encryption` for more details on encryption in Bcfg2 +in general. + +XInclude +======== + +.. versionadded:: 0.9.0 + +`XInclude <http://www.w3.org/TR/xinclude/>`_ is a W3C specification +for the inclusion of external XML documents into XML source files, +allowing complex definitions to be split into smaller, more manageable +pieces. For instance, in the :ref:`server-plugins-grouping-metadata` +``groups.xml`` file, you might do: + +.. code-block:: xml + + <Groups xmlns:xi="http://www.w3.org/2001/XInclude"> + <xi:include href="my-groups.xml" /> + <xi:include href="their-groups.xml" /> + </Groups> + +To enable XInclude on a file, you need only add the XInclude namespace +to the top-level tag. + +You can also *optionally* include a file that may or may not exist +with the ``fallback`` tag: + +.. code-block:: xml + + <Groups xmlns:xi="http://www.w3.org/2001/XInclude"> + <xi:include href="my-groups.xml"/> + <xi:include href="their-groups.xml"><xi:fallback/></xi:include> + </Groups> + +In this case, if ``their-groups.xml`` does not exist, no error will be +raised and everything will work fine. (You can also use ``fallback`` +to include a different file, or explicit content in the case that the +parent include does not exist.) + +XInclude can only include complete, well-formed XML files. In some +cases, it may not be entirely obvious or intuitive how to structure +such an included file to conform to the schema, although in general +the included files should be structure exactly like the parent file. + +Wildcard XInclude +----------------- + +.. versionadded:: 1.3.1 + +Bcfg2 supports an extension to XInclude that allows you to use shell +globbing in the hrefs. (Stock XInclude doesn't support this, since +the href is supposed to be a URL.) + +For instance: + +.. code-block:: xml + + <Groups xmlns:xi="http://www.w3.org/2001/XInclude"> + <xi:include href="groups/*.xml"/> + </Groups> + +This would include all ``*.xml`` files in the ``groups`` subdirectory. + +Note that if a glob finds no files, that is treated the same as if a +single included file does not exist. You should use the ``fallback`` +tag, described above, if a glob may potentially find no files. + +Feature Matrix +============== + ++---------------------------------------------------+--------------+--------+------------+------------+ +| File | Group/Client | Genshi | Encryption | XInclude | ++===================================================+==============+========+============+============+ +| :ref:`ACL ip.xml <server-plugins-misc-acl>` | No | No | No | Yes | ++---------------------------------------------------+--------------+--------+------------+------------+ +| :ref:`ACL metadata.xml | Yes | Yes | Yes | Yes | +| <server-plugins-misc-acl>` | | | | | ++---------------------------------------------------+--------------+--------+------------+------------+ +| :ref:`Bundler | Yes | Yes | Yes | Yes | +| <server-plugins-structures-bundler-index>` | | | | | ++---------------------------------------------------+--------------+--------+------------+------------+ +| :ref:`info.xml <server-info>` | Yes [#f1]_ | Yes | Yes | Yes | ++---------------------------------------------------+--------------+--------+------------+------------+ +| :ref:`privkey.xml and pubkey.xml | Yes | Yes | Yes | Yes [#f2]_ | +| <server-plugins-generators-cfg-sshkeys>` | | | | | ++---------------------------------------------------+--------------+--------+------------+------------+ +| :ref:`authorizedkeys.xml | Yes | Yes | Yes | Yes | +| <server-plugins-generators-cfg-sshkeys>` | | | | | ++---------------------------------------------------+--------------+--------+------------+------------+ +| :ref:`sslcert.xml and sslkey.xml | Yes | Yes | Yes | Yes | +| <server-plugins-generators-cfg-ssl-certificates>` | | | | | ++---------------------------------------------------+--------------+--------+------------+------------+ +| :ref:`Decisions | Yes | Yes | Yes | Yes | +| <server-plugins-generators-decisions>` | | | | | ++---------------------------------------------------+--------------+--------+------------+------------+ +| :ref:`Defaults | Yes | Yes | Yes | Yes | +| <server-plugins-structures-defaults>` | | | | | ++---------------------------------------------------+--------------+--------+------------+------------+ +| :ref:`FileProbes | Yes | Yes | Yes | Yes | +| <server-plugins-probes-fileprobes>` | | | | | ++---------------------------------------------------+--------------+--------+------------+------------+ +| :ref:`GroupPatterns | No | No | No | Yes | +| <server-plugins-grouping-grouppatterns>` | | | | | ++---------------------------------------------------+--------------+--------+------------+------------+ +| :ref:`Metadata clients.xml | No | No | No | Yes | +| <server-plugins-grouping-metadata-clients-xml>` | | | | | ++---------------------------------------------------+--------------+--------+------------+------------+ +| :ref:`Metadata groups.xml | Yes [#f3]_ | No | No | Yes | +| <server-plugins-grouping-metadata-groups-xml>` | | | | | ++---------------------------------------------------+--------------+--------+------------+------------+ +| :ref:`NagiosGen | Yes | Yes | Yes | Yes | +| <server-plugins-generators-nagiosgen>` | | | | | ++---------------------------------------------------+--------------+--------+------------+------------+ +| :ref:`Packages | Yes | Yes | Yes | Yes | +| <server-plugins-generators-packages>` | | | | | ++---------------------------------------------------+--------------+--------+------------+------------+ +| :ref:`Pkgmgr | Yes | No | No | No | +| <server-plugins-generators-pkgmgr>` | | | | | ++---------------------------------------------------+--------------+--------+------------+------------+ +| :ref:`Properties | Yes [#f4]_ | Yes | Yes | Yes | +| <server-plugins-connectors-properties>` | | | | | ++---------------------------------------------------+--------------+--------+------------+------------+ +| :ref:`Rules <server-plugins-generators-rules>` | Yes | Yes | Yes | Yes | ++---------------------------------------------------+--------------+--------+------------+------------+ + +.. rubric:: Footnotes + +.. [#f1] ``info.xml`` also supports conditional Path tags; see + :ref:`server-info` for more. +.. [#f2] XInclude is supported, but the schema has not been modified + to allow including files that are structured exactly like the + parent. You may need to read the schema to understand how to + use XInclude properly. +.. [#f3] The semantics of Group tags in ``groups.xml`` is slightly + different; see + :ref:`server-plugins-grouping-metadata-groups-xml` for + details. +.. [#f4] Group and Client tags in XML Properties are not automatic by + default; they can be resolved by use of either the + ``Match()`` or ``XMLMatch()`` methods, or by use of the + :ref:`server-plugins-connectors-properties-automatch` + feature. See :ref:`server-plugins-connectors-properties-xml` + for details. diff --git a/doc/unsorted/index.txt b/doc/unsorted/index.txt index a369ee1b3..74d045990 100644 --- a/doc/unsorted/index.txt +++ b/doc/unsorted/index.txt @@ -13,7 +13,6 @@ list below. .. _TitleIndex: https://trac.mcs.anl.gov/projects/bcfg2/wiki/TitleIndex -* `Plugins/Snapshots` * `PrecompiledPackages` * `SchemaEvolution` * `SecurityDevPlan` diff --git a/doc/unsorted/writing_specification.txt b/doc/unsorted/writing_specification.txt index 700c1ab72..e7763cee1 100644 --- a/doc/unsorted/writing_specification.txt +++ b/doc/unsorted/writing_specification.txt @@ -26,12 +26,12 @@ that a client needs the Bcfg2 package with .. code-block:: xml - <Package name=bcfg2/> + <Package name="bcfg2"/> but this does not explicitly identify that an RPM package version 0.9.2 should be loaded from http://rpm.repo.server/bcfg2-0.9.2-0.1.rpm. -The abstract configuration is defined in the xml configuration files -for the Base and Bundles plugins. +The abstract configuration is defined in the XML configuration files +for the Bundler plugin. A combination of a clients metadata (group memberships) and abstract configuration is then used to generate the clients literal configuration. @@ -57,35 +57,13 @@ Abstract Configuration (Structures) =================================== A clients Abstract Configuration is the inventory of configuration -entities that should be installed on a client. Two plugins provide the -basis for the abstract configuration, the Bundler and Base. +entities that should be installed on a client. The Bundler plugin +usually provides the abstract configuration. The plugin Bundler builds descriptions of interrelated configuration entities. These are typically used for the representation of services, or other complex groups of entities. -The Base provides a laundry list of configuration entities that need to -be installed on hosts. These entities are independent from one another, -and can be installed individually without worrying about the impact on -other entities. - -Usage of Groups in Base and Bundles ------------------------------------ - -Groups are used by the Base and Bundles plugins for selecting -Configuration Entity Types for inclusion in a clients abstract -configuration. They can be thought of as:: - - if client is a member of group1 then - assign to abstract config - -Nested groups are conjunctive (logical and).:: - - if client is a member of group1 and group2 then - assign to abstract config - -Group membership maybe negated. See "Writing Bundles" for an example. - Configuration Entity Types -------------------------- @@ -121,9 +99,8 @@ consist of If any of these pieces are installed or updated, all should be rechecked and any associated services should be restarted. -All files in the Bundles/ subdirectory of the repository are processed. -Each bundle must be defined in its own file and the filename must be the -same as the bundle name with a .xml suffix.:: +All files in the Bundles/ subdirectory of the repository are +processed. Each bundle must be defined in its own file:: # ls Bundler Glide3.xml @@ -144,17 +121,6 @@ same as the bundle name with a .xml suffix.:: atftp.xml .... -Groups can be used inside of bundles to differentiate which entries -particular clients will receive. This is useful for the case where -entries are named differently across systems; for example, one linux -distro may have a package called openssh while another uses the name ssh. -Configuration entries nested inside of Group elements only apply to -clients who are a member of those groups; multiply nested groups must -all apply. - -Also, groups may be negated; entries included in such groups will only -apply to clients who are not a member of said group. - When packages in a bundle are verified by the client toolset, the Paths included in the same bundle are taken into consideration. That is, a package will not fail verification from a Bcfg2 perspective if the @@ -165,7 +131,7 @@ The following is an annotated copy of a bundle: .. code-block:: xml - <Bundle name='ssh' version='2.0'> + <Bundle> <Path name='/etc/ssh/ssh_host_dsa_key'/> <Path name='/etc/ssh/ssh_host_rsa_key'/> <Path name='/etc/ssh/ssh_host_dsa_key.pub'/> @@ -268,26 +234,3 @@ A Generator can take care of a particular configuration element. Any time this element is requested by the client, the server dynamically generates it either by crunching data and creating new information or by reading a file off of disk and passes it down to the client for installation. - -Usage of Groups in Generators ------------------------------ - -Similar to Abstract Configuration plugins, groups are used by generator -plugins for selecting Configuration Entities for inclusion in a clients -literal configuration. They can be thought of as:: - - if client is a member of group1 then - assign to abstract config - -Nested groups are conjunctive (logical and).:: - - if client is a member of group1 and group2 then - assign to abstract config - -How the groups are configured is specific to the plugin, but here are -two common methods: - -* xml configuration file (Pkgmgr, Rules) -* file name encoding (Cfg, SSHBase) - -Details are included on each plugin's page. |