diff options
37 files changed, 623 insertions, 280 deletions
diff --git a/doc/appendix/guides/fedora.txt b/doc/appendix/guides/fedora.txt index d8d3b1b34..1dd4f6db0 100644 --- a/doc/appendix/guides/fedora.txt +++ b/doc/appendix/guides/fedora.txt @@ -12,22 +12,33 @@ This guide is work in progess. This is a complete getting started guide for Fedora. With this -document you should be able to install a Bcfg2 server, a Bcfg2 client, +document you should be able to install a Bcfg2 server, a Bcfg2 client, and change the ``/etc/motd`` file on the client. +Prerequisites +============= + +To setup a configuration management system based on Bcfg2 only a few +prerequisites need to be fullfilled. + +* A server machine that can host the Bcfg2 +* Internet access for the installation process +* A working network with DNS + + Install Bcfg2 From RPM ====================== -The fastest way to get Bcfg2 onto your system is to use ``yum`` -or PackageKit. `` -um`` will pull all dependencies of Bcfg2 +The fastest way to get Bcfg2 onto your system is to use ``yum`` +or PackageKit. ``yum`` will pull all dependencies of Bcfg2 automatically in. :: $ su -c 'yum install bcfg2-server bcfg2' -Your system should now have the necessary software to use Bcfg2. +Your system should now have the necessary software to use Bcfg2. The next step is to set up your Bcfg2 :term:`repository`. + Initialize your repository ========================== @@ -38,14 +49,14 @@ is a tool which allows you to automate this: .. code-block:: sh # bcfg2-admin init - Store bcfg2 configuration in [/etc/bcfg2.conf]: - Location of bcfg2 repository [/var/lib/bcfg2]: + Store bcfg2 configuration in [/etc/bcfg2.conf]: + Location of bcfg2 repository [/var/lib/bcfg2]: Directory /var/lib/bcfg2 exists. Overwrite? [y/N]:y - Input password used for communication verification (without echoing; leave blank for a random): + Input password used for communication verification (without echoing; leave blank for a random): What is the server's hostname: [config01.local.net] - Input the server location [https://config01.local.net:6789]: + Input the server location [https://config01.local.net:6789]: Input base Operating System for clients: - 1: Redhat/Fedora/RHEL/RHAS/Centos + 1: Red Hat/Fedora/RHEL/RHAS/Centos 2: SUSE/SLES 3: Mandrake 4: Debian @@ -68,7 +79,7 @@ Change responses as necessary. Start the server ================ -You are now ready to start your bcfg2 server for the first time:: +You are now ready to start your Bcfg2 server for the first time:: $ su -c '/etc/init.d/bcfg2-server start' Starting Configuration Management Server: bcfg2-server [ OK ] @@ -90,12 +101,12 @@ Run ``bcfg2`` to be sure you are able to communicate with the server: .. code-block:: sh $ su -c 'bcfg2 -vqne' - + /usr/lib/python2.6/site-packages/Bcfg2/Client/Tools/rpmtools.py:23: DeprecationWarning: the md5 module is deprecated; use hashlib instead import md5 Loaded plugins: presto, refresh-packagekit Loaded tool drivers: - Action Chkconfig POSIX YUMng + Action Chkconfig POSIX YUMng Extra Package imsettings-libs 0.108.0-2.fc13.i686. Extra Package PackageKit-device-rebind 0.6.4-1.fc13.i686. ... @@ -114,11 +125,11 @@ Run ``bcfg2`` to be sure you are able to communicate with the server: Incorrect entries: 0 Total managed entries: 0 Unmanaged entries: 1314 - Package:ConsoleKit Package:jasper-libs Package:pcsc-lite-libs - Package:ConsoleKit-libs Package:java-1.5.0-gcj Package:perf - ... - Package:iw Package:pcre Service:sshd - Package:jack-audio-connection-kit Package:pcsc-lite Service:udev-post + Package:ConsoleKit Package:jasper-libs Package:pcsc-lite-libs + Package:ConsoleKit-libs Package:java-1.5.0-gcj Package:perf + ... + Package:iw Package:pcre Service:sshd + Package:jack-audio-connection-kit Package:pcsc-lite Service:udev-post The ``bcfg2.conf`` file contains only standard plugins so far. @@ -161,17 +172,17 @@ The ``bcfg2.conf`` file contains only standard plugins so far. Add the machines to Bcfg2 ------------------------- -``bcfg2-admin`` can be used to add a machine to Bcfg2 easily. You +``bcfg2-admin`` can be used to add a machine to Bcfg2 easily. You need to know the Fully Qualified Domain Name (FQDN) of ever system -you want to control through Bcfg2. :: +you want to control through Bcfg2. :: - bcfg2-admin client add <FQDN machine> + bcfg2-admin client add <FQDN machine> Bring your first machine under Bcfg2 control -------------------------------------------- -Now it is time to get the first machine's configuration into the -Bcfg2 repository. The server will be the first machine. It's +Now it is time to get the first machine's configuration into the +Bcfg2 repository. The server will be the first machine. It's already in the ``Metadata/client.xml``. @@ -181,13 +192,13 @@ Setup the `Packages`_ plugin .. _Packages: http://trac.mcs.anl.gov/projects/bcfg2/wiki/Plugins/Packages First, replace **Pkgmgr** with **Packages** in the plugins -line of ``bcfg2.conf``. Then create `Packages/` directory in +line of ``bcfg2.conf``. Then create `Packages/` directory in ``/var/lib/bcfg2`` :: $ su -c 'mkdir /var/lib/bcfg2/Packages' -Create a ``config.xml`` file for the packages in -``/var/lib/bcfg2/Packages`` with the following content. Choose a +Create a ``config.xml`` file for the packages in +``/var/lib/bcfg2/Packages`` with the following content. Choose a mirror near your location according the `Mirror list`_ . .. _Mirror list: http://mirrors.fedoraproject.org/publiclist/ @@ -229,11 +240,11 @@ something like this <Group name='suse'/> <Group name='mandrake'/> <Group name='solaris'/> - </Groups> + </Groups> .. note:: - When editing your xml files by hand, it is useful to occasionally - run ``bcfg2-repo-validate`` to ensure that your xml validates + When editing your xml files by hand, it is useful to occasionally + run ``bcfg2-repo-validate`` to ensure that your xml validates properly. Add a probe @@ -241,7 +252,7 @@ Add a probe The next step for the client will be to have the proper arch group membership. For this, we will make use of the -:ref:`server-plugins-grouping-dynamic_groups` capabilities of +:ref:`server-plugins-grouping-dynamic_groups` capabilities of the Probes plugin. Add **Probes** to your plugins line in ``bcfg2.conf`` and create the Probe: @@ -271,7 +282,7 @@ Start managing packages +++++++++++++++++++++++ Add a base-packages bundle. Let's see what happens when we just populate -it with the *yum* package. Create the ``base-packages.xml`` in your +it with the *yum* package. Create the ``base-packages.xml`` in your ``Bundler/`` directory with a entry for ``yum``. .. code-block:: xml @@ -295,22 +306,22 @@ Now if we run the client, we can see what this has done for us.:: output -As you can see, the Packages plugin has generated the dependencies -required for the yum package automatically. The ultimate goal should -be to move all the packages from the **Unmanaged** entries section -to the **Managed** entries section. So, what exactly *are* those +As you can see, the Packages plugin has generated the dependencies +required for the yum package automatically. The ultimate goal should +be to move all the packages from the **Unmanaged** entries section +to the **Managed** entries section. So, what exactly *are* those Unmanaged entries?:: output -Now you can go through these and continue adding the packages you -want to your Bundle. After a while, I ended up with a minimal bundle +Now you can go through these and continue adding the packages you +want to your Bundle. After a while, I ended up with a minimal bundle that looks like this .. code-block:: xml <Bundle name='base-packages'> - + </Bundle> Now when I run the client, you can see I have only one unmanaged @@ -319,8 +330,8 @@ package:: outout The gpg-pubkey packages are special in that they are not really -packages. Currently, the way to manage them is using -:ref:`BoundEntries <boundentries>`. So, after adding them, our +packages. Currently, the way to manage them is using +:ref:`BoundEntries <boundentries>`. So, after adding them, our Bundle now looks like this .. note:: This does not actually control the contents of the files, @@ -352,7 +363,7 @@ Bundle now looks like this To actually push the gpg keys out via Bcfg2, you will need to manage the files as well. This can be done by adding Path entries for each -of the gpg keys you want to manage +of the gpg keys you want to manage .. code-block:: xml @@ -452,12 +463,12 @@ Git .. _Git tutorial: http://www.kernel.org/pub/software/scm/git/docs/gittutorial.html Adding the :ref:`server-plugins-version-git` plugins can preserve -versioning information. The first step is to add *Git* to your +versioning information. The first step is to add *Git* to your plugin line:: plugins = Base,Bundler,Cfg,...,Git -For tracking the configuration files in the ``/var/lib/bcfg2`` +For tracking the configuration files in the ``/var/lib/bcfg2`` directory a git repository need to be established:: git init diff --git a/doc/client/index.txt b/doc/client/index.txt index 5e6551387..e232d8084 100644 --- a/doc/client/index.txt +++ b/doc/client/index.txt @@ -23,22 +23,11 @@ client, for a few reasons: deployment engine that can be driven by anything that writes a compatible configuration description. -Available client tools ----------------------- - -.. toctree:: - :maxdepth: 2 - :glob: - - tools/* - -Other client-related documentation ----------------------------------- - .. toctree:: :maxdepth: 2 + modes + tools + metadata agent debugging - metadata - modes diff --git a/doc/client/tools/index.txt b/doc/client/tools.txt index 9bec46316..676feb175 100644 --- a/doc/client/tools/index.txt +++ b/doc/client/tools.txt @@ -1,6 +1,6 @@ .. -*- mode: rst -*- -.. _client-tools-index: +.. _client-tools: Client Tool Drivers =================== @@ -26,6 +26,12 @@ the system. Currently these are the tool drivers that are distributed with Bcfg2: +.. toctree:: + :maxdepth: 2 + :glob: + + tools/* + Action ------ diff --git a/doc/help/faq/client.txt b/doc/help/faq/client.txt index 8eb04e620..636d318ad 100644 --- a/doc/help/faq/client.txt +++ b/doc/help/faq/client.txt @@ -14,3 +14,13 @@ on the client. Then add the following on the client.:: [communication] ca = /etc/bcfg2.ca + +.. FIXME: What is the solution for this? +.. **Server failure** +.. +.. On Fedora 14 and above it can happen that no connection is possible. +.. +.. # bcfg2 -vqne +.. Server failure: Protocol Error: 401 Unauthorized +.. Failed to download probes from bcfg2 +.. Server Failure diff --git a/doc/installation/distributions.txt b/doc/installation/distributions.txt index 38b34f14d..a4ce9fb5d 100644 --- a/doc/installation/distributions.txt +++ b/doc/installation/distributions.txt @@ -38,7 +38,7 @@ at `CustomDebianRepository`_. Fedora ====== -The fastest way to get Bcfg2 Packages_ onto your Fedora_ system is to +The fastest way to get Bcfg2 packages onto your Fedora_ system is to use `yum` or PackageKit. Yum will pull in all dependencies of Bcfg2 automatically. :: diff --git a/doc/server/admin/backup.txt b/doc/server/admin/backup.txt new file mode 100644 index 000000000..0c4758bda --- /dev/null +++ b/doc/server/admin/backup.txt @@ -0,0 +1,17 @@ +.. -*- mode: rst -*- + +.. _server-admin-backup-index: + +backup +====== + +.. _Samples repository: https://github.com/solj/bcfg2-repo + +Create an archive of the whole Bcfg2 :term:`repository`. The archive is stored +directly in your Bcfg2 repository (e.g. ``/var/lib/bcfg2/``) and named with +the current date and time:: + + bcfg2-admin backup + +A backup is recommanded before you start using the `Samples repository`_ of +Bcfg2. diff --git a/doc/server/admin/bundle.txt b/doc/server/admin/bundle.txt new file mode 100644 index 000000000..101666bb9 --- /dev/null +++ b/doc/server/admin/bundle.txt @@ -0,0 +1,30 @@ +.. -*- mode: rst -*- + +.. _server-admin-bundle: + +bundle +====== + +For a list of all available xml bundles use ``list-xml``. ``list-genshi`` +will list all available genshi bundles.:: + + # bcfg2-admin bundle list-xml + # bcfg2-admin bundle list-genshi + +``show`` provides an interactive dialog to get details about the available +bundles.:: + + # bcfg2-admin bundle show + Available bundles (Number of bundles: 4) + ---------------------------------------- + [0] motd.xml + [1] snmpd.xml + [2] bcfg2.xml + [3] ntp.xml + Enter the line number of a bundle for details: 3 + Details for the "ntp" bundle: + Package: xntp + Path: /etc/sysconfig/xntp + Path: /etc/sysconfig/clock + Path: /etc/ntp.conf + Service: xntpd diff --git a/doc/server/admin/client.txt b/doc/server/admin/client.txt new file mode 100644 index 000000000..02b46bda1 --- /dev/null +++ b/doc/server/admin/client.txt @@ -0,0 +1,31 @@ +.. -*- mode: rst -*- + +.. _server-admin-client: + +client +====== + +Create, delete, list, or modify client entries. :: + + bcfg2-admin client add <client> attr1=val1 attr2=val2 + +Allowed attributes are *profile*, *uuid*, *password*, *location*, *secure*, +and *address*. + +A full example is shown below:: + + bcfg2-admin client add laptop02.example.com profile="basic" + +For more details please refer to the +:ref:`Metadata section <server-plugins-grouping-metadata>`. + +With ``list`` the file ``clients.xml`` is parsed and all entries are shown:: + + bcfg2-admin client list + server01.example.com + laptop02.example.com + +This is useful for a quick check after adding an entry. If you want more +in-depth information about a client, ``bcfg2-info clients`` can provide +that. Please refer to the :ref:`bcfg2-info <server-bcfg2-info>` +section for further details. diff --git a/doc/server/admin/compare.txt b/doc/server/admin/compare.txt new file mode 100644 index 000000000..6a770055e --- /dev/null +++ b/doc/server/admin/compare.txt @@ -0,0 +1,16 @@ +.. -*- mode: rst -*- + +.. _server-admin-compare: + +compare +======= + +Determine differences between files or directories of client +specification instances.:: + + bcfg2-admin compare <file1> <file2> + +If you want to compare two directories recursively then use ``-r`` as an +option. :: + + bcfg2-admin compare -r <dir1> <dir2> diff --git a/doc/server/admin/index.txt b/doc/server/admin/index.txt index e850d9df0..ee03cedda 100644 --- a/doc/server/admin/index.txt +++ b/doc/server/admin/index.txt @@ -7,82 +7,24 @@ Admin ===== The ``bcfg2-admin`` command provides you an interface which allows you -to interact with your Bcfg2 repository in an administrative fashion. To +to interact with your Bcfg2 :term:`repository` in an administrative fashion. To get started, run ``bcfg2-admin help``. You will be presented with a list of different *modes* which each provide various administrative functionality. Available modes are listed below. -FIXME: Need examples for each command listed below. - -client -====== - -Create, delete, or modify client entries. - -compare -======= - -Determine differences between files or directories of client -specification instances. - -init -==== - -Interactively initialize a new repository. - -minestruct -========== - -Extract extra entry lists from statistics. - -perf -==== - -Query server for performance data. - -pull -==== - -Integrate configuration information from clients into the server -repository. - -query -===== - -Query clients. - -The default result format is suitable for consumption by `pdsh`_. -This example queries the server for all clients in the *ubuntu* group:: - - bcfg2-admin query g=ubuntu - -.. _pdsh: http://sourceforge.net/projects/pdsh/ - -snapshots -========= - -Interact with the Snapshots system. - -tidy -==== - -Clean up useless files in the repo. - -viz -=== - -Produce graphviz diagrams of metadata structures. - -The following command will produce a graphviz image which includes hosts, -bundles, and a key:: - - bcfg2-admin viz -H -b -k -o ~/bcfg2.png - -.. note:: The graphviz package available via DAG/RPMforge has been known - to have dependency issues. We recommend installing the package - from EPEL. - -xcmd -==== - -XML-RPC Command Interface. +.. toctree:: + :maxdepth: 1 + + backup + bundle + client + compare + init + minestruct + perf + pull + query + snapshots + tidy + viz + xcmd diff --git a/doc/server/admin/init.txt b/doc/server/admin/init.txt new file mode 100644 index 000000000..0e8b3afd3 --- /dev/null +++ b/doc/server/admin/init.txt @@ -0,0 +1,47 @@ +.. -*- mode: rst -*- + +.. _server-admin-init: + +init +==== + +Interactively initialize a new repository. Most values are automatically +detected or a default value is provided. :: + + bcfg2-admin init + Store bcfg2 configuration in [/etc/bcfg2.conf]: + Location of bcfg2 repository [/var/lib/bcfg2]: + Input password used for communication verification (without echoing; leave blank for a random): + What is the server's hostname [conf01.example.com]: + Input the server location [https://conf01.example.com:6789]: + Input base Operating System for clients: + 1: Red Hat/Fedora/RHEL/RHAS/Centos + 2: SUSE/SLES + 3: Mandrake + 4: Debian + 5: Ubuntu + 6: Gentoo + 7: FreeBSD + : 1 + + Generating a 2048 bit RSA private key + .....................+++ + .....................+++ + writing new private key to '/etc/bcfg2.key' + ----- + Signature ok + subject=/C=US/ST=Illinois/L=Argonne/CN=conf01.example.com + Getting Private key + +A toplevel repository structure was created under the provided path. :: + + /var/lib/bcfg2 + |-- Base + |-- Bundler + |-- Cfg + |-- etc + |-- Metadata + |-- Pkgmgr + |-- Rules + `-- SSHbase + diff --git a/doc/server/admin/minestruct.txt b/doc/server/admin/minestruct.txt new file mode 100644 index 000000000..d1de72426 --- /dev/null +++ b/doc/server/admin/minestruct.txt @@ -0,0 +1,14 @@ +.. -*- mode: rst -*- + +.. _server-admin-minestruct: + +minestruct +========== + +Extract extra entry lists from statistics.:: + + bcfg2-admin minestruct <client> [-f xml-file] [-g groups] + +Hierarchy of groups in which to place the extra entries in can be +determined with ``-g <groups>``. The ``-f <xml-output-file>`` option +specifies the xml file in which to write the extra entries. diff --git a/doc/server/admin/perf.txt b/doc/server/admin/perf.txt new file mode 100644 index 000000000..9a126ec71 --- /dev/null +++ b/doc/server/admin/perf.txt @@ -0,0 +1,18 @@ +.. -*- mode: rst -*- + +.. _server-admin-perf: + +perf +==== + +Query server for performance data.:: + + bcfg2-admin perf + ================ ========== ========== ========== ======= + Name Min Max Mean Count + ================ ========== ========== ========== ======= + RecvStats 0.000378 0.001716 0.001367 5 + GetConfig 0.018624 0.039495 0.023589 5 + component_lock 0.000002 0.000057 0.000016 20 + GetProbes 0.000523 0.000666 0.000591 5 + RecvProbeData 0.002260 0.004550 0.002979 diff --git a/doc/server/admin/pull.txt b/doc/server/admin/pull.txt new file mode 100644 index 000000000..e19485d53 --- /dev/null +++ b/doc/server/admin/pull.txt @@ -0,0 +1,24 @@ +.. -*- mode: rst -*- + +.. _server-admin-pull: + +pull +==== + +Integrate configuration information from clients into the server +repository. :: + + bcfg2-admin pull [-v] [-f][-I] [-s] <client> <entry type> <entry name> + +The following options are available: + + +``-v`` verbose +``-f`` force +``-I`` interactive +``-s`` stdin + +.. FIXME: No example yet +.. A full example is shown below. :: + +.. bcfg2-admin pull diff --git a/doc/server/admin/query.txt b/doc/server/admin/query.txt new file mode 100644 index 000000000..65851a43d --- /dev/null +++ b/doc/server/admin/query.txt @@ -0,0 +1,15 @@ +.. -*- mode: rst -*- + +.. _server-admin-query: + +query +===== + +Query clients. + +The default result format is suitable for consumption by `pdsh`_. +This example queries the server for all clients in the *ubuntu* group:: + + bcfg2-admin query g=ubuntu + +.. _pdsh: http://sourceforge.net/projects/pdsh/ diff --git a/doc/server/admin/snapshots.txt b/doc/server/admin/snapshots.txt new file mode 100644 index 000000000..25a7286c2 --- /dev/null +++ b/doc/server/admin/snapshots.txt @@ -0,0 +1,8 @@ +.. -*- mode: rst -*- + +.. _server-admin-snapshots: + +snapshots +========= + +Interact with the Snapshots system. diff --git a/doc/server/admin/tidy.txt b/doc/server/admin/tidy.txt new file mode 100644 index 000000000..816d6cdb3 --- /dev/null +++ b/doc/server/admin/tidy.txt @@ -0,0 +1,8 @@ +.. -*- mode: rst -*- + +.. _server-admin-tidy: + +tidy +==== + +Clean up useless files in the repo. diff --git a/doc/server/admin/viz.txt b/doc/server/admin/viz.txt new file mode 100644 index 000000000..1e7712ef9 --- /dev/null +++ b/doc/server/admin/viz.txt @@ -0,0 +1,18 @@ +.. -*- mode: rst -*- + +.. _server-admin-viz: + +viz +=== + +Produce graphviz diagrams of metadata structures. Make sure that the +graphviz package is installed. + +The following command will produce a graphviz image which includes hosts, +bundles, and a key:: + + bcfg2-admin viz -H -b -k -o ~/bcfg2.png + +.. note:: The graphviz package available via DAG/RPMforge has been known + to have dependency issues. We recommend installing the package + from EPEL. diff --git a/doc/server/admin/xcmd.txt b/doc/server/admin/xcmd.txt new file mode 100644 index 000000000..fc40f1243 --- /dev/null +++ b/doc/server/admin/xcmd.txt @@ -0,0 +1,27 @@ +.. -*- mode: rst -*- + +.. _server-admin-xcmd: + +xcmd +==== + +XML-RPC Command Interface.:: + + xcmd <xmlrpc method> <optional arguments> + +For debbuging the following command can help:: + + bcfg2-admin xcmd Metadata.toggle_debug + +Those two examples can alos be found in the +:ref:`Package section <server-plugins-generators-packages>`. + +To rebuild the packages plugin cache:: + + bcfg2-admin xcmd Packages.Refresh + +To perform a soft reload to reread the configuration file and +download only missing sources.:: + + bcfg2-admin xcmd Packages.Reload + diff --git a/doc/appendix/guides/using-bcfg2-info.txt b/doc/server/bcfg2-info.txt index b2354652f..35d176e96 100644 --- a/doc/appendix/guides/using-bcfg2-info.txt +++ b/doc/server/bcfg2-info.txt @@ -1,6 +1,6 @@ .. -*- mode: rst -*- -.. _appendix-guides-using_bcfg2_info: +.. _server-bcfg2-info: ================ Using bcfg2-info diff --git a/doc/server/index.txt b/doc/server/index.txt index 0de02c400..9c427a0f4 100644 --- a/doc/server/index.txt +++ b/doc/server/index.txt @@ -6,6 +6,19 @@ The Bcfg2 Server ================ +The Bcfg2 server is responsible for taking a comprehensive infrastructure +description and turning it into a series of configuration specifications +for particular clients. It also manages probed data and tracks statistics +for clients. + +The Bcfg2 server takes information from two sources when generating client +configuration specifications. The first is a pool of metadata that describes +clients as members of an aspect-based classing system. That is, clients are +defined in terms of aspects of their behavior. The other is a file system +repository that contains mappings from metadata to literal configuration. +These are combined to form the literal configuration specifications for +clients. + .. toctree:: :maxdepth: 2 @@ -14,3 +27,4 @@ The Bcfg2 Server configurationentries info snapshots/index + bcfg2-info diff --git a/doc/server/plugins/generators/tcheetah.txt b/doc/server/plugins/generators/tcheetah.txt index 185b25ef5..5819bba96 100644 --- a/doc/server/plugins/generators/tcheetah.txt +++ b/doc/server/plugins/generators/tcheetah.txt @@ -28,7 +28,7 @@ files, ``template`` and ``info``. The template is a standard Cheetah template with two additions: * `self.metadata` is the client's :ref:`metadata <server-plugins-grouping-metadata-clientmetadata>` -* `self.properties` is an xml document of unstructured data +* `self.metadata.Properties` is an xml document of unstructured data The ``info`` file is formatted like ``:info`` files from Cfg. @@ -47,8 +47,8 @@ self.metadata variables self.metadata is an instance of the class ClientMetadata and documented :ref:`here <server-plugins-grouping-metadata-clientmetadata>`. -self.properties -=============== +self.metadata.Properties +======================== Properties is a python `ElementTree <http://codespeak.net/lxml/>`_ object, loaded from the data in ``/var/lib/bcfg2/Properties/<properties @@ -70,13 +70,13 @@ You may use any of the ElementTree methods to access data in your template. Several examples follow, each producing an identical result on the host 'www.example.com':: - $self.Properties['example.xml'].find('host').find('www.example.com').find('rootdev').text - $self.Properties['example.xml'].find('host').find($self.metadata.hostname).find('rootdev').text - ${self.Properties['example.xml'].xpath('host/www.example.com/rootdev')[0].text} - ${self.Properties['example.xml'].xpath('host/' + self.metadata.hostname + '/rootdev')[0].text} + $self.metadata.Properties['example.xml'].data.find('host').find('www.example.com').find('rootdev').text + $self.metadata.Properties['example.xml'].data.find('host').find($self.metadata.hostname).find('rootdev').text + ${self.metadata.Properties['example.xml'].data.xpath('host/www.example.com/rootdev')[0].text} + ${self.metadata.Properties['example.xml'].data.xpath('host/' + self.metadata.hostname + '/rootdev')[0].text} #set $path = 'host/' + $self.metadata.hostname + '/rootdev' - ${self.Properties['example.xml'].xpath($path)[0].text} - ${self.Properties['example.xml'].xpath(path)[0].text} + ${self.metadata.Properties['example.xml'].data.xpath($path)[0].text} + ${self.metadata.Properties['example.xml'].data.xpath(path)[0].text} Simple Example ============== diff --git a/doc/server/plugins/grouping/metadata.txt b/doc/server/plugins/grouping/metadata.txt index fc452eb2f..071d4239d 100644 --- a/doc/server/plugins/grouping/metadata.txt +++ b/doc/server/plugins/grouping/metadata.txt @@ -10,8 +10,8 @@ The metadata mechanism has two types of information, client metadata and group metadata. The client metadata describes which top level group a client is associated with.The group metadata describes groups in terms of what bundles and other groups they include. Each aspect grouping -and clients' memberships are reflected in the Metadata/groups.xml and -Metadata/clients.xml files, respectively. +and clients' memberships are reflected in the ``Metadata/groups.xml`` and +``Metadata/clients.xml`` files, respectively. Usage of Groups in Metadata =========================== @@ -22,13 +22,13 @@ groups. Client membership of all other groups is by those groups being associated with the profile or public groups. This file can be indirectly modified from clients through use of the -p flag to bcfg2. -Clients are associated with profile groups in Metadata/clients.xml as +Clients are associated with profile groups in ``Metadata/clients.xml`` as shown below. Metadata/clients.xml ==================== -The Metadata/clients.xml file contains the mappings of Profile Groups +The ``Metadata/clients.xml`` file contains the mappings of Profile Groups to clients. The file is just a series of <Client /> tags, each of which describe one host. A sample file is below: @@ -58,7 +58,7 @@ The Clients tag has the following possible attributes: Client Tag ---------- -Each entry in clients.xml **must** have the following properties: +Each entry in ``clients.xml`` **must** have the following properties: +---------+---------------------------------------+--------+ | Name | Description | Values | diff --git a/doc/server/plugins/version/svn2.txt b/doc/server/plugins/version/svn2.txt new file mode 100644 index 000000000..773f39f8c --- /dev/null +++ b/doc/server/plugins/version/svn2.txt @@ -0,0 +1,38 @@ +.. -*- mode: rst -*- + +.. _server-plugins-version-svn2: + +==== +Svn2 +==== + +The Svn2 plugin is useful if you would like to track changes to your +bcfg2 repository using a `Subversion <http://subversion.tigris.org/>`_ +backend. It is a rewrite of the Svn plugin using the +`pysvn <http://pysvn.tigris.org/>`_ api. + +Currently, it enables you to get revision information out of your +repository for reporting purposes. Once the plugin is enabled, every time +a client checks in, it will include the current repository revision in +the reports/statistics. If any changes are made from the bcfg2-server, +the plugin will commit them back into the repositroy. For example, +``/usr/sbin/bcfg2-admin pull client Path /some/file`` will update the +servers repository and commit the changes into svn. + +How to enable the Svn2 plugin +============================= + +Simply add Svn2 to your plugins line in ``/etc/bcfg2.conf``:: + + [server] + plugins = Base,Bundler,Cfg,..,Svn2 + +Auto Commit feautre +=================== + +Svn2 adds the ability to commit changes back into the repository. In +order for this feature to work the repository should already be under version +control and the user the bcfg2-server runs as must be able to commit without +any interaction. For example, a file base repository url +(``file:///path/to/repo``). + diff --git a/man/bcfg2-admin.8 b/man/bcfg2-admin.8 index e2fdc9e0a..c0b8a09ce 100644 --- a/man/bcfg2-admin.8 +++ b/man/bcfg2-admin.8 @@ -16,164 +16,177 @@ Perform Bcfg2 repository administration .B \-C <config-file> .RS Specify the location of the configuration file (if it is not in -/etc/bcfg2.conf) +/etc/bcfg2.conf). .RE .SH MODES -.PP +.PP .B init .RS -Initialize a new repository (interactive) +Initialize a new repository (interactive). +.RE +.B backup +.RS +Create an archive of the whole Bcfg2 repository. +.RE +.B bundle <action> +.RS +Display details about the available bundles. .RE .B client <action> <client> [attribute=value] .RS -Add or remove clients from metadata +Add, edit, or remove clients entries in metadata. .RE .B query [g=group] [p=profile] [-f output-file] [-n] [-c] .RS -Search for clients based on group or profile +Search for clients based on group or profile. .RE .B compare <old> <new> .RS Compare two client configurations. Can be used to verify consistent -behavior between releases +behavior between releases. Determine differences between files or +directories. .RE .B minestruct <client> [-f xml-file] [-g groups] .RS -Build structure entries based on client statistics extra entries +Build structure entries based on client statistics extra entries. .RE .B pull <client> <entry-type> <entry-name> .RS Install configuration information into repo based on client bad -entries +entries. .RE .B snapshots [init|dump|query|reports] .RS -Interact with the Snapshots database +Interact with the Snapshots database. .RE .B tidy .RS -Remove unused files from repository +Remove unused files from repository. .RE .B viz [-H] [-b] [-k] [-o png-file] [-r] .RS -Create a graphviz diagram of client, group and bundle information +Create a graphviz diagram of client, group and bundle information. .RE -.B bundle <action> +.SH BUNDLE OPTIONS +.PP +.B mode .RS -Display details about the available bundles +List all available xml bundles 'list-xml' or for all available genshi +bundles 'list-genshi'. 'show' provides an interactive dialog to get +details about the available bundles. .RE .SH CLIENT OPTIONS .PP .B mode .RS -Add a client 'add' or delete a client 'del' +Add a client 'add', delete a client 'del', or 'list' all client entries. .RE .B client .RS -Specify the client's name +Specify the client's name. .RE .B attribute=value .RS -Set attribute values when adding a new client +Set attribute values when adding a new client. Allowed attributes +are 'profile', 'uuid', 'password', 'location', 'secure', and 'address'. .RE .SH QUERY OPTIONS .PP .B g=group .RS -Specify a group to search within +Specify a group to search within. .RE .B p=profile .RS -Specify a profile to search within +Specify a profile to search within. .RE .B \-f <output-file> .RS -Write the results of the query to a file +Write the results of the query to a file. .RE .B \-n .RS -Print the results, one on each line +Print the results, one on each line. .RE .B \-c .RS -Print the results, separated by commas +Print the results, separated by commas. .RE .SH COMPARE OPTIONS .PP .B old .RS -Specify the location of the old configuration file +Specify the location of the old configuration file. .RE .B new .RS -Specify the location of the new configuration file +Specify the location of the new configuration file. .RE .SH MINESTRUCT OPTIONS .PP .B client .RS -Client whose metadata is to be searched for extra entries +Client whose metadata is to be searched for extra entries. .RE .B \-g <groups> .RS -Hierarchy of groups in which to place the extra entries in +Hierarchy of groups in which to place the extra entries in. .RE .B \-f <xml-output-file> .RS -Specify the xml file in which to write the extra entries +Specify the xml file in which to write the extra entries. .RE .SH PULL OPTIONS .PP .B client .RS -Specify the name of the client to search for +Specify the name of the client to search for. .RE .B entry-type .RS -Specify the type of the entry to pull +Specify the type of the entry to pull. .RE .B entry-name .RS -Specify the name of the entry to pull +Specify the name of the entry to pull. .RE .SH SNAPSHOTS OPTIONS .PP .B init .RS -Initialize the snapshots database +Initialize the snapshots database. .RE .B query .RS -Query the snapshots database +Query the snapshots database. .RE .B dump .RS -Dump some of the contents of the snapshots database +Dump some of the contents of the snapshots database. .RE .B reports [-a] [-b] [-e] [--date=<MM-DD-YYYY>] .RS -Generate reports for clients in the snapshots database +Generate reports for clients in the snapshots database. .RE .SH VIZ OPTIONS .PP .B \-H .RS -Include hosts in diagram +Include hosts in diagram. .RE .B \-b .RS -Include bundles in diagram +Include bundles in diagram. .RE .B \-o <output file> .RS -Write to outfile file instead of stdout +Write to outfile file instead of stdout. .RE .B \-r .RS -Produce raw graphviz output +Produce raw graphviz output. .RE .B \-k .RS -Add a shape/color key +Add a shape/color key. .RE - diff --git a/misc/logos/favicon.ico b/misc/logos/favicon.ico Binary files differnew file mode 100644 index 000000000..c8d2e1e9e --- /dev/null +++ b/misc/logos/favicon.ico @@ -38,7 +38,7 @@ setup(cmdclass=cmdclass, glob('reports/xsl-transforms/*.xsl')), ('share/bcfg2/xsl-transforms/xsl-transform-includes', glob('reports/xsl-transforms/xsl-transform-includes/*.xsl')), - ('share/bcfg2', ['reports/reports.wsgi']), + ('share/bcfg2', glob('reports/reports.wsgi')), ('share/man/man1', glob("man/bcfg2.1")), ('share/man/man5', glob("man/*.5")), ('share/man/man8', glob("man/*.8")), diff --git a/src/lib/Server/Admin/Bundle.py b/src/lib/Server/Admin/Bundle.py index 41cd5727e..893fde489 100644 --- a/src/lib/Server/Admin/Bundle.py +++ b/src/lib/Server/Admin/Bundle.py @@ -46,7 +46,7 @@ class Bundle(Bcfg2.Server.Admin.MetadataCore): # except MetadataConsistencyError: # print "Error in deleting bundle." # raise SystemExit(1) - #Lists all available xml bundles + # Lists all available xml bundles elif args[0] in ['list-xml', 'ls-xml']: bundle_name = [] for bundle_path in xml_list: @@ -54,7 +54,7 @@ class Bundle(Bcfg2.Server.Admin.MetadataCore): bundle_name.append(rg.search(bundle_path).group(1)) for bundle in bundle_name: print bundle.split('.')[0] - #Lists all available genshi bundles + # Lists all available genshi bundles elif args[0] in ['list-genshi', 'ls-gen']: bundle_name = [] for bundle_path in genshi_list: @@ -62,9 +62,11 @@ class Bundle(Bcfg2.Server.Admin.MetadataCore): bundle_name.append(rg.search(bundle_path).group(1)) for bundle in bundle_name: print bundle.split('.')[0] - #Shows a list of all available bundles and prints bundle - #details after the user choose one bundle. - #FIXME: Add support for detailed output of genshi bundles + # Shows a list of all available bundles and prints bundle + # details after the user choose one bundle. + # FIXME: Add support for detailed output of genshi bundles + # FIXME: This functionality is almost identical with + # bcfg2-info bundles elif args[0] in ['show']: bundle_name = [] bundle_list = xml_list + genshi_list @@ -84,7 +86,7 @@ class Bundle(Bcfg2.Server.Admin.MetadataCore): else: if '%s/Bundler/%s' % \ (repo, bundle_name[int(lineno)]) in genshi_list: - print "Detailed output for *.genshi bundle is not supported." + print "Detailed output for *.genshi bundles is not supported." else: print 'Details for the "%s" bundle:' % \ (bundle_name[int(lineno)].split('.')[0]) diff --git a/src/lib/Server/Admin/Init.py b/src/lib/Server/Admin/Init.py index 384ba6929..c6d1f9e3d 100644 --- a/src/lib/Server/Admin/Init.py +++ b/src/lib/Server/Admin/Init.py @@ -66,7 +66,7 @@ clients = '''<Clients version="3.0"> # Mapping of operating system names to groups os_list = [ - ('Redhat/Fedora/RHEL/RHAS/Centos', 'redhat'), + ('Red Hat/Fedora/RHEL/RHAS/Centos', 'redhat'), ('SUSE/SLES', 'suse'), ('Mandrake', 'mandrake'), ('Debian', 'debian'), diff --git a/src/lib/Server/Admin/Pull.py b/src/lib/Server/Admin/Pull.py index aa732c67f..289852f79 100644 --- a/src/lib/Server/Admin/Pull.py +++ b/src/lib/Server/Admin/Pull.py @@ -135,4 +135,9 @@ class Pull(Bcfg2.Server.Admin.MetadataCore): except Bcfg2.Server.Plugin.PluginExecutionError: self.errExit("Configuration upload not supported by plugin %s" \ % (plugin.name)) - # FIXME svn commit if running under svn + # commit if running under a VCS + for vcsplugin in self.bcore.plugins.values(): + if isinstance(vcsplugin, Bcfg2.Server.Plugin.Version): + files = "%s/%s" % (plugin.data, ename) + comment = 'file "%s" pulled from host %s' % (files, client) + vcsplugin.commit_data([files], comment) diff --git a/src/lib/Server/Admin/Reports.py b/src/lib/Server/Admin/Reports.py index a4dd19064..a4dd19064 100755..100644 --- a/src/lib/Server/Admin/Reports.py +++ b/src/lib/Server/Admin/Reports.py diff --git a/src/lib/Server/Admin/__init__.py b/src/lib/Server/Admin/__init__.py index bb5c41895..a22d8521a 100644 --- a/src/lib/Server/Admin/__init__.py +++ b/src/lib/Server/Admin/__init__.py @@ -1,8 +1,23 @@ __revision__ = '$Revision$' -__all__ = ['Mode', 'Client', 'Compare', 'Init', 'Minestruct', 'Perf', - 'Pull', 'Query', 'Reports', 'Snapshots', 'Tidy', 'Viz', - 'Xcmd', 'Group', 'Backup'] +__all__ = [ + 'Backup', + 'Bundle', + 'Client', + 'Compare', + 'Group', + 'Init', + 'Minestruct', + 'Mode', + 'Perf', + 'Pull', + 'Query', + 'Reports', + 'Snapshots', + 'Tidy', + 'Viz', + 'Xcmd' + ] import ConfigParser import logging diff --git a/src/lib/Server/Plugin.py b/src/lib/Server/Plugin.py index 95569e3ac..186d1198a 100644 --- a/src/lib/Server/Plugin.py +++ b/src/lib/Server/Plugin.py @@ -293,7 +293,7 @@ class Version(object): """Interact with various version control systems.""" def get_revision(self): return [] - def commit_data(self): + def commit_data(self, file_list, comment=None): pass # the rest of the file contains classes for coherent file caching diff --git a/src/lib/Server/Plugins/Svn2.py b/src/lib/Server/Plugins/Svn2.py new file mode 100644 index 000000000..875e9e6a6 --- /dev/null +++ b/src/lib/Server/Plugins/Svn2.py @@ -0,0 +1,102 @@ +import os +try: + import pysvn + missing = False +except: + missing = True +import Bcfg2.Server.Plugin + +class Svn2(Bcfg2.Server.Plugin.Plugin, + Bcfg2.Server.Plugin.Version): + """Svn is a version plugin for dealing with Bcfg2 repos.""" + name = 'Svn2' + __version__ = '$Id$' + __author__ = 'bcfg-dev@mcs.anl.gov' + + conflicts = ['Svn'] + experimental = True + __rmi__ = Bcfg2.Server.Plugin.Plugin.__rmi__ + ['Update','Commit'] + + def __init__(self, core, datastore): + Bcfg2.Server.Plugin.Plugin.__init__(self, core, datastore) + + if missing: + self.logger.error("Svn2: Missing PySvn") + raise Bcfg2.Server.Plugin.PluginInitError + + self.client = pysvn.Client() + + self.core = core + self.datastore = datastore + self.svn_root = None + self.revision = None + + # Read revision from bcfg2 repo + revision = self.get_revision() + if not self.revision: + raise Bcfg2.Server.Plugin.PluginInitError + + self.logger.debug("Initialized svn plugin with svn root %s at revision %s" \ + % (self.svn_root, revision)) + + def get_revision(self): + """Read svn revision information for the Bcfg2 repository.""" + try: + info = self.client.info(self.datastore) + self.revision = info.revision + self.svn_root = info.url + return str(self.revision.number) + except: + self.logger.error("Svn2: Failed to get revision", exc_info=1) + self.revision = None + return str(-1) + + def commit_data(self, file_list, comment=None): + """Commit changes into the repository""" + if not comment: + comment = 'Svn2: autocommit' + + # First try to update + if not self.Update(): + self.logger.error("Failed to update svn repository, refusing to commit changes") + return + + #FIXME - look for conflicts? + + for file in file_list: + stat = self.client.status(file) + self.client.add([f.path for f in stat \ + if f.text_status == pysvn.wc_status_kind.unversioned]) + try: + self.revision = self.client.checkin([self.datastore], comment, + recurse=True) + self.revision = self.client.update(self.datastore, recurse=True)[0] + self.logger.info("Svn2: Commited changes. At %s" % self.revision.number) + except: + self.logger.error("Svn2: Failed to commit changes", exc_info=1) + + def Update(self): + '''Svn2.Update() => True|False\nUpdate svn working copy\n''' + try: + old_revision = self.revision.number + self.revision = self.client.update(self.datastore, recurse=True)[0] + except: + self.logger.error("Svn2: Failed to update server repository", exc_info=1) + return False + + if old_revision == self.revision.number: + self.logger.debug("repository is current") + else: + self.logger.info("Updated %s from revision %s to %s" % \ + (self.datastore, old_revision, self.revision.number)) + return True + + def Commit(self): + """Svn2.Commit() => True|False\nCommit svn repository\n""" + try: + self.commit_changes([]) + return True + except: + return False + + diff --git a/tools/README b/tools/README index 68591fd78..09aa49aeb 100644 --- a/tools/README +++ b/tools/README @@ -1,8 +1,4 @@ -This directory contains repository maintenance tools. +This directory contains repository maintenance tools. -bundlecvt.py <bundle directory> <translation directory> - - converts v1 bundles into (non-optimal bcfg2 bundles) -basebuilder.py <image directory> +basebuilder.py <image directory> - builds v2 base.xml from bcfg1 repo -validate_repo <repo directory> <schema directory> - - check all repo files against schemas
\ No newline at end of file diff --git a/tools/debian-build-cron.sh b/tools/debian-build-cron.sh deleted file mode 100644 index c03111412..000000000 --- a/tools/debian-build-cron.sh +++ /dev/null @@ -1,33 +0,0 @@ -#!/bin/sh -# rsync all the repo stuff to keep it up to date -# this pulls the debian repo from mirror.mcs.anl.gov if the currect sync is finished. -# If it isn't finished it will just back off and try again when the script runs next time. - -if ! /root/bin/rsync-debian-repo ; then - echo "repo not synced "; -fi - -# now rebuild the local repo so that it will grab everything that was added today. -if ! /root/bin/local-deb-repo-maker ; then - echo "local repos are not up to date "; -fi - -#make a back up of the current pkglist so we can diff later -cp /disks/bcfg2/Pkgmgr/debian-sarge.xml /tmp/debian-sarge.xml.current - -if /root/bin/create-debian-pkglist.pl ; then - diff /tmp/debian-sarge.xml.current /disks/bcfg2/Pkgmgr/debian-sarge.xml - rm -f /tmp/debian-sarge.xml.current - - #this is mainly cause this dies from time to time.. so this is protection - /etc/init.d/apt-proxy restart -else - echo "there was a problem with creating the new pkglist.xml"; - exit 1; -fi - -#this was problem-matic so I found that it was more realiable to just make its -#own cron job to restart every couple of days to reduce the memory consumption -#restart the server.. shouldn't be needed but memory leaks make it needed. -#/etc/init.d/bcfg2-server restart - diff --git a/tools/stable-check.py b/tools/stable-check.py deleted file mode 100755 index b7416456c..000000000 --- a/tools/stable-check.py +++ /dev/null @@ -1,40 +0,0 @@ -#!/usr/bin/env python - -import os, sys - -def do_merge(revision_string): - os.system("svnmerge merge -r %s" % revision_string) - os.system("svn commit -F svnmerge-commit-message.txt") - os.system("svn up") - -if __name__ == '__main__': - os.popen('svn up').read() - availrev = os.popen('svnmerge avail').read().strip() - if not availrev: - raise SystemExit, 0 - bf = [] - other = [] - for avail in availrev.split(','): - if '-' in avail: - start, stop = [int(x) for x in avail.split('-')] - else: - start = stop = int(avail) - - for rev in range(start, stop + 1): - log = os.popen("svn log https://svn.mcs.anl.gov/repos/bcfg/trunk/bcfg2 -r %s" % rev).read() - if "[bugfix]" in log: - bf.append(rev) - else: - other.append(rev) - if '-v' in sys.argv: - print log, - - mrevs = ','.join([str(x) for x in bf]) - if '-c' in sys.argv: - print "Revisions %s need merging" % (mrevs) - elif '-f' in sys.argv: - do_merge(mrevs) - else: - a = raw_input('Merge revisions %s: [yN] ' % mrevs) - if a in ['y', 'Y']: - do_merge(mrevs) |