diff options
author | Sol Jerome <solj@ices.utexas.edu> | 2010-02-01 02:28:39 +0000 |
---|---|---|
committer | Sol Jerome <solj@ices.utexas.edu> | 2010-02-01 02:28:39 +0000 |
commit | ad377a77ed88a8a4b0615dd2d2e984bef5b15d93 (patch) | |
tree | d14452a1ab4d77ffe1fb65cc989c4bc2d9acbff9 /doc/plugins/generators | |
parent | 31a9fbebcbcc0aafed741fa48b253163bcae2c69 (diff) | |
download | bcfg2-ad377a77ed88a8a4b0615dd2d2e984bef5b15d93.tar.gz bcfg2-ad377a77ed88a8a4b0615dd2d2e984bef5b15d93.tar.bz2 bcfg2-ad377a77ed88a8a4b0615dd2d2e984bef5b15d93.zip |
doc: Add server/reports sections
Signed-off-by: Sol Jerome <solj@ices.utexas.edu>
git-svn-id: https://svn.mcs.anl.gov/repos/bcfg/trunk/bcfg2@5709 ce84e21b-d406-0410-9b95-82705330c041
Diffstat (limited to 'doc/plugins/generators')
-rw-r--r-- | doc/plugins/generators/account.txt | 26 | ||||
-rw-r--r-- | doc/plugins/generators/cfg.txt | 105 | ||||
-rw-r--r-- | doc/plugins/generators/decisions.txt | 31 | ||||
-rw-r--r-- | doc/plugins/generators/deps.txt | 50 | ||||
-rw-r--r-- | doc/plugins/generators/hostbase.txt | 190 | ||||
-rw-r--r-- | doc/plugins/generators/nagiosgen.txt | 168 | ||||
-rw-r--r-- | doc/plugins/generators/packages.txt | 201 | ||||
-rw-r--r-- | doc/plugins/generators/pkgmgr.txt | 288 | ||||
-rw-r--r-- | doc/plugins/generators/rules.txt | 201 | ||||
-rw-r--r-- | doc/plugins/generators/sshbase.txt | 40 | ||||
-rw-r--r-- | doc/plugins/generators/tcheetah.txt | 163 | ||||
-rw-r--r-- | doc/plugins/generators/tgenshi.txt | 94 |
12 files changed, 0 insertions, 1557 deletions
diff --git a/doc/plugins/generators/account.txt b/doc/plugins/generators/account.txt deleted file mode 100644 index 9a37b2c69..000000000 --- a/doc/plugins/generators/account.txt +++ /dev/null @@ -1,26 +0,0 @@ -.. -*- mode: rst -*- - -.. _plugins-generators-account: - -======= -Account -======= - -The account plugin manages authentication data, including - -* /etc/passwd -* /etc/group -* /etc/security/limits.conf -* /etc/sudoers -* /root/.ssh/authorized_keys - -User access data is stored in three files in the Account directory: - -* superusers (a list of users who always have root privs) -* rootlist (a list of user:host pairs for scoped root privs) -* useraccess (a list of user:host pairs for login access) - -SSH keys are stored in files named $username.key; these are installed into root's authorized keys for users in the superusers list as well as for the pertitent users in the rootlike file (for the current system). - -Authentication data is read in from (static|dyn).(passwd|group) The static ones are for system local ones, while the dyn. versions are for external synchronization (from ldap/nis/etc) -There is also a static.limits.conf that provides the limits.conf header and any static entries. diff --git a/doc/plugins/generators/cfg.txt b/doc/plugins/generators/cfg.txt deleted file mode 100644 index 3870e8386..000000000 --- a/doc/plugins/generators/cfg.txt +++ /dev/null @@ -1,105 +0,0 @@ -.. -*- mode: rst -*- - -.. _plugins-generators-cfg: - -=== -Cfg -=== - -The Cfg plugin provides a repository to describe configuration file contents for clients. In its simplest form, the Cfg repository is just a directory tree modeled off of the directory tree on your client machines. - -The Cfg Repository -================== - -The Cfg plugin is enabled by including `Cfg` on the `plugins` line of the `[server]` section of your Bcfg2 server config file. The repository itself lives in `/var/lib/bcfg2/Cfg`, assuming you are using the default repository location of `/var/lib/bcfg2`. The contents of this directory are a series of directories corresponding to the real-life locations of the files on your clients, starting at the root level. For example:: - - lueningh@tg-prez:~/bcfg2/repository> ls Cfg - bin/ boot/ etc/ opt/ root/ usr/ var/ - -Specific config files go in like-named directories in this heirarchy. For example the password file, `/etc/passwd`, goes in `Cfg/etc/passwd/passwd`, while the ssh pam module config file, `/etc/pam.d/sshd`, goes in `Cfg/etc/pam.d/sshd/sshd`. The reason for the like-name directory is to allow multiple versions of each file to exist, as described below. Note that these files are exact copies of what will appear on the client machine - no templates, XML wrappers, etc. - -Group-Specific Files -==================== - -It is often that you want one version of a config file for all of your machines except those in a particular group. For example, `/etc/fstab` should look alike on all of your desktop machines, but should be different on your file servers. Bcfg can handle this case through use of group-specific files. - -As mentioned above, all Cfg entries live in like-named directories at the end of their directory tree. In the case of `fstab`, the file at `Cfg/etc/fstab/fstab` will be handed out by default to any client that asks for a copy of `/etc/fstab`. Group-specific files are located in the same directory and are named with the syntax:: - - /path/to/filename/filename.GNN_groupname - -in which ''NN'' is a priority number where '00' is lowest and '99' is highest, and ''groupname'' is the name of a group defined in `Metadata/groups.xml`. Back to our `fstab` example, we might have a `Cfg/etc/fstab/` directory that looks like:: - - fstab - fstab.G50_server - fstab.G99_fileserver - -By default, clients will receive the plain `fstab` file when they request `/etc/fstab`. Any machine that is in the `server` group, however, will instead receive the `fstab.G50_server` file. Finally, any machine that is in the `fileserver` group will receive the `fstab.G99_fileserver` file, even if they are also in the `server` group. - -Host-Specific Files -=================== - -Similar to the case with group-specific files, there are cases where a specific machine should have a different version of a file than all others. This can be accomplished with host-specific files. The format of a host-specific file name is:: - - /path/to/filename/filename.H_host.example.com - -Host-specific files have a higher priority than group specific files. Again, the `fstab` example:: - - fstab - fstab.G50_server - fstab.G99_fileserver - fstab.H_host.example.com - -In this case, `host.example.com` will always get the host-specific version, even if it is part of the `server` or `fileserver` (or both) classes. - -.. note:: If you have the ability to choose between using a group-specific file and a host-specific file, it is almost always best to use a group-specific one. That way if a hostname changes or an extra copy of a particular client is built, it will get the same changes as the original. - -Info files -========== - -By default, Cfg writes files to the filesystem with owner `root`, group `root`, and mode 644 (read and write for owner, read only for group and other). These options, and a few others, can be overridden through use of `:info` files. Each config file directory can have a `:info` file if needed. The possible fields in a `:info` file are: - -+-----------+-------------------+------------------------------------------------------+---------+ -| Field | Possible values | Description | Default | -+===========+===================+======================================================+=========+ -| owner: | Any valid user | Sets owner of the file | root | -+-----------+-------------------+------------------------------------------------------+---------+ -| group: | Any valid group | Sets group of the file | root | -+-----------+-------------------+------------------------------------------------------+---------+ -| perms: | Numeric file mode | Sets the permissions of the file | 0644 | -+-----------+-------------------+------------------------------------------------------+---------+ -| encoding: | ascii | base64 | Encoding of the file. Use base64 for non-ASCII files | ascii | -+-----------+-------------------+------------------------------------------------------+---------+ -| paranoid: | yes | no | Backup file before replacement? | no | -+-----------+-------------------+------------------------------------------------------+---------+ - -A sample `:info` file for CGI script on a web server might look like:: - - owner: www - group: www - perms: 0755 - -Back to the `fstab` example again, our final `Cfg/etc/fstab/` directory might look like:: - - :info - fstab - fstab.G50_server - fstab.G99_fileserver - fstab.H_host.example.com - -info.xml files -============== - -This feature is included in version 0.9.5pre3 and newer of the bcfg2 server. - -info.xml files add the ability to specify different sets of file metadata on a group by group basis. These files are XML, and work similarly to those used by [wiki:Plugins/Rules Rules] or [wiki:Plugins/Pkgmgr Pkgmgr]. - -The following specifies a different global set of permissions (root/sys/0651) than on clients in group webserver (root/root/0652) - -.. code-block:: xml - - <FileInfo> - <Group name='webserver'> - <Info owner='root' group='root' perms='0652'/> - </Group> - <Info owner='root' group='sys' perms='0651'/> - </FileInfo> diff --git a/doc/plugins/generators/decisions.txt b/doc/plugins/generators/decisions.txt deleted file mode 100644 index 67860b86c..000000000 --- a/doc/plugins/generators/decisions.txt +++ /dev/null @@ -1,31 +0,0 @@ -.. -*- mode: rst -*- - -.. _plugins-generators-decisions: - -========= -Decisions -========= - -This page describes the Decisions plugin. The client has support for a centralized set of per-entry installation decisions. This approach is needed when particular changes are deemed "high risk"; this gives the ability to centrally specify these changes, but only install them on clients when administrator supervision is available. Because collaborative configuration is one of the remaining hard issues in configuration management, these issues typically crop up in environments with several administrators and much configuration variety. - -In these cases, the client can be configured to run in either a whitelist or blacklist mode, wherein a list of entries is downloaded from the server. The client uses this list to determine which incorrect entries should be corrected during the current run of the installation tool. The Decision plugin is the only stock plugin that generates entries for client's whitelists or blacklists. - -The Decision plugin uses a directory in the bcfg2 repository called Decisions. Files in the Decisions subdirectory are named similarly to files managed by Cfg, probes, TCheetah, and TGenshi. File basenames are either whitelist or blacklist. These files have a simple format; the following is an example. - -.. code-block:: xml - - $ cat Decisions/whitelist - <Decisions> - <Decision type='Service' name='*'/> - <Decision type='Path' name='/etc/apt/apt.conf'/> - </Decisions> - -.. note:: To add syntax highlighting in vim, you can add a modeline such as this: - - <!-- - vim: ft=xml - --> - -This example, included as a whitelist due to its name, enables all services, and the path entry named /etc/apt/apt.conf to be installed on systems running in whitelist mode; all other entry installation will be surpressed. - -When a client askes for its whitelist or blacklist, all of the files pertaining to that client of the correct type are aggregated into a single list. This list is sent to the client. Note that this list is only generated when a client is run with the appropriate option (-l (whitelist|blacklist)); client behavior is not controlled unless this option is used. diff --git a/doc/plugins/generators/deps.txt b/doc/plugins/generators/deps.txt deleted file mode 100644 index 890fe0c39..000000000 --- a/doc/plugins/generators/deps.txt +++ /dev/null @@ -1,50 +0,0 @@ -.. -*- mode: rst -*- - -.. _plugins-generators-deps: - -==== -Deps -==== - -The Deps Plugin allows you to make a series of assertions like "Package X requires Package Y (and optionally also Package Z etc.) -Note that only configuration entries, like Package, !ConfigFile, etc can be used. Groupings (like Bundle) are not supported. - -Here are some examples: - -Deps/bcfg2.xml -============== - -.. code-block:: xml - - <Dependencies priority='0'> - <Package name='bcfg2'> - <Package name='python-lxml'/> - <Package name='isprelink'/> - </Package> - </Dependencies> - -This basically causes any configuration specification that includes Package bcfg2 to include python-lxml and isprelink, in a second base clause. - -Deps/bcfg2-server.xml -===================== - -.. code-block:: xml - - <Dependencies priority='0'> - <Package name='bcfg2-server'> - <Package name='python-cheetah'/> - <Package name='gamin-python'/> - <Package name='sqlite'/> - <Package name='python-sqlite'/> - <Package name='Django'/> - <Package name='mod_python'/> - <Package name='graphviz'/> - <Package name='xorg-x11-font-utils'/> - <Package name='chkfontpath'/> - <Package name='ttmkfdir'/> - <Package name='xorg-x11-xfs'/> - <Package name='urw-fonts'/> - </Package> - </Dependencies> - -This states that the bcfg2-server package (it's a separate package on some distros) depends on a long list of other packages. diff --git a/doc/plugins/generators/hostbase.txt b/doc/plugins/generators/hostbase.txt deleted file mode 100644 index 18bda9964..000000000 --- a/doc/plugins/generators/hostbase.txt +++ /dev/null @@ -1,190 +0,0 @@ -.. -*- mode: rst -*- - -.. _plugins-generators-hostbase: - -======== -Hostbase -======== - -IP management system built on top of Bcfg2. It has four main parts: a django data model, a web frontend, command-line utilities, and a Bcfg2 plugin that generates dhcp, dns, and yp configuration files. - -Installation -============ - -Overview --------- - -Installation of Hostbase requires installation of a python module, configuration of database (mysql or postgres), and configuration of an Apache webserver with mod_python. Hostbase was developed using MySQL, so this document is aimed at MySQL users. - -Prerequisites -------------- - - * mysql - * python-mysqldb - * `Django <http://www.djangoproject.com>`_ - -Install -------- - -'''Install Hostbase python module:''' - -As of Bcfg2 v0.8.7; the Hostbase module has been renamed Bcfg2.Server.Hostbase, and is automatically installed; there is nothing to do in this step. - -'''Configure the database''' - -Create the hostbase database and a user. For MySQL users:: - - mysql> CREATE DATABASE hostbase - mysql> quit - - systemprompt#: mysql -u root hostbase - mysql> GRANT ALL PRIVILEGES ON *.* TO hostbaseuser@mycomputer.private.net IDENTIFIED - BY 'password' WITH GRANT OPTION; - mysql> quit - -As of Bcfg2 v0.8.7 configuration options for Hostbase have moved to -``/etc/bcfg2.conf``. There is an example bcfg2.conf with Hostbase -options located at ``bcfg2-tarball/examples/bcfg2.confHostbase``. -Edit the hostbase options to correspond to the database you've -initialized and copy the configuration to ``/etc/bcfg2.conf``. To -finish creating the database, from your ``path to -python/Bcfg2/Server/Hostbase`` directory, run ``python manage.py -syncdb`` to do all table creation. - -'''Configure the web interface''' - -Now it's possible to explore the Hostbase web interface. For -curiosity, you can run Django's built-in development server to take a -peek. Do this by running ``python manage.py runserver -[servername:port]`` from your Hostbase directory. Django will -default to ``localhost:8000`` if no server or port is entered. Now -you can explore the web interface. Try adding a host and a zone. -You'll see that a ".rev" zone already exists. This is where -information for reverse files will go. - -For production, you'll want to have this configured for Apache with mod_python. Here is an example of how to configure Hostbase as a virtual host. - -.. code-block:: html - - <VirtualHost hostbase.mcs.anl.gov:80> - ServerAdmin systems@mcs.anl.gov - - DocumentRoot /var/www/hostbase/ - <Directory /> - AllowOverride None - </Directory> - - # Possible values include: debug, info, notice, warn, error, crit, - # alert, emerg. - LogLevel warn - - ServerSignature Off - - # Stop TRACE/TRACK vulnerability - <IfModule mod_rewrite.c> - RewriteEngine on - RewriteCond %{REQUEST_METHOD} ^(TRACE|TRACK) - RewriteRule .* - [F] - </IfModule> - - Redirect / https://hostbase.mcs.anl.gov/ - </VirtualHost> - - <VirtualHost hostbase.mcs.anl.gov:443> - ServerAdmin systems@mcs.anl.gov - - DocumentRoot /var/www/hostbase/ - <Directory /> - AllowOverride None - </Directory> - - # Possible values include: debug, info, notice, warn, error, crit, - # alert, emerg. - LogLevel warn - - ServerSignature Off - - # Stop TRACE/TRACK vulnerability - <IfModule mod_rewrite.c> - RewriteEngine on - RewriteCond %{REQUEST_METHOD} ^(TRACE|TRACK) - RewriteRule .* - [F] - </IfModule> - - SSLEngine On - SSLCertificateFile /etc/apache2/ssl/hostbase_server.crt - SSLCertificateKeyfile /etc/apache2/ssl/hostbase_server.key - - <Location "/"> - SetHandler python-program - PythonHandler django.core.handlers.modpython - SetEnv DJANGO_SETTINGS_MODULE Bcfg2.Server.Hostbase.settings - PythonDebug On - </Location> - <Location "/site_media/"> - SetHandler None - </Location> - </VirtualHost> - - -You'll need to copy the contents of ``Hostbase/media`` into ``/var/www/hostbase/site_media`` in this configuration to serve the correct css files. - -'''Enable the Hostbase plugin''' - -Now that the database is accessible and there is some data in it, you can enable the Hostbase plugin on your bcfg2 server to start generating some configuration files. All that needs to be done is to add ``Hostbase`` to the end of the list of generators in your bcfg2.conf file. To see what's being generated by Hostbase, fire up a bcfg2 development server: ``bcfg2-info``. For more information on how to use the bcfg2 development server, type help at the prompt. For our purposes, type ``debug``. This will bring you to an interactive python prompt where you can access bcfg's core data. - -.. code-block:: python - - for each in bcore.plugins['Hostbase'].filedata: - print each - - -The above loop will print out the name of each file that was generated by Hostbase. You can see the contents of any of these by typing ``print bcore.plugins['Hostbase'].filedata[filename]``. - -'''Create a bundle''' - -Bcfg2 needs a way to distribute the files generated by Hostbase. We'll do this with a bundle. In bcfg's ``Bundler`` directory, touch ``hostbase.xml``. - -.. code-block:: xml - - <Bundle name='hostbase' version='0.1'> - <Package name='dhcp3-server'/> - <Package name='bind9'/> - <Service name='dhcp3-server'/> - <Service name='bind9'/> - <ConfigFile name='/etc/dhcp3/dhcpd.conf'/> - <ConfigFile name='/etc/bind/[your domain]'/> - <ConfigFile name='/etc/bind/xxx.xxx.xxx.rev'/> - </Bundle> - -The above example is a bundle that will deliver both dhcp and dns files. This can be trivially split into separate bundles. It is planned that Hostbase will eventually be able to generate the list of ``ConfigFiles`` in its bundles automatically. - - -'''Do a Hostbase push''' - -You'll want to be able to trigger the Hostbase plugin to rebuild it's config files and push them out when data has been modified in the database. This can be done through and XMLRPC function available from the Bcfg2 server. From a client that is configured to receive one or more hostbase bundles, you'll need to first edit your ``python/site-packages/Bcfg2/Client/Proxy.py`` file. Add ``'Hostbase.rebuildState'`` to the list of methods in the bcfg2 client proxy object. The modified list is shown below: - -.. code-block:: python - - class bcfg2(ComponentProxy): - '''bcfg2 client code''' - name = 'bcfg2' - methods = ['AssertProfile', 'GetConfig', 'GetProbes', 'RecvProbeData', 'RecvStats', 'Hostbase.rebuildState'] - -Now copy the file ``hostbasepush.py`` from ``bcfg2/tools`` in the bcfg2 source to your machine. When this command is run as root, it triggers the Hostbase to rebuild it's files, then runs the bcfg2 client on your local machine to grab the new configs. - - -Authentication -============== - -Edit Django settings --------------------- - -Django allows for custom authentication backends to its login procedure. Hostbase has an NIS authentication backend that verifies a user to be in the unix group allowed to modify Hostbase. - -To enable this feature: - * first edit your ``Hostbase/settings.py`` file and uncomment the line ``'Hostbase.backends.NISBackend',`` in the list of ``AUTHENTICATION_BACKENDS`` - * enter the name of the unix group you want to give access to Hostbase in the ``AUTHORIZED_GROUP`` variable - * in your ``Hostbase/hostbase/views.py`` file at the very bottom, uncomment the block(s) of lines that give you the desired level of access - -Hostbase will now direct the user to a login page if he or she is not authorized to view a certain page. Users should log in with their regular Unix username and password. diff --git a/doc/plugins/generators/nagiosgen.txt b/doc/plugins/generators/nagiosgen.txt deleted file mode 100644 index 026109f21..000000000 --- a/doc/plugins/generators/nagiosgen.txt +++ /dev/null @@ -1,168 +0,0 @@ -.. -*- mode: rst -*- - -.. _plugins-generators-nagiosgen: - -========= -NagiosGen -========= - -This page describes the installation and use of the `NagiosGen -<http://trac.mcs.anl.gov/projects/bcfg2/browser/trunk/bcfg2/src/lib/Server/Plugins/NagiosGen.py>`_ -plugin. - -Update /etc/bcfg2.conf, adding NagiosGen to plugins:: - - plugins = SSHbase,Cfg,Pkgmgr,Rules,TCheetah,TWbase,NagiosGen - -Create the NagiosGen directory:: - - $ mkdir /var/lib/bcfg2/NagiosGen - -Create default host, and group specs in: - -* /var/lib/bcfg2/NagiosGen/default-host.cfg:: - - define host{ - name default - check_command check-host-alive - check_interval 5 - check_period 24x7 - contact_groups admins - event_handler_enabled 1 - failure_prediction_enabled 1 - flap_detection_enabled 1 - initial_state o - max_check_attempts 10 - notification_interval 0 - notification_options d,u,r - notification_period workhours - notifications_enabled 1 - process_perf_data 0 - register 0 - retain_nonstatus_information 1 - retain_status_information 1 - retry_interval 1 - } - -* /var/lib/bcfg2/NagiosGen/default-group.cfg:: - - define service{ - name default-service - active_checks_enabled 1 - passive_checks_enabled 1 - obsess_over_service 0 - check_freshness 0 - notifications_enabled 1 - event_handler_enabled 1 - flap_detection_enabled 1 - process_perf_data 0 - retain_status_information 1 - retain_nonstatus_information 1 - is_volatile 0 - - check_period 24x7 - max_check_attempts 4 - check_interval 5 - retry_interval 1 - contact_groups admins - notification_options w,u,c,r - notification_interval 0 - notification_period workhours - } - -Create group configuration files (Named identical to Bcfg2 groups) and -add services, and commands specific to the hostgroup (Bcfg2 group) in - -* /var/lib/bcfg2/NagiosGen/base-group.cfg:: - - define hostgroup{ - hostgroup_name base - alias base - notes Notes - } - - define service{ - service_description NTP - check_command check_ntp! - use default-service - hostgroup_name base - } - - define command{ - command_name check_ssh - command_line $USER1$/check_ssh $ARG1$ $HOSTADDRESS$ - } - - define service{ - service_description SSH - check_command check_ssh! - use default-service - hostgroup_name base - } - -* /var/lib/bcfg2/NagiosGen/web-server-group.cfg:: - - define hostgroup{ - hostgroup_name web-server - alias Port 80 Web Servers - notes UC/ANL Teragrid Web Servers Running on Port 80 - } - - define command{ - command_name check_http_80 - command_line $USER1$/check_http $HOSTADDRESS$ - } - - define service{ - service_description HTTP:80 - check_command check_http_80! - use default-service - hostgroup_name web-server - } - -Create a nagios bcfg2 bundle /var/lib/bcfg2/Bundler/nagios.xml - -.. code-block:: xml - - <Bundle name='nagios' version='2.0'> - <ConfigFile name='/etc/nagiosgen.status'/> - <Group name='rh'> - <Group name='nagios-server'> - <ConfigFile name='/etc/nagios/nagiosgen.cfg'/> - <Package name='libtool-libs'/> - <Package name='nagios'/> - <Package name='nagios-www'/> - <Service name='nagios'/> - </Group> - </Group> - <Group name='debian-lenny'> - <Group name='nagios-server'> - <ConfigFile name='/etc/nagios3/nagiosgen.cfg' - altsrc='/etc/nagios/nagiosgen.cfg'/> - <Package name='nagios3'/> - <Package name='nagios3-common'/> - <Package name='nagios3-doc'/> - <Package name='nagios-images'/> - <Service name='nagios3'/> - </Group> - </Group> - </Bundle> - -Assign clients to nagios groups in -/var/lib/bcfg2/Metadata/groups.xml - -.. code-block:: xml - - <Group name='nagios'> - <Bundle name='nagios'/> - </Group> - - <Group name='nagios-server'> - <Bundle name='nagios'/> - </Group> - -Update nagios configuration file to use nagiosgen.cfg:: - - cfg_file=/etc/nagios/nagiosgen.cfg - -Note that some of these files are built on demand, each time a client in group "nagios-server" checks in with the bcfg2 server. Local nagios instances can be configured to use the !NagiosGen directory in the bcfg2 repository directly. diff --git a/doc/plugins/generators/packages.txt b/doc/plugins/generators/packages.txt deleted file mode 100644 index 5efe5706d..000000000 --- a/doc/plugins/generators/packages.txt +++ /dev/null @@ -1,201 +0,0 @@ -.. -*- mode: rst -*- - -.. _plugins-generators-packages: - -======== -Packages -======== - -This page documents the Packages plugin. Packages is an alternative to [wiki:Plugins/Pkgmgr Pkgmgr] for specifying package entries for clients. Where Pkgmgr explicitly specifies package entry information, Packages delegates control of package version information to the underlying package manager, installing the latest version available through those channels. - -"Magic Groups" -============== - -Packages is the only plugin that uses "magic groups". Most plugins operate based on client group memberships, without any concern for the particular names chosen for groups by the user. The Packages plugin is the sole exception to this rule. Packages needs to "know" two different sorts of facts about clients. The first is the basic OS/distro of the client, enabling classes of sources. The second is the architecture of the client, enabling sources for a given architecture. In addition to these magic groups, each source may also specify a non-magic group to limit the source's applicability to group member clients. - -+-----------+----------+--------------+ -| Source | OS Group | Architecture | -+===========+==========+==============+ -| APTSource | debian | i386 | -+-----------+----------+--------------+ -| APTSource | ubuntu | amd64 | -+-----------+----------+--------------+ -| APTSource | nexenta | | -+-----------+----------+--------------+ -| YUMSource | redhat | i386 | -+-----------+----------+--------------+ -| YUMSource | centos | x86_64 | -+-----------+----------+--------------+ - -Limiting sources to groups -========================== - -Each source can also specify explicit group memberships. In the following example, the ubuntu-hardy group is also required. Finally, clients must be a member of the appropriate architecture group, in this case, either i386 or amd64. In total, in order for this source to be associated with a client is for the client to be in one of the sentinel groups (debian, ubuntu, or nexenta), the explicit group ubuntu-hardy, and any of the architecture groups (i386 or amd64). - -Memberships in architecture groups is needed so that Packages can map software sources to clients. There is no other way to handle this than to impose - -When multiple sources are specified, clients are associated with each source to which they apply (based on group memberships, as described above). Packages and dependencies are resolved from all applicable sources. - -Setup -===== -Three basic steps are required for Packages to work properly. - 1. Create Packages/config.xml. This file should look approximately like the example below, and describes both which software repositories should be used, and which clients are eligible to use each one. - 2. Ensure that clients are members of the proper groups. Each client should be a member of one of the sentinel groups listed above (debian/ubuntu/redhat/centos/nexenta), all of the groups listed in the source (like ubuntu-intrepid or centos-5.2 in the following examples), and one of the architecture groups listed in the source configuration (i386, amd64 or x86_64 in the following examples). '''Failure to do this will result in the source either not applying to the client, or only architecture independent packages being made available to the client.''' - 3. Add Package entries to bundles. - 4. Sit back and relax, as dependencies are resolved, and automatically added to client configurations. - -Prerequisite Resolution -======================= - -Packages provides a prerequisite resolution mechanism which has no analogue in Pkgmgr. During configuration generation, all structures are processed. After this phase, but before entry binding, a list of packages and the client metadata instance is passed into Packages' resolver. This process determines a superset of packages that will fully satisfy dependencies of all package entries included in structures, and reports any prerequisites that cannot be satisfied. This facility should largely remove the need to use the [wiki:Plugins/Base Base] plugin. - -Example usage -============= - -Create a config.xml file in the Packages directory that looks something like this: - -.. code-block:: xml - - <Sources> - <APTSource> - <Group>ubuntu-intrepid</Group> - <URL>http://us.archive.ubuntu.com/ubuntu</URL> - <Version>intrepid</Version> - <Component>main</Component> - <Component>universe</Component> - <Arch>i386</Arch> - <Arch>amd64</Arch> - </APTSource> - </Sources> - -Yum sources can be similarly specified: - -.. code-block:: xml - - <Sources> - <YUMSource> - <Group>centos-5.2</Group> - <URL>http://mirror.centos.org/centos/</URL> - <Version>5.2</Version> - <Component>os</Component> - <Component>updates</Component> - <Component>extras</Component> - <Arch>i386</Arch> - <Arch>x86_64</Arch> - </YUMSource> - </Sources> - -Configuration Updates -===================== - -Packages will reload its configuration upon an explicit command via bcfg2-admin.:: - - [0:3711] bcfg2-admin xcmd Packages.Refresh - True - -During this command (which will take some time depending on the quantity and size of the sources listed in the configuration file), the server will report information like:: - - Packages: Updating http://mirror.anl.gov/ubuntu//dists/jaunty/main/binary-i386/Packages.gz - Packages: Updating http://mirror.anl.gov/ubuntu//dists/jaunty/main/binary-amd64/Packages.gz - Packages: Updating http://mirror.anl.gov/ubuntu//dists/jaunty/universe/binary-i386/Packages.gz - Packages: Updating http://mirror.anl.gov/ubuntu//dists/jaunty/universe/binary-amd64/Packages.gz - ... - Packages: Updating http://mirror.centos.org/centos/5/extras/x86_64/repodata/filelists.xml.gz - Packages: Updating http://mirror.centos.org/centos/5/extras/x86_64/repodata/primary.xml.gz - -Once line per file download needed. Packages/config.xml will be reloaded at this time, so any source specification changes (new or modified sources in this file) will be reflected by the server at this point. - -Availability -============ - -Packages was added in 1.0pre1, with APT support. YUM support was added for 1.0pre2. Support for other package managers (Portage, Zypper, IPS, etc) remain to be added. - -Validation -========== - -A schema for Packages/config.xml is included; config.xml can be validated using bcfg2-repo-validate. Note that the schema requires that elements be specified in the above order. - -Limitations -=========== - -Packages does not do traditional caching as other plugins do. Changes to the Packages config file require a server restart for the time being. - -Package Verification -==================== - -In order to do disable per-package verification Pkgmgr style, you will need to use [wiki:BoundEntry BoundEntries] like below - -.. code-block:: xml - - <BoundPackage name="mem-agent" priority="1" version="auto" type="yum" verify="false"/> - -Debugging unexpected behavior -============================= - -Using bcfg2-info ----------------- - -The dependency resolver used in Packages can be run in debug mode:: - - - $ bcfg2-info - ... - Handled 20 events in 0.004s - > debug - dropping to python interpreter; press ^D to resume - ... - (debug_shell) - >>> m = self.build_metadata('ubik3') - >>> self.plugins['Packages'].complete(m, ['ssh'], debug=True) - Package ssh: adding new deps ['openssh-client', 'openssh-server'] - Package openssh-server: adding new deps ['libc6', 'libcomerr2', 'libkrb53', 'libpam0g', 'libselinux1', 'libssl0.9.8 - ', 'libwrap0', 'zlib1g', 'debconf', 'libpam-runtime', 'libpam-modules', 'adduser', 'dpkg', 'lsb-base'] - Package debconf: adding new deps ['debconf-i18n'] - Package libpam-modules: adding new deps ['libdb4.7'] - Package openssh-client: adding new deps ['libedit2', 'libncurses5', 'passwd'] - Package lsb-base: adding new deps ['sed', 'ncurses-bin'] - Package adduser: adding new deps ['perl-base'] - Package debconf-i18n: adding new deps ['liblocale-gettext-perl', 'libtext-iconv-perl', 'libtext-wrapi18n-perl', 'libtext-charwidth-perl'] - Package passwd: adding new deps ['debianutils'] - Package libtext-charwidth-perl: adding new deps ['perlapi-5.10.0'] - VPackage perlapi-5.10.0: got provides ['perl-base'] - Package libkrb53: adding new deps ['libkeyutils1'] - Package libtext-iconv-perl: adding new deps ['perlapi-5.10.0'] - Package libc6: adding new deps ['libgcc1', 'findutils'] - Package libgcc1: adding new deps ['gcc-4.3-base'] - (set(['debconf', 'libgcc1', 'lsb-base', 'libtext-wrapi18n-perl', 'libtext-iconv-perl', 'sed', 'passwd', 'findutils', 'libpam0g', 'openssh-client', 'debconf-i18n', 'libselinux1', 'zlib1g', 'adduser', 'libwrap0', 'ncurses-bin', 'libssl0.9.8', 'liblocale-gettext-perl', 'libkeyutils1', 'libpam-runtime', 'libpam-modules', 'openssh-server', 'libkrb53', 'ssh', 'libncurses5', 'libc6', 'libedit2', 'libcomerr2', 'dpkg', 'perl-base', 'libdb4.7', 'libtext-charwidth-perl', 'gcc-4.3-base', 'debianutils']), set([]), 'deb') - -This will show why the resolver is acting as it is. Replace "ubik3" and ['ssh'] with a client name and list of packages, respectively. Also, a more polished interface to this functionality is coming as well. - -Each line starting with Package: <name> describes a set of new prerequisites pulled in by a package. Lines starting with VPackage <vname> describe provides entries and their mappings to required names. The last line describes the overall results of the resolver, with three fields: a list of packages that should be installed, a list of unresolved requirements, and a type for these packages. - -Using bcfg2-server ------------------- - -Once the server is started, enable debugging via bcfg2-admin:: - - $ bcfg2-admin xcmd Packages.toggle_debug - -TODO list -========= - - * Zypper support - * Portage support - * Explicit version pinning (a la Pkgmgr) - -Developing for Packages -======================= - -In order to support a given client package tool driver, that driver must support use of the auto value for the version attribute in Package entries. In this case, the tool driver views the current state of available packages, and uses the underlying package manager's choice of correct package version in lieu of an explicit, centrally-specified, version. This support enables Packages to provide a list of Package entries with version='auto'. Currently, the APT and YUMng drivers support this feature. Note that package management systems without any network support cannot operate in this fashion, so RPMng and SYSV will never be able to use Packages. Emerge, Zypper, IPS, and Blastwave all have the needed features to be supported by Packages, but support has not yet been written. - -Packages fills two major functions in configuration generation. The first is to provide entry level binding support for Package entries included in client configurations. This function is quite easy to implement; Packages determines (based on client group membership) if the package is available for the client system, and which type it has. Because version='auto' is used, no version determination needs to be done. - -The second major function is more complex. Packages ensures that client configurations include all package-level prerequisites for package entries explicitly included in the configuration. In order to support this, Packages needs to directly process network data for package management systems (the network sources for apt or yum, for examples), process these files, and build data structures describing prerequisites and the providers of those functions/paths. To simplify implementations of this, there is a generic base class (Bcfg2.Server.Plugins.Packages.Source) that provides a framework for fetching network data via HTTP, processing those sources (with subclass defined methods for processing the specific format provided by the tool), a generic dependency resolution method, and a caching mechanism that greatly speeds up server/bcfg2-info startup. - -Each source type must define: - * a get_urls attribute (and associated urls property) that describes the URLS where to get data from. - * a read_files method that reads and processes the downloaded files - -Sources may define a get_provides method, if provides are complex. For example, provides in rpm can be either rpm names or file paths, so multiple data sources need to be multiplexed. - -The APT source in src/lib/Server/Plugins/Packages.py provides a relatively simple implementation of a source. diff --git a/doc/plugins/generators/pkgmgr.txt b/doc/plugins/generators/pkgmgr.txt deleted file mode 100644 index 89e67e5b7..000000000 --- a/doc/plugins/generators/pkgmgr.txt +++ /dev/null @@ -1,288 +0,0 @@ -.. -*- mode: rst -*- - -.. _plugins-generators-pkgmgr: - -====== -Pkgmgr -====== - -.. note:: - - See [wiki:ClientTools/RPMng#PackageTagNewStyleandAttributes - RPMng#PackageTagNewStyleandAttributes].''' The way of showing the - architecture of the RPM has changed. The new way is "arch". The - old way is "multiarch". '''This document needs to be updated and - include version of Bcfg2 where change took place.''' - -The Pkgmgr plugin resolves the Abstract Configuration Entity "Package" to a package specification that the client can use to detect, verify and install the specified package. - -For a package specification to be included in the Literal configuration the name attribute from an Abstract Package Tag (from Base or Bundler) must match the name attribute of a Package tag in Pkgmgr, along with the appropriate group associations of course. - -Each file in the Pkgmgr directory has a priority. This allows the same package to be served by multiple files. The priorities can be used to break ties in the case that multiple files serve data for the same package. - -Usage of Groups in Pkgmgr -========================= - -Groups are used by the Pkgmgr plugin, along with host metadata, for selecting the package entries to include in the clients literal configuration. They can be thought of as:: - - if client is a member of group1 then - assign to literal config - -Nested groups are conjunctive (logical and).:: - - if client is a member of group1 and group2 then - assign to literal config - -Group membership may be negated. - -Tag Attributes in Pkgmgr -======================== - -PackageList Tag ---------------- - -The PackageList Tag may have the following attributes: - -+-----------+-----------------------------------------------+----------------+ -| Name | Description | Values | -+===========+===============================================+================+ -| priority | Sets the priority for packages in the package | Integer | -| | list. The higher value wins. | | -+-----------+-----------------------------------------------+----------------+ -| type | Package type that applies to all packages in | deb|rpm|blast| | -| | the list. This value is inherited by all | encap|sysv| | -| | packages without an explicit type attribute. | portage|yum | -+-----------+-----------------------------------------------+----------------+ -| uri | URI to prepend to filename sto fetch packages | String | -| | in this list. | | -+-----------+-----------------------------------------------+----------------+ -| multiarch | Comma-separated list of architectures that | String | -| | apply to all packages in this list. Inherited | | -| | by all package entries in the file that does | | -| | not have this attribute explicitly. | | -+-----------+-----------------------------------------------+----------------+ -| srcs | To be used with multiarch support. Inherited | String | -| | by all Package entries without this attribute | | -+-----------+-----------------------------------------------+----------------+ - -Pkgmgr Group Tag ----------------- - -The Pkgmgr Group Tag may have the following attributes: - -+--------+----------------------------------------------+------------+ -| Name | Description | Values | -+========+==============================================+============+ -| name | Group Name | String | -+--------+----------------------------------------------+------------+ -| negate | Negate group membership (is not a member of) | True|False | -+--------+----------------------------------------------+------------+ - -Package Tag ------------ - -The Package Tag may have the following attributes: - -+------------+----------------------------------------------+------------+ -| Name | Description | Values | -+============+==============================================+============+ -| name | Package Name | String | -+------------+----------------------------------------------+------------+ -| version | Package version, set to ``auto`` to install | String | -| | the latest version in the client's cache, or | | -| | ``any`` to verify that any version of the | | -| | package is installed on the client | | -+------------+----------------------------------------------+------------+ -| file | Package file name. Several other attributes | String | -| | (name, version) can be automatically defined | | -| | based on regular expressions defined in the | | -| | Pkgmgr plugin. | | -+------------+----------------------------------------------+------------+ -| simplefile | Package file name. No name parsing is | String | -| | performed, so no extra fields get set | | -+------------+----------------------------------------------+------------+ -| verify | Whether package verification should be done | True|False | -+------------+----------------------------------------------+------------+ -| multiarch | Comma-separated list of the architectures of | String | -| | this package that should be installed. | | -| | (Temporary work-around) | | -+------------+----------------------------------------------+------------+ -| srcs | File name creation rules for multiarch | String | -| | packages. (Temporary work-around) | | -+------------+----------------------------------------------+------------+ -| type | Package type (rpm, yum, apt, sysv, blast) | String | -+------------+----------------------------------------------+------------+ - -Client Tag ----------- - -The Client Tag is used in a PackageList for selecting the package entries to include in the clients literal configuration. Its function is similar to the Group tag in this context. It can be thought of as:: - - if client is name then - assign to literal config - -The Client Tag may have the following attributes: - -+--------+----------------------------------------------+------------+ -| Name | Description | Values | -+========+==============================================+============+ -| name | Client Name | String | -+--------+----------------------------------------------+------------+ -| negate | Negate client selection (if not client name) | True|False | -+--------+----------------------------------------------+------------+ - -Pkgmgr Directory -================ - -The Pkgmgr/ directory keeps the XML files that define what packages are available for a host or image and where to find those packages. All the files in the directory are processed. - -The names of the XML files have no special meaning to Bcfg2; they are simply named so it's easy for the administrator to know what the contents hold. All Packages could be kept in a single file if so desired. Bcfg2 simply uses the Groups in the files and priorities to determine how to assign Packages to a host's literal configuration. - -Listed detailed below is one possible structure for the Pkgmgr directory. - -The files are structured to contain particular portions of distribution repositories. - -The files in the directory are:: - - $ ls Pkgmgr/ - centos-4-noarch-updates.xml - centos-4-x86_64-updates.xml - centos-4-x86_64.xml - backup.example.com.xml - fedora-core-4-noarch-updates.xml - fedora-core-4-x86-updates.xml - fedora-core-4-x86.xml - rhel-as-4-noarch-updates.xml - rhel-as-4-x86-updates.xml - rhel-as-4-x86.xml - rhel-es-4-noarch-updates.xml - rhel-es-4-x86-updates.xml - rhel-es-4-x86.xml - rhel-ws-4-noarch-udpates.xml - rhel-ws-4-x86_64-updates.xml - rhel-ws-4-x86_64.xml - rhel-ws-4-x86-updates.xml - rhel-ws-4-x86.xml - -As can be seen the file names have been selected to indicate what the contents are and have been split by Vendor, product and repository area. - -A partial listing of the centos-4-x86_64.xml is below - -.. code-block:: xml - - $ cat centos-4-x86_64.xml - <PackageList uri='http://www.example.com/yam/Centos44-x86_64/RPMS.os/' type='yum' priority='0'> - <Group name='Centos4.4-Standard'> - <Group name='x86_64'> - <Package name='audit-libs' version='1.0.13-1.EL4' type='yum'/> - <Package name='audit' version='1.0.13-1.EL4' type='yum'/> - <Package name='basesystem' version='8.0-2' type='yum'/> - <Package name='bash' version='3.0-18.1' type='yum'/> - <Package name='bcfg2' version='0.9.1-0.1' type='yum'/> - <Package name='beecrypt' version='3.1.0-3' type='yum'/> - ... - <Package name='VMwareTools' version='6530-29996' type='yum'/> - <Package name='yum' version='2.4.2-1' type='yum'/> - <Package name='zlib' version='1.2.1.2-1.2' type='yum'/> - </Group> - </Group> - </PackageList> - -.. code-block:: xml - - $ cat centos-4-x86_64-updates.xml - <PackageList uri='http://www.example.com/yam/Centos44-x86_64/RPMS.updates/' type='yum' priority='5'> - <Group name='Centos4.4-Standard'> - <Group name='x86_64'> - <Package name='audit-libs' version='1.0.14-1.EL4' type='yum'/> - <Package name='audit' version='1.0.14-1.EL4' type='yum'/> - <Package name='basesystem' version='8.0-4' type='yum'/> - <Package name='bash' version='3.0-19.3' type='yum'/> - <Package name='bcfg2' version='0.9.2-0.1' type='yum'/> - <Package name='beecrypt' version='3.1.0-6' type='yum'/> - ... - <Package name='VMwareTools' version='6530-29996' type='yum'/> - <Package name='yum' version='2.4.3-1' type='yum'/> - <Package name='zlib' version='1.2.1.2-1.2' type='yum'/> - </Group> - </Group> - </PackageList> - -Here it can be seen that the data is encapsulated in a !PackageList Tag which describes the URI of the files described, the type of package, and the priority of the files in this list. - -The priority is used to decide which specific file to use when there are multiple files that could be used for a particular host. The highest priority file is the one that is used. - -Using this system, it is possible to have a file that contains all the Packages from the original installation, centos-4-x86_64.xml in this case, and then create a new file that contains updates that are made available afterwards, centos-4-x86_64-updates.xml and centos-4-noarch-updates.xml in this case. The priority of the update PackageLists just needs to be higher so that they will be selected instead of the original installation Packages. - -The backup.example.com.xml contains a packalist for a specific host which is qualified by the Client tag. Its Packages have a higher priority than the update Packages. This is because this particular host requires special Packages that are older than the ones available in the updates. - -.. code-block:: xml - - <PackageList uri='http://www.example.com/yam/Centos44-x86_64/RPMS.os/' type='yum' priority='1000'> - <Client name='server86.example.com'> - <Package name='audit-libs' version='1.0.13-1.EL4' type='yum'/> - <Package name='audit' version='1.0.13-1.EL4' type='yum'/> - <Package name='basesystem' version='8.0-2' type='yum'/> - <Package name='bash' version='3.0-18.1' type='yum'/> - <Package name='bcfg2' version='0.9.1-0.1' type='yum'/> - <Package name='beecrypt' version='3.1.0-3' type='yum'/> - ... - <Package name='VMwareTools' version='6530-29996' type='yum'/> - <Package name='yum' version='2.4.2-1' type='yum'/> - <Package name='zlib' version='1.2.1.2-1.2' type='yum'/> - </Client> - </PackageList> - -Simplifying Multi-Architecture Environments with [wiki:altsrc Altsrc] -===================================================================== - -This feature is available in 0.9.5pre3 or newer versions of the bcfg2 server. - -Frequently multi-architecture environments (typically x86_64) will run into problems needing to specify different architectures on different groups for clients. For example, desktop machines may install 32-bit compatibility packages in addition to 64-bit ones, while servers may install only 64-bit packages. Specifying this in the Pkgmgr was onerous, because different package targets (64bit, 32+64, etc) needed to be specified on a package by group basis. Two features have been implemented that should ease this situation considerably. -* the [wiki:altsrc] feature adds the ability to add a "bind as" directive to entries. For example, the following entry, in a bundle: - -.. code-block:: xml - - <Package name='foo' altsrc='bar'/> - - would bind as if it were named bar, while the entry would still appear named "foo" in the client configuration specification. - -* Pkgmgr now builds virtual package targets for any package with Instance client elements. This means that if a client attempts to bind: - -.. code-block:: xml - - <Package name='libfoo:x86_64,i686'/> - It will only include the instances listed in the package. - By using these features together, a bundle can include: - -.. code-block:: xml - - <Package name='libfoo' altsrc='libfoo:i386,i686'/> - - This in conjunction with a Pkgmgr entry that looks like: - -.. code-block:: xml - - <Package name='libfoo'> - <Instance arch='i386' version='1.0.4-12'/> - <Instance arch='i586' version='1.0.4-12'/> - <Instance arch='i686' version='1.0.4-12'/> - <Instance arch='x86_64' version='1.0.4-12'/> - </Package> - - Will result in a bound entry that looks like: - -.. code-block:: xml - - <Package name='libfoo'> - <Instance arch='i386' version='1.0.4-12'/> - <Instance arch='i686' version='1.0.4-12'/> - </Package> - -Altogether, this should move policy decisions about package architectures to bundles/base. - -Client Driver (Plugins) Specific Attributes -=========================================== - -Not all the attributes that are available in Pkgmgr are honoured by all the client drivers. The following client drivers (plugins) have driver specific attributes: -* [wiki:RPMng RPMng/YUMng] diff --git a/doc/plugins/generators/rules.txt b/doc/plugins/generators/rules.txt deleted file mode 100644 index d05fef773..000000000 --- a/doc/plugins/generators/rules.txt +++ /dev/null @@ -1,201 +0,0 @@ -.. -*- mode: rst -*- - -.. _plugins-generators-rules: - -===== -Rules -===== - -The Rules plugin resolves the following Abstract Configuration Entities: - -* Service -* Directory -* SymLink -* Package -* Path -* Permissions -* Action - -to literal configuration entries suitable for the client drivers to consume. - -For an entity specification to be included in the Literal configuration the name attribute from an Abstract Entity Tag (from Base or Bundler) must match the name attribute of an Entity tag in Rules, along with the appropriate group associations of course. - -Each file in the Rules directory has a priority. This allows the same Entities to be served by multiple files. The priorities can be used to break ties in the case that multiple files serve data for the same Entity. - - -Usage of Groups in Rules -======================== - -Groups are used by the Rules plugin, along with host metadata, for selecting the Configuration Entity entries to include in the clients literal configuration. They can be thought of as:: - - if client is a member of group1 then - assign to literal config - -Nested groups are conjunctive (logical and).:: - - if client is a member of group1 and group2 then - assign to literal config - -Group membership may be negated. - -Tag Attributes in Rules -======================= - -Rules Tag ---------- - -The Rules Tag may have the following attributes: - -|| '' '''Name''' '' || '' '''Description''' '' || '' '''Values''' '' || -|| priority || Sets the priority for Rules in this Rules list.The higher value wins. || String || - -Rules Group Tag ---------------- - -The Rules Group Tag may have the following attributes: - -|| '' '''Name''' '' || '' '''Description''' '' || '' '''Values''' '' || -|| name || Group Name || String || -|| negate || Negate group membership (is not a member of) || (True|False) || - -Package Tag ------------ - -The Package Tag may have the following attributes: - -|| '' '''Name''' '' || '' '''Description''' '' || '' '''Values''' '' || -|| name || Package Name || String || -|| version || Package Version orversion='noverify' to not do version checking in the Yum driver only (temporary work a round). || String || -|| file || Package file name. Several other attributes (name, version) can be automatically defined based on regular expressions defined in the Pkgmgr plugin. || String || -|| simplefile || Package file name. No name parsing is performed, so no extra fields get set || String || -|| verify || verify='false' - do not do package verification || String || -|| reloc || RPM relocation path. || String || -|| multiarch || Comma separated list of the architectures of this package that should be installed. || String || -|| srcs || Filename creation rules for multiarch packages. || String || -|| type || Package type. (rpm, yum, apt,sysv,blast) || String || - -Permissions Tag ---------------- - -The Permissions tag is - -|| '' '''Name''' '' || '' '''Description''' '' || '' '''Values''' '' || -|| name || Name of the file. || String || -|| perms || Permissions of the file. || || -|| owner || Owner of the file. || || -|| group || Group of the file. || || - -Action Tag ----------- - -See [wiki:Plugins/Actions Actions] - -Service Tag ------------ - -The Service tag is - -|| '' '''Name''' '' || '' '''Description''' '' || '' '''Values''' '' || -|| name || Service Name || String || -|| status || Should the service be on or off. || (on|off) || -|| restart || Service command for restart (default:restart) || String || -|| type || Driver to use on the client to manage this service. || (chkconfig|deb|rc-update|smf) || -|| sequence || Order for service startup (debian services only) || integer || - -Directory Tag -------------- - -The Directory tag is - -|| '' '''Name''' '' || '' '''Description''' '' || '' '''Values''' '' || -|| name || Directory Name || String || -|| perms || Permissions of the directory. || String || -|| owner || Owner of the directory || String || -|| group || Group Owner of the directory || String || -|| prune || prune unspecified entries from the Directory || true|false || - -SymLink Tag ------------ - -The SymLink tag is - -|| '' '''Name''' '' || '' '''Description''' '' || '' '''Values''' '' || -|| name || Name of the symlink. || String || -|| to || File to link to || String || - -Client Tag ----------- - -The Client Tag is used in Rules for selecting the package entries to include in the clients literal configuration. Its function is similar to the Group tag in this context. It can be thought of as:: - - if client is name then - assign to literal config - -The Client Tag may have the following attributes: - -|| '' '''Name''' '' || '' '''Description''' '' || '' '''Values''' '' || -|| name || Client Name || String || -|| negate || Negate client selection (if not client name) || (True|False) || - -Path Tag --------- - -The Path tag is - -|| '' '''Name''' '' || '' '''Description''' '' || '' '''Values''' '' || -|| name || Name of the file || String || -|| type || Type of file || nonexistent || - - -Rules Directory -=============== - -The Rules/ directory keeps the XML files that define what rules are available for a host. All the files in the directory are processed. - -The names of the XML files have no special meaning to Bcfg2; they are simply named so it's easy for the administrator to know what the contents hold. All Rules could be kept in a single file if so desired. Bcfg2 simply uses the Groups in the files and priorities to determine how to assign Rules to a host's literal configuration. - -.. code-block:: xml - - <Rules priority="0"> - <Directory group="root" name="/autonfs" owner="root" perms="0755"/> - <Directory group="utmp" name="/var/run/screen" owner="root" perms="0775"/> - <Directory group="root" name="/autonfs/stage" owner="root" perms="0755"/> - <Directory group="root" name="/exports" owner="root" perms="0755"/> - <Directory name="/etc/condor" owner="root" group="root" perms="0755"/> - <Directory name="/logs" group="wwwtrans" owner="root" perms="0775"/> - <Directory name="/mnt" group="root" owner="root" perms="0755"/> - <Directory name="/my" owner="root" group="root" perms="0755"/> - <Directory name="/my/bin" owner="root" group="root" perms="0755"/> - <Directory name="/nfs" owner="root" group="root" perms="0755"/> - <Directory name="/sandbox" perms="0777" owner="root" group="root"/> - <Directory name="/software" group="root" owner="root" perms="0755"/> - <Permissions perms="0555" group="audio" owner="root" name="/dev/dsp"/> - <Permissions perms="0555" group="audio" owner="root" name="/dev/mixer"/> - <SymLink name="/bin/whatami" to="/mcs/adm/bin/whatami"/> - <SymLink name="/chibahomes" to="/nfs/chiba-homefarm"/> - <SymLink name="/home" to="/nfs/mcs-homefarm"/> - <SymLink name="/homes" to="/home"/> - <SymLink name="/mcs" to="/nfs/mcs"/> - <SymLink name="/my/bin/bash" to="/bin/bash"/> - <SymLink name="/my/bin/tcsh" to="/bin/tcsh"/> - <SymLink name="/my/bin/zsh" to="/bin/zsh"/> - <SymLink name="/software/common" to="/nfs/software-common"/> - <SymLink name="/software/linux" to="/nfs/software-linux"/> - <SymLink name="/software/linux-debian_sarge" to="/nfs/linux-debian_sarge"/> - <SymLink name="/usr/bin/passwd" to="/usr/bin/yppasswd"/> - <SymLink name="/usr/bin/yppasswd" to="/mcs/bin/passwd"/> - <SymLink name="/usr/lib/libgd.so.1.8" to="/usr/lib/libgd.so.1.8.4"/> - <SymLink name="/usr/lib/libtermcap.so.2" to="/usr/lib/libtermcap.so"/> - <SymLink name="/usr/local/bin/perl" to="/usr/bin/perl"/> - <SymLink name="/usr/local/bin/perl5" to="/usr/bin/perl"/> - <SymLink name="/usr/local/bin/tcsh" to="/bin/tcsh"/> - <Service name='ntpd' status='on' type='chkconfig'/> - <Service name='haldaemon' status='on' type='chkconfig'/> - <Service name='messagebus' status='on' type='chkconfig'/> - <Service name='netfs' status='on' type='chkconfig'/> - <Service name='network' status='on' type='chkconfig'/> - <Service name='rawdevices' status='on' type='chkconfig'/> - <Service name='sshd' status='on' type='chkconfig'/> - <Service name='syslog' status='on' type='chkconfig'/> - <Service name='vmware-tools' status='on' type='chkconfig'/> - </Rules> diff --git a/doc/plugins/generators/sshbase.txt b/doc/plugins/generators/sshbase.txt deleted file mode 100644 index 7ce066cc7..000000000 --- a/doc/plugins/generators/sshbase.txt +++ /dev/null @@ -1,40 +0,0 @@ -.. -*- mode: rst -*- - -.. _plugins-generators-sshbase: - -======= -SSHbase -======= - -SSHbase is a purpose build bcfg2 plugin for managing ssh host keys. It is responsible for making ssh keys persist beyond a client rebuild and building a proper ssh_known_hosts file, including a correct localhost record for the current system. - -It has two functions: - -* Generating new ssh keys -- When a client requests a dsa, rsa, or v1 key, and there is no existing key in the repository, one is generated. -* Maintaining the ssh_known_hosts file -- all current known public keys (and extra public key stores) are integrated into a single ssh_known_hosts file, and a localhost record for the current client is added. The ssh_known_hosts file data is updated whenever any keys change, are added, or deleted. - -Interacting with SSHbase -======================== - -* Pre-seeding with existing keys -- Currently existing keys will be overwritten by new, sshbase-managed ones by default. Pre-existing keys can be added to the repository by putting them in <repo>/SSHbase/<key filename>.H_<hostname> -* Pre-seeding can also be performed using bcfg2-admin pull ConfigFile /name/of/ssh/key -* Revoking existing keys -- deleting <repo>/SSHbase/\*.H_<hostname> will remove keys for an existing client. - -Aliases -======= - -SSHbase has support for Aliases listed in clients.xml. The address for the entries are specified either through DNS (e.g. a CNAME), or via the address attribute to the Alias. - -Getting started -=============== - -#. Add SSHbase to the generators line (plugins line in 1.0 or greater) in /etc/bcfg2.conf and restart the server -- This enables the SSHbase plugin in the bcfg2 server. -#. Add Path entries for /etc/ssh/ssh_known_hosts, and /etc/ssh/ssh_host_dsa_key, etc to a bundle or base. -#. Enjoy. - -At this point, SSHbase will generate new keys for any client without a recorded key in the repository, and will generate an ssh_known_hosts file appropriately. - -Blog post -========= - -http://www.ducea.com/2008/08/24/using-the-bcfg2-sshbase-plugin/ diff --git a/doc/plugins/generators/tcheetah.txt b/doc/plugins/generators/tcheetah.txt deleted file mode 100644 index 829e25901..000000000 --- a/doc/plugins/generators/tcheetah.txt +++ /dev/null @@ -1,163 +0,0 @@ -.. -*- mode: rst -*- - -.. _plugins-generators-tcheetah: - -======== -TCheetah -======== - -This document reflects the ``TCheetah`` plugin. - -The ``TCheetah`` plugin allows you to use the `cheetah templating system -<http://www.cheetahtemplate.org/>`_ to create files, instead of the -various diff-based methods offered by the ``Cfg`` plugin. It also allows -you to include the results of probes executed on the client in the -created files. - -To begin, you will need to download and install the Cheetah templating -engine from http://www.cheetahtemplate.org/. Once it is installed, -you can enable it by adding ``TCheetah`` to the ``plugins`` line in -``/etc/bcfg2.conf`` on your Bcfg server. For example:: - - generators = SSHbase,Cfg,Pkgmgr,Svcmgr,Rules,TCheetah - -The ``TCheetah`` plugin makes use of a ``Cfg``-like directory structure -located in in a ``TCheetah`` subdirectory of your repository, usually -``/var/lib/bcfg2/TCheetah``. Each file has a directory containing two -files, ``template`` and ``info``. The template is a standard Cheetah -template with two additions: - -* `self.metadata` is the client's metadata -* `self.properties` is an xml document of unstructured data - -The ``info`` file is formatted like ``:info`` files from Cfg. - -Mostly, people will want to use client metadata. - -self.metadata variables -======================= - -The following variables are available for self.metadata: - -* hostname -* bundles -* groups -* toolset -* categories -* probes -* uuid -* password - -self.metadata is an instance of the class ClientMetadata of file `Bcfg2/Server/Plugins/Metadata.py <http://trac.mcs.anl.gov/projects/bcfg2/browser/trunk/bcfg2/src/lib/Server/Plugins/Metadata.py>`_. - -self.properties -=============== - -properties is a python `ElementTree <http://codespeak.net/lxml/>`_ -object, loaded from the data in ``/var/lib/bcfg2/etc/properties.xml``. -That file should have a ``Properties`` node at its root. - -Example ``properties.xml``: - -.. code-block:: xml - - <Properties> - <host> - <www.example.com> - <rootdev>/dev/sda</rootdev> - </www.example.com> - </host> - </Properties> - -You may use any of the ElementTree methods to access data in your template. Several examples follow, each producing an identical result on the host 'www.example.com':: - - $self.properties.find('host').find('www.example.com').find('rootdev').text - $self.properties.find('host').find($self.metadata.hostname).find('rootdev').text - ${self.properties.xpath('host/www.example.com/rootdev')[0].text} - ${self.properties.xpath('host/' + self.metadata.hostname + '/rootdev')[0].text} - #set $path = 'host/' + $self.metadata.hostname + '/rootdev' - ${self.properties.xpath($path)[0].text} - ${self.properties.xpath(path)[0].text} - -Simple Example -============== - -bcfg2/TCheetah/foo/template ---------------------------- - -.. code-block:: none - - > buildfile /foo <clientname> - Hostname is $self.metadata.hostname - Groups: - #for $group in $self.metadata.groups: - * $group - #end for - Categories: - #for $category in $self.metadata.categories: - * $category -- $self.metadata.categories[$category] - #end for - - Probes: - #for $probe in $self.metadata.probes: - * $probe -- $self.metadata.probes[$probe] - #end for - -bcfg2/TCheetah/foo/info ------------------------ - -.. code-block:: none - - perms: 624 - -Output ------- - -The following output can be generated with bcfg2-info. Note that probe information is not persistent, hence, it only works when clients directly query the server. For this reason, bcfg2-info output doesn't reflect current client probe state. - -.. code-block:: xml - - <Path type="file" name="/foo" owner="root" perms="0624" group="root"> - Hostname is topaz.mcs.anl.gov - Groups: - * desktop - * mcs-base - * ypbound - * workstation - * xserver - * debian-sarge - * debian - * a - Categories: - * test -- a - - Probes: - </Path> - -Example: Replace the crontab plugin -=================================== - -In many cases you can use the TCheetah plugin to avoid writing custom plugins in Python. This example replaces the [source:tags/bcfg2_0_8_4/bcfg2/src/lib/Server/Plugins/Crontab.py crontab plugin] (Bcfg2.Server.Plugins.Crontab). This plugin randomizes the time of cron.daily execution with a stable result. Cron.daily is run at a consistent, randomized time between midnight and 7am.:: - - #import random - #silent random.seed($self.metadata.hostname) - - # /etc/crontab: system-wide crontab - # Unlike any other crontab you don't have to run the `crontab` - # command to install the new version when you edit this file. - # This file also has a username field, that none of the other crontabs do. - - SHELL=/bin/sh - PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin://bin - - # m h dom mon dow user command - 17 * * * * root run-parts --report /etc/cron.hourly - $random.randrange(0,59) $random.randrange(0,6) * * * root test -x /usr/sbin/anacron || run-parts --report /etc/cron.daily - 47 6 * * 7 root test -x /usr/sbin/anacron || run-parts --report /etc/cron.weekly - 52 6 1 * * root test -x /usr/sbin/anacron || run-parts --report /etc/cron.monthly. - -.. note:: Comments and Cheetah - As Cheetah processes your templates it will consider hash "#" style comments to be actual comments in the template and will strip them from the final config file. If you would like to preserve the comment in the final config file you need to escape the hash character '\#' which will tell Cheetah (and Python) that you do in fact want the comment to appear in the final config file.:: - - # This is a comment in my template which will be stripped when it's processed through Cheetah - \# This comment will appear in the generated config file. diff --git a/doc/plugins/generators/tgenshi.txt b/doc/plugins/generators/tgenshi.txt deleted file mode 100644 index bc0de8cbc..000000000 --- a/doc/plugins/generators/tgenshi.txt +++ /dev/null @@ -1,94 +0,0 @@ -.. -*- mode: rst -*- - -.. _plugins-generators-tgenshi: - -======= -TGenshi -======= - -This page documents the TGenshi plugin. This plugin works with version 0.4 and newer of the genshi library. - -The TGenshi plugin allows you to use the `Genshi -<http://genshi.edgewall.org>`_ templating system to create files, -instead of the various diff-based methods offered by the Cfg -plugin. It also allows you to include the results of probes executed -on the client in the created files. - -To begin, you will need to download and install the Genshi templating engine. - -To install on CentOS or RHEL 5, run:: - - sudo yum install python-genshi - -Once it is installed, you can enable it by adding ``TGenshi`` to the -generators line in ``/etc/bcfg2.conf`` on your Bcfg server. For example:: - - generators = SSHbase,Cfg,Pkgmgr,Svcmgr,Rules,TGenshi - -The TGenshi plugin makes use of a Cfg-like directory structure located in in a TGenshi subdirectory of your repository, usually ``/var/lib/bcfg2/TGenshi``. Each file has a directory containing two file types, template and info. Templates are named according to the genshi format used; template.txt uses the genshi text format, and template.xml uses the XML format. - -If used with Genshi 0.5 or later the plugin also supports the `new -style -<http://genshi.edgewall.org/wiki/Documentation/0.5.x/text-templates.html>`_ -text template format for files named template.newtxt. One of the -advantages of the new format is that it does not use # as a command -delimiter, making it easier to utilize for configuration files that -use # as a comment character. - -Only one template format may be used per file served. Info files are -identical to those used in ``Cfg``, and ``info.xml`` files are -supported. - -Inside of templates -=================== - -* metadata is the client's metadata -* properties.properties is an xml document of unstructured data - -See the genshi `documentation -<http://genshi.edgewall.org/wiki/Documentation>`_ for examples of -Genshi syntax. - -Examples -======== - -See [wiki:Plugins/TGenshi/examples] for some TGenshi template examples, including the great starting point [wiki:Plugins/TGenshi/examples/motd /etc/motd]. They will use the new style of Genshi syntax, unless noted otherwise. - -Examples: Old Genshi Syntax ---------------------------- - -Genshi's web pages recommend against using this syntax, as it may disappear from future releases. - -Group Negation --------------- - -Templates are also useful for cases where more sophisticated boolean operations than those supported by Cfg are needed. For example, the template:: - - #if "ypbound" in metadata.groups and "workstation" in metadata.groups - client is ypbound workstation - #end - #if "ubuntu" not in metadata.groups and "desktop" in metadata.groups - client is a desktop, but not an ubuntu desktop - #end - -Produces: - -.. code-block:: xml - - <Path type="file" name="/bar.conf" owner="root" perms="0644" group="root">client is ypbound workstation - client is a desktop, but not an ubuntu desktop - </Path> - -This flexibility provides the ability to build much more compact and succinct definitions of configuration contents than Cfg can. - -FAQs -==== - -'''Question:''' How do I escape the $ (dollar sign) in a TGenshi text template? For example, if I want to include SVN (subversion) keywords like $Id$ or $HeadURL$ in TGenshi-generated files, or am templating a bourne shell (sh/bash) script or Makefile (make). - -'''Answer:''' Use $$ (double dollar sign) to output a literal $ -(dollarsign) in a TGenshi text template. So instead of $Id$, you'd use -$$Id$$. See also Genshi tickets `#282: Document $$ escape convention -<http://genshi.edgewall.org/ticket/282>`_ and `#283: Allow for -redefinition of template syntax per-file -<http://genshi.edgewall.org/ticket/283>`_. |