diff options
| -rw-r--r-- | debian/buildsys/2.3/control.in | 1 | ||||
| -rw-r--r-- | debian/buildsys/2.4/control.in | 1 | ||||
| -rw-r--r-- | debian/buildsys/pycentral/control.in | 1 | ||||
| -rwxr-xr-x | debian/rules | 3 | ||||
| -rw-r--r-- | doc/Makefile | 13 | ||||
| -rw-r--r-- | doc/architecture.xml | 556 | ||||
| -rw-r--r-- | doc/deployment.xml | 117 | ||||
| -rw-r--r-- | doc/development.xml | 165 | ||||
| -rw-r--r-- | doc/images/composed.tif | bin | 811816 -> 0 bytes | |||
| -rw-r--r-- | doc/install.xml | 152 | ||||
| -rw-r--r-- | doc/manual.xml | 88 | ||||
| -rw-r--r-- | doc/reports.xml | 246 | ||||
| -rw-r--r-- | doc/specs.xml | 540 | ||||
| -rwxr-xr-x | tools/export.sh | 1 |
14 files changed, 0 insertions, 1884 deletions
diff --git a/debian/buildsys/2.3/control.in b/debian/buildsys/2.3/control.in index 36d394c42..f4581b7d5 100644 --- a/debian/buildsys/2.3/control.in +++ b/debian/buildsys/2.3/control.in @@ -3,7 +3,6 @@ Section: admin Priority: optional Maintainer: Sami Haahtinen <ressu@debian.org> Build-Depends: @cdbs@, python2.3-dev, python -Build-Depends-Indep: xsltproc, docbook-xsl Standards-Version: 3.7.2.0 Package: bcfg2 diff --git a/debian/buildsys/2.4/control.in b/debian/buildsys/2.4/control.in index 903a94e5f..3597623bf 100644 --- a/debian/buildsys/2.4/control.in +++ b/debian/buildsys/2.4/control.in @@ -3,7 +3,6 @@ Section: admin Priority: optional Maintainer: Sami Haahtinen <ressu@debian.org> Build-Depends: @cdbs@, python2.4-dev -Build-Depends-Indep: xsltproc, docbook-xsl Standards-Version: 3.7.2.0 Package: bcfg2 diff --git a/debian/buildsys/pycentral/control.in b/debian/buildsys/pycentral/control.in index cc97f9c1e..3655ea10a 100644 --- a/debian/buildsys/pycentral/control.in +++ b/debian/buildsys/pycentral/control.in @@ -3,7 +3,6 @@ Section: admin Priority: optional Maintainer: Sami Haahtinen <ressu@debian.org> Build-Depends: @cdbs@ -Build-Depends-Indep: xsltproc, docbook-xsl Standards-Version: 3.7.2.0 XS-Python-Version: >= 2.3 diff --git a/debian/rules b/debian/rules index f4a10a6fe..b8da8b62a 100755 --- a/debian/rules +++ b/debian/rules @@ -43,6 +43,3 @@ binary-install/bcfg2-server :: dh_installinit -p$(cdbs_curpkg) -o endif -# Build the manual page too -build/bcfg2-server:: - cd doc && make manual.html diff --git a/doc/Makefile b/doc/Makefile deleted file mode 100644 index 40932b19d..000000000 --- a/doc/Makefile +++ /dev/null @@ -1,13 +0,0 @@ -SOURCES = *.xml - -all : manual.pdf manual.html - -%.fo: %.xml $(SOURCES) - xsltproc --nonet --stringparam body.font.family Helvetica --stringparam fop.extensions 1 /usr/share/xml/docbook/stylesheet/nwalsh/fo/docbook.xsl $< > $@ - -%.html: %.xml $(SOURCES) - xsltproc --nonet -o $@ /usr/share/xml/docbook/stylesheet/nwalsh/xhtml/docbook.xsl $< - -%.pdf: %.fo - fop -fo $< -pdf $@ - diff --git a/doc/architecture.xml b/doc/architecture.xml deleted file mode 100644 index ef9d6bca8..000000000 --- a/doc/architecture.xml +++ /dev/null @@ -1,556 +0,0 @@ -<chapter id='chap:architecture'> -<title>Bcfg2 Architecture</title> - - <para> - Bcfg2 is based on a client-server architecture. The client is - reponsible for interpreting (but not processing) the configuration - served by the server. This configuration is literal, so no local - process is required. After completion of the configuration - process, the client uploads a set of statistics to the - server. This section will describe the goals and then the - architecture motivated by it. - </para> - - <section id='sec:goals'> - <title>Goals</title> - - <itemizedlist> - <listitem> - <para> - Model configurations using declarative - semantics. Declarative semantics maximize the utility of - configuration management tools; they provide the most - flexibility for the tool to determine the right course of - action in any given situation. This means that users can - focus on the task of describing the desired configuration, - while leaving the task of transitioning clients states to - the tool. - </para> - </listitem> - <listitem> - <para> - Configuration descriptions should be comprehensive. This - means that configurations served to the client should be - sufficent to reproduce all desired functionality. This - assumption allows the use of heuristics to detect extra - configuration, aiding in reliable, comprehensive - configuration definitions. - </para> - </listitem> - <listitem> - <para> - Provide a flexible approach to user interactions. Most - configuration management systems take a rigid approach to - user interactions; that is, either the client system is - always correct, or the central system is. This means that - users are forced into an overly proscribed model where the - system asserts where correct data is. Configuration data - modification is frequently undertaken on both the - configuration server and clients. Hence, the existance of a - single canonical data location can easily pose a problem - during normal tool use. - </para> - <para> - Bcfg2 takes a different approach. The default assumption is - that data on the server is correct, however, the client has - option to run in another mode where local changes are - catalogued for server-side integration. If the Bcfg2 client is - run in dry run mode, it can help to reconcile differences - between current client state and the configuration described - on the server. - </para> - <para> - The Bcfg2 client also searches for extra configuration; that - is, configuration that is not specified by the configuration - description. When extra configuration is found, either - configuration has been removed from the configuration - description on the server, or manual configuration has - occurred on the client. Options related to two-way - verification and removal are useful for configuration - reconciliation when interactive access is used. - </para> - </listitem> - <listitem> - <para> - Plugins and administrative applications. - </para> - </listitem> - <listitem> - <para> - Imcremental operations. - </para> - </listitem> - </itemizedlist> - </section> - - <section id='sec:client'> - <title>The Bcfg2 Client</title> - - <para>The Bcfg2 client performs all client configuration or - reconfiguration operations. It renders a declarative configuration - specification, provided by the Bcfg2 server, into a set of - configuration operations which will, if executed, attempt to - change the client's state into that described by the configuration - specification. Conceptually, the Bcfg2 client serves to isolate - the Bcfg2 server and specification from the imperative operations - required to implement configuration changes. This isolation allows - declarative specifications to be manipulated symbolically on the - server, without needing to understand the properties of the - underlying system tools. In this way, the Bcfg2 client acts as a - sort of expert system that "knows" how to implement declarative - configuration changes.</para> - - <para>The operation of the Bcfg2 client is intended to be as - simple as possible. The normal configuration process consists of - four main steps:</para> - -<!-- <procedure> --> -<!-- <title>Bcfg2 Client Execution Process</title> --> - -<!-- <step>foo</step> --> -<!-- </procedure> --> - - <orderedlist> - <listitem> - <para>Probe Execution</para> - <para>During the probe execution stage, the client connects to - the server and downloads a series of probes to execute. These - probes reveal local facts to the Bcfg2 server. For example, a - probe could discover the type of video card in a system. The - Bcfg2 client returns this data to the server, where it can - influence the client configuration generation - process. - </para> - </listitem> - <listitem> - <para>Configuration Download and Inventory</para> - <para> - The Bcfg2 client now downloads a configuration - specification from the Bcfg2 server. The configuration - describes the complete target state of the machine. That is, - all aspects of client configuration should be represented in - this specification. For example, all software packages and - services should be represented in the configuration - specification. - </para> - - <para> - The client now performs a local system inventory. This - process consists of verifying each entry present in the - configuration specification. After this check is completed, - heuristic checks for configuration not included in the - configuration specification. We refer to this inventory - process as 2-way validation, as first we verify that the - client contains all configuration that is included in the - specification, then we check if the client has any extra - configuration that isn't present. This provides a fairly - rigorous notion of client configuration congruence. - </para> - - <para> - Once the 2-way verification process has been - performed, the client has built a list of all configuration - entries that are out of spec. This list has two parts: - specified configuration that is incorrect (or missing) and - unspecified configuration that should be removed. - </para> - </listitem> - <listitem> - <para>Configuration Update</para> - <para> - The client now attempts to update its configuration to - match the specification. Depending on options, changes may - not (or only partially) be performed. First, if extra - configuration correction is enabled, extra configuration can - be removed. Then the remaining changes are processed. The - Bcfg2 client loops while progress is made in the correction - of these incorrect configuration entries. This loop results - in the client being able to accomplish all it will be able - to during one execution. Once all entries are fixed, or no - progress is being made, the loop terminates. - </para> - - <para> - Once all configuration changes that can be performed have - been, bundle dependencies are handled. Bundle groupings - result in two different behaviors. Contained entries are - assumed to be inter-dependant. To address this, the client - re-verifies each entry in any bundle containing an updates - configuration entry. Also, services contained in modified - bundles are restarted. - </para> - </listitem> - <listitem> - <para>Statistics Upload</para> - <para> - Once the reconfiguration process has concluded, the - client reports information back to the server about the - actions it performed during the reconfiguration - process. Statistics function as a detailed return code from - the client. The server stores statistics - information. Information included in this statistics update - includes (but is not limited to): - </para> - - <itemizedlist> - <listitem> - <para>Overall client status (clean/dirty)</para> - </listitem> - <listitem> - <para>List of modified configuration entries</para> - </listitem> - <listitem> - <para>List of uncorrectable configuration entries</para> - </listitem> - </itemizedlist> - </listitem> - </orderedlist> - - <section id='sec:arch-abstraction'> - <title>Architecture Abstraction</title> - - <para>The Bcfg2 client internally supports the administrative - tools available on different architectures. For example, rpm and - apt-get are both supported, allowing operation of Debian, - Redhat, SUSE, and Mandriva systems. The client toolset is - specified in the configuration specification. The client merely - includes a series of libraries which describe how to interact - with the system tools on a particular platform. - </para> - - <para> - Three of the libraries exist. There is a base set of - functions, which contain definitions describing how to perform - POSIX operations. Support for configuration files, - directories, and symlinks are included here. Two other - libraries subclass this one, providing support for Debian and - rpm-based systems. - </para> - - <para> - The Debian toolset includes support for apt-get and - update-rc.d. These tools provide the ability to install and - remove packages, and to install and remove services. - </para> - - <para> - The Redhat toolset includes support for rpm and chkconfig. Any - other platform that uses these tools can also use this - toolset. Hence, all of the other familiar rpm-based - distributions can use this toolset without issue. - </para> - - <para> - Other platforms can easily use the POSIX toolset, ignoring - support for packages or services. Alternatively, adding - support for new toolsets isn't difficult. Each toolset - consists of about 125 lines of python code. - </para> - </section> - </section> - <section id='sec:server'> - <title>The Bcfg2 Server</title> - <para> - The Bcfg2 server is responsible for taking a network description - and turning it into a series of configuration specifications for - particular clients. It also manages probed data and tracks - statistics for clients. - </para> - - <para> - The Bcfg2 server takes information from two sources when - generating client configuration specifications. The first is a - pool of metadata that describes clients as members of an - aspect-based classing system. That is, clients are defined in - terms of aspects of their behavior. The other is a file system - repository that contains mappings from metadata to literal - configuration. These are combined to form the literal - configuration specifications for clients. - </para> - - <section id='sec:construction'> - <title>The Configuration Specification Construction Process</title> - <para> - As we described in the previous section, the client connects - to the server to request a configuration specification. The - server uses the client's metadata and the file system - repository to build a specification that is tailored for the - client. This process consists of the following steps: - </para> - - <orderedlist> - <listitem> - <para>Metadata Lookup</para> - <para> - The server uses the client's IP address to initiate the - metadata lookup. This initial metadata consists of a - (profile, image) tuple. If the client already has - metadata registered, then it is used. If not, then - default values are used and stored for future use. - </para> - - <para> - This metadata tuple is expanded using some profile and - class definitions also included in the metadata. The end - result of this process is metadata consisting of - hostname, profile, image, a list of classes, a list of - attributes and a list of bundles. - </para> - </listitem> - <listitem> - <para>Abstract Configuration Construction</para> - <para> - Once the server has the client metadata, it is used to - create an abstract configuration. An abstract - configuration contains all of the configuration elements - that will exist in the final specification without any - specifics. All entries will be typed (ie the tagname - will be one of Package, ConfigurationFile, Service, - Symlink, or Directory) and will include a name. These - configuration entries are grouped into bundles, which - document installation time interdependencies. - </para> - </listitem> - <listitem> - <para>Configuration Binding</para> - <para> - The abstract configuration determines the structure of - the client configuration, however, it doesn't contain - literal configuration information. After the abstract - configuration is created, each configuration entry must - be bound to a client-specific value. The Bcfg2 server - uses plugins to provide these client-specific bindings. - </para> - - <para> - The Bcfg2 server core contains a dispatch table that - describes which plugins can handle requests of a - particular type. The responsible plugin is located - for each entry. It is called, passing in the - configuration entry and the client's metadata. The - behavior of plugins is explicitly undefined, so as to - allow maximum flexibility. The behaviors of the stock - plugins are documented elsewhere in this manual. - </para> - - <para> - Once this binding process is completed, the server has a - literal, client-specific configuration - specification. This specification is complete and - comprehensive; the client doesn't need to process it at - all in order to use it. It also represents the totality - of the configuration specified for the client. - </para> - </listitem> - </orderedlist> - </section> - </section> - - <section id='sec:literal'> - <title>The Literal Configuration Specification</title> - - <para> - Literal configuration specifications are served to clients by - the Bcfg2 server. This is a differentiating factor for Bcfg2; - all other major configuration management systems use a - non-literal configuration specification. That is, the clients - receive a symbolic configuration that they process to implement - target states. We took the literal approach for a few reasons: - </para> - - <itemizedlist> - <listitem> - <para> - A small list of configuration element types can be defined, - each of which can have a set of defined semantics. This - allows the server to have a well-formed model of - client-side operations. Without a static lexicon with - defined semantics, this isn't possible. This allows the - server, for example, to record the update of a package as a - coherent event. - </para> - </listitem> - <listitem> - <para> - Literal configurations do not require client-side - processing. Removing client-side processing reduces the - critical footprint of the tool. That is, the Bcfg2 client - (and the tools it calls) need to be functional, but the rest - of the system can be in any state. Yet, the client will - receive a correct configuration. - </para> - </listitem> - <listitem> - <para> - Having static, defined element semantics also requires that all - operations be defined and implemented in advance. The - implementation can maximize reliability and robustness. In - more ad-hoc setups, these operations aren't necessarily - safely implemented. - </para> - </listitem> - </itemizedlist> - - <section id='sec:spec-structure'> - <title>The Structure of Specifications</title> - <para> - Configuration specifications contain some number of - clauses. Two types of clauses exist. Bundles are groups of - inter-dependant configuration entities. The purpose of bundles - is to encode installation-time dependencies such that all new - configuration is properly activated during reconfiguration - operations. That is, if a daemon configuration file is - changed, its daemon should be restarted. Another example of - bundle usage is the reconfiguration of a software package. If - a package contains a default configuration file, but it gets - overwritten by an environment-specific one, then that updated - configuration file should survive package upgrade. The purpose - of bundles is to describe services, or reconfigured software - packages. Independent clauses contains groups of configuration - entities that aren't related in any way. This provides a - convenient mechanism that can be used for bulk installations - of software. - </para> - - <para> - Each of these clauses contains some number of configuration - entities. Five types of configuration entities exist: - ConfigurationFile, Package, SymLink, Directory, and - Service. Each of these correspond to the obvious system item. - Configuration specifications can get quite large; many systems - have specifications that top one megabyte in size. An example - of one is included in an appendix. These configurations can be - written by hand, or generated by the server. The easiest way - to start using Bcfg2 is to write small static configurations for - clients. Once configurations get larger, this process gets - unwieldy; at this point, using the server makes more sense. - </para> - </section> - </section> - - <section id='sec:design-considerations'> - <title>Design Considerations</title> - - <para> - This section will discuss several aspects of the design of - bcfg2, and the particular use cases that motivated - them. Initially, this will consist of a discussion of the system - metadata, and the intended usage model for package indices as - well. - </para> - - <section id='sec:metadata'> - <title>System Metadata</title> - - <para> - Bcfg2 system metadata describes the underlying patterns in - system configurations. It describes commonalities and - differences between these specifications in a rigorous way. The - groups used by bcfg2's metadata are responsible for - differentiating clients from one another, and building - collections of allocatable configuration. - </para> - - <para> - The Bcfg2 metadata system has been designed with several - high-level goals in mind. Flexibility and precision are - paramount concerns; no configuration should be undescribable - using the constructs present in the bcfg2 repository. We have - found (generally the hard way) that any assumptions about the - inherent simplicity of configuration patterns tend to be - wrong, so obscenely complex configurations must be - representable, even if these requirements seem illogical - during the implementation. - </para> - - <para> - In particular, we wanted to streamline several operations that - commonly occurred in our environment. - </para> - - <orderedlist> - <listitem> - <para>Copying one node's profile to another - node.</para> - - <para> - In many environments, many nodes are instances of a common - configuration specification. They all have similar roles - and software. In our environment, desktop machines were - the best example of this. Other than strictly per-host - configuration like SSH keys, all desktop machines use a - common configuration specification. This trivializes the - process of creating a new desktop machine. - </para> - </listitem> - <listitem> - <para>Creating a specialized version of a - currently existing profile. - </para> - - <para> - In environments with highly varied configurations, - departmental infrastructure being a good example, "another - machine like X but with extra software" is a common - requirement. For this reason, it must be trivially - possible to inherit most of a configuration specification - from some more generic source, while being able to - describe overriding aspects in a convenient fashion. - </para> - </listitem> - <listitem> - <para> - Compose several pre-existing configuration aspects to - create a new profile. - </para> - - <para> - The ability to compose configuration aspects allows the - easy creation of new profiles based on a series of - predefined set of configuration specification - fragments. The end result is more agility in environments - where change is the norm. - </para> - </listitem> - </orderedlist> - - <para> - In order for a classing system to be comprehensive, it must be - usable in complex ways. The Bcfg2 metadata system has - constructs that map cleanly to first-order logic. This implies - that any complex configuration pattern can be represented (at - all) by the metadata, as first-order logic is provably - comprehensive. (There is a discussion later in the document - describing the metadata system in detail, and showing how it - corresponds to first-order logic) - </para> - - <para> - These use cases motivate several of the design decisions that - we made: - </para> - - <orderedlist> - <listitem> - <para> - There must be a many to one correspondence between clients - and groups. Membership in a given profile group must imbue - a client with all of its configuration properties. - </para> - </listitem> - </orderedlist> - </section> - - <section id='sec:pkg-management'> - <title>Package Management</title> - - <para> - The interface provided in the bcfg2 repository for package - specification was designed with automation in mind. The goal - was to support an append only interface to the repository, so - that users do not need to continuously re-write already - existing bits of specification. - </para> - </section> - </section> -</chapter> diff --git a/doc/deployment.xml b/doc/deployment.xml deleted file mode 100644 index 4c4cc9bf6..000000000 --- a/doc/deployment.xml +++ /dev/null @@ -1,117 +0,0 @@ -<chapter> - <title>Deploying Bcfg2</title> - - <para> - Bcfg2 can be deployed in several different ways. The strategy - chosen varies based on the level of complexity accepted by the - administrators. The more literal a representation used, the less - powerful and reusable it is. We will describe three strategies for - Bcfg2 deployment, ranging from a cfengine-like deployment, to a - highly abstract configuration. While the abstract configuration is - much more powerful, the cfengine-like deployment is much easier to - understand and manipulate. - </para> - - <section> - <title>Simple Deployments</title> - - <para> - The Bcfg2 server will build configurations based on a set of - high-level specifications that use class-based abstractions to - provide reusability. This approach works pretty well; however, - it can be hard to deploy and may be too complicated to solve - simple problems. - </para> - - <para> - This issue can be addressed through the use of the Bcfg2 client - with a static configuration specification. This method works as - follows: important configuration details are statically - specified in a file on each system. The Bcfg2 client runs - periodically, and ensures that all aspects of configuration - included in the static specification are correct. It then - performs any update operations needed on the client. - </para> - - <para> - The format of static specifications is identical to that - provided by the Bcfg2 server, when it is used. It consists of a - series of "Bundle" and "Independent" clauses. Independent - clauses contain a series of configuration elements that can be - installed without any install time dependence on other - configuration elements. Bundles are series of dependent - clauses. This means that configuration elements may interfere - with one another, or that services may need to be restarted upon - configuration update. - </para> - - <para> - Each of these containers consists of a series of configuration - elements. The same elements may appear in either type of - clauses. These are basic types that are the same across all OS - ports. - </para> - - </section> - - <section> - <title>A Near-Literal Deployment</title> - <para> - The next easiest method to deploy is one where the configuration - specification is as simple and literal as possible. This style - of configuration specification can be characterized as near - copies of parts of the system. - </para> - - <para> - This style of deployment uses the stock generators: Cfg, Pkgmgr, - and Svcmgr. These manage configuration files, packages and - services, respectively. Copies of configuration files are placed - in the Cfg repository, in as generic a location as possible. - </para> - </section> - - <section> - <title>An Abstract Deployment</title> - <para/> - </section> - - <section> - <title>Bcfg2 Server Administration</title> - - <para/> - - </section> - - <section> - <title>An example application of bcfg2</title> - <para> - In my computing environment there are quite a diverse set of machines - and requirements for their operation. What this meant was that I needed - to devise a build system for machines that would allow me to easily - customize the software and services on the machine while still being - able to easily manage them and keep them secure. What I came up with - that solved this problem was that the initial install needed to be the - smallest subset of software that all machines had in common and - install this with whatever automated install system fit the OS. The - goal being that the OS automated installer( ie: kickstart, or - systemimager ) would put the initial bits on disk and take care of - hardware stuff and then as part of the postinstall process I run bcfg2 - to insure that the rest of the software and configuration occurs based - on the machines metadata. The overall goal was met. I could now build - any type of machine that I needed just by using the common buildsystem - and let bcfg2 determine what was different machine to machine. - </para> - - <para> - My current build process is centered around systemimager and - bcfg2. I have done some small enhancements to systemimager so that - with one floppy or cdrom any administrator can build any number of - machine profiles automatically. This is all done with some of the new - features that allow the encoding of the profile and image in the - clientside command so that the back end metadata can be asserted from - the client, which overrides the defaults specified in the metadata.xml - file. - </para> - </section> -</chapter> diff --git a/doc/development.xml b/doc/development.xml deleted file mode 100644 index b6e7b62ce..000000000 --- a/doc/development.xml +++ /dev/null @@ -1,165 +0,0 @@ -<chapter id='chap:development'> -<title>Developing for Bcfg2</title> - - <para> - While the Bcfg2 server provides a good interface for representing - general system configurations, its plugin interface offers the - ability to implement configuration interfaces and representation - tailored to problems encountered by a particular site. This - chapter describes what plugins are good for, what they can do, and - how to implement them. - </para> - - <section id='sec:plugins'> - <title>Bcfg2 Plugins</title> - - <para> - Bcfg2 plugins are loadable python modules that the Bcfg2 server - loads at initialization time. These plugins can contribute to - the functions already offered by the Bcfg2 server or can extend - its functionality. In general, plugins will provide some portion - of the configuration for clients, with a data representation - that is tuned for a set of common tasks. Much of the core - functionality of Bcfg2 is implemented by several plugins, - however, they are not special in any way; new plugins could - easily supplant one or all of them. - </para> - - <table id='table:plugin-functions'> - <title>Bcfg2 Plugin Functions</title> - <tgroup cols='2'> - <colspec colnum='1'/> - <colspec colnum='2'/> - <thead> - <row><entry>Name</entry><entry>Description</entry></row> - </thead> - <tbody> - <row><entry>Probes</entry><entry>Plugins can send executable - code to clients, where local contributions to configuration - state can be gathered.</entry></row> - <row><entry>Abstract Configuration Structures</entry> - <entry>A plugin can define new groups of interdependent - and independent configuration entities</entry></row> - <row><entry>Literal Configuration Entities</entry> - <entry>Plugins can provide literal configuration entity - information.</entry></row> - <row><entry>XML-RPC Functions</entry> - <entry>Plugins can expose a set of functions through the - Bcfg2 server's authenticated XML-RPC interface.</entry></row> - </tbody> - </tgroup> - </table> - - </section> - - <section id='sec:writing-plugins'> - <title>Writing Bcfg2 Plugins</title> - - <para> - Bcfg2 plugins are python classes that subclass from - Bcfg2.Server.Plugin.Plugin. Several plugin-specific values must - be set in the new plugin. These values dictate how the new - plugin will behave with respect to the above four functions. - </para> - - - <table id='table:plugin-members'> - <title>Bcfg2 Plugin Members</title> - <tgroup cols='3'> - <colspec colnum='1' colwidth='1*'/> - <colspec colnum='2' colwidth='3*'/> - <colspec colnum='3' colwidth='2*'/> - <thead> - <row><entry>Name</entry><entry>Description</entry><entry>Format</entry></row> - </thead> - <tbody> - <row><entry>__name__</entry><entry>The name of the - plugin</entry><entry>string</entry> - </row> - <row><entry>__version__</entry> - <entry>The plugin version (generally tied to revctl - keyword expansion).</entry><entry>string</entry></row> - <row><entry>__author__</entry> - <entry>The plugin author.</entry><entry>string</entry></row> - <row><entry>__rmi__</entry> - <entry>Set of functions to be exposed as XML-RPC - functions</entry> - <entry>List of function names (strings)</entry></row> - <row><entry>Entries</entry><entry>Multidimentional - dictionary of keys that point to the function used to bind - literal contents for a given configuration - entity.</entry><entry>Dictionary of - ConfigurationEntityType, Name keys and function reference - values</entry></row> - <row><entry>BuildStructures</entry><entry>Function that - returns a list of the structures for a given - client</entry><entry>Member function</entry></row> - <row><entry>GetProbes</entry><entry>Function that returns a - list of probes that a given client should - execute</entry><entry>Member function</entry></row> - <row><entry>ReceiveData</entry><entry>Function that accepts - the probe results for a given client.</entry><entry>Member - function</entry></row> - </tbody> - </tgroup> - </table> - - <section id='sec:example-plugin'> - <title>An Example Plugin</title> - - <example id='ex:simple-plugin'> - <title>A Simple Plugin</title> - <programlisting>import socket, Bcfg2.Server.Plugin - -class Chiba(Bcfg2.Server.Plugin.Plugin): - '''the Chiba plugin builds the following files: - -> /etc/network/interfaces''' - - __name__ = 'Chiba' - __version__ = '$Id: chiba.py 1702 2006-01-19 20:20:51Z desai ' - __author__ = 'bcfg-dev@mcs.anl.gov' - Entries = {'ConfigFile':{}} - - def __init__(self, core, datastore): - Bcfg2.Server.Plugin.Plugin.__init__(self, core, datastore) - self.repo = Bcfg2.Server.Plugin.DirectoryBacked(self.data, - self.core.fam) - self.Entries['ConfigFile']['/etc/network/interfaces'] \ - = self.build_interfaces - - def build_interfaces(self, entry, metadata): - '''build network configs for clients''' - entry.attrib['owner'] = 'root' - entry.attrib['group'] = 'root' - entry.attrib['perms'] = '0644' - try: - myriaddr = socket.gethostbyname("%s-myr" % \ - metadata.hostname) - except socket.gaierror: - self.LogError("Failed to resolve %s-myr"% metadata.hostname) - raise Bcfg2.Server.Plugin.PluginExecutionError, ("%s-myr" \ - % metadata.hostname, 'lookup') - entry.text = self.repo.entries['interfaces-template'].data % \ - myriaddr - </programlisting> - </example> - - <para> - Bcfg2 server plugins must subclass the - Bcfg2.Server.Plugin.Plugin class. Plugin constructors must - take two arguments: an instance of a Bcfg2.Core object, and a - location for a datastore. __name__, __version__, __author__, - and Entries are used to describe what the plugin is and how it - works. Entries describes a set of configuration entries that - can be provided by the generator, and a set of handlers that - can bind in the proper data. build_interfaces is an example of - a handler. It gets client metadata and an configuration entry - passed in, and binds data into entry as appropriate. This - results in a <filename>/etc/network/interfaces</filename> file - that has static information derived from DNS for a given host. - </para> - - </section> - - </section> -</chapter> diff --git a/doc/images/composed.tif b/doc/images/composed.tif Binary files differdeleted file mode 100644 index 0f79e3457..000000000 --- a/doc/images/composed.tif +++ /dev/null diff --git a/doc/install.xml b/doc/install.xml deleted file mode 100644 index 1113e6178..000000000 --- a/doc/install.xml +++ /dev/null @@ -1,152 +0,0 @@ -<chapter> - <title>Installing Bcfg2</title> - - <sect1> - <title>Pre-requisites</title> - <para> - Bcfg2 is written in python using several modules not included - with most distributions. lxml provides convenient xml - handling. - </para> - - <para> - The Bcfg2 server requires a few more packages. It uses either - FAM or Gamin to coherently cache repository files - and update them when they change. It also requires pyOpenSSL to - use SSL functions. - </para> - - <para>lxml is required for xml parsing. It can be downloaded from - http://www.codespeak.net/lxml. It, in turn, requires libxml2, - libxslt, and pyrex. - </para> - - <para> - The python fam binding can be downloaded from - python-fam.sourceforge.net. FAM (on several linux distributions) - has been depricated in favor of gamin. The Bcfg server will - autodetect which modules are available, and use appropriate file - caching logic. It can be installed by running the setup.py script. - </para> - - <table> - <title>Bcfg2 Software Prerequisites</title> - <tgroup cols='3'> - <colspec colnum='1' colwidth='2*'/> - <colspec colnum='2' colwidth='5*'/> - <colspec colnum='3' colwidth='8*'/> - <thead> - <row><entry>Name</entry><entry>Description</entry><entry>URL</entry></row> - </thead> - <tbody> - <row><entry>lxml</entry><entry>XML - Processing</entry><entry><ulink - url="http://codespeak.net/lxml"/></entry></row> - <row><entry>pyrex</entry><entry>C to Python language - interoperability (needed for lxml)</entry><entry><ulink - url="http://www.cosc.canterbury.ac.nz/~greg/python/Pyrex"/></entry></row> - <row><entry>pyOpenSSL</entry> - <entry>OpenSSL bindings for Python</entry><entry><ulink - url="http://pyopenssl.sourceforge.net/"/></entry></row> - <row><entry>Fam</entry><entry>File Alteration - Monitor</entry><entry><ulink - url="http://oss.sgi.com"/></entry></row> - <row><entry>Gamin</entry><entry>Alternate File Alteration - Monitor</entry><entry><ulink - url="http://www.gnome.org/~veillard/gamin/"/></entry></row> - <row><entry>Python-fam</entry><entry>Python bindings for fam - (not needed with - gamin)</entry><entry><ulink url="http://python-fam.sourceforge.net"/></entry></row> - </tbody> - </tgroup> - </table> - - </sect1> - <sect1> - <title>Bcfg2 Initial Setup and Testing</title> - <para>Once the Bcfg2 software is installed, the configuration file - and repository must be created. The example configuration file in - <filename>bcfg2/examples/bcfg2.conf</filename> can be used, with - minor modifications. This should be placed in - <filename>/etc/bcfg2.conf</filename>. If it is placed in another - location, each program takes a command line argument to specify - its alternate location. - </para> - - <example> - <title>/etc/bcfg2.conf</title> - <programlisting>[server] -repository = /disks/bcfg2 -structures = Bundler,Base -generators = SSHbase,Cfg,Pkgmgr,Svcmgr</programlisting> - </example> - - <para> - This configuration file sets the top level location of the - configuration repository. It also activates two structures, and - four generators. Both structures and generators are instances of - Bcfg2 server plugins. Structures generate abstract configuration - fragments. These form the inventory of the - configuration. Generators provide client-specific literal values - for each configuration entity contained in the abstract - configuration. - </para> - </sect1> - - <sect1> - <title>Daemon Configuration</title> - - <para> - Bcfg2 uses XML-RPC over HTTPS for all communications. - All communications occur over this transport. HTTPS provides - data security, while an embedded username and password provide - authentication. - </para> - - <sect2> - <title>SSL Certificate Generation</title> - - <para>SSL is used for channel-level data encryption. The - requisite SSL certificates must be generated on the server - side. The following command will generate a server key: - </para> - - <programlisting> -openssl req -x509 -nodes -days 1000 -newkey rsa:1024 \ - -out bcfg2.key -keyout bcfg2.key - </programlisting> - - <para> - This command will generate an SSL key including both an - RSA key and a certificate. This is suitable for use with the - Bcfg2 server. The path to this key should be put in the - bcfg2 configuration file in section communication, setting - key. - </para> - - </sect2> - - <sect2> - <title>Client Communication Setup</title> - - <para> - The Bcfg2 client must be able to find the server's - location. This is accomplished through the use of the - communication settings in <filename>/etc/bcfg2.conf</filename> - Several settings must be included in this file: the server - url, a username and a password. - </para> - - <example> - <title>/etc/bcfg2.conf</title> - <programlisting>[communication] -protocol = xmlrpc/ssl -password = pwd -user = root - -[components] -bcfg2 = https://bcfg2server:8765</programlisting> - </example> - </sect2> - </sect1> -</chapter>
\ No newline at end of file diff --git a/doc/manual.xml b/doc/manual.xml deleted file mode 100644 index 3213d27d5..000000000 --- a/doc/manual.xml +++ /dev/null @@ -1,88 +0,0 @@ -<!DOCTYPE book PUBLIC "-//OASIS//DTD DocBook V4.4//EN" -"http://www.oasis-open.org/docbook/xml/4.4/docbookx.dtd" -[ -<!ENTITY architecture SYSTEM "architecture.xml"> -<!ENTITY install SYSTEM "install.xml"> -<!ENTITY specification SYSTEM "specs.xml"> -<!ENTITY deployment SYSTEM "deployment.xml"> -<!ENTITY development SYSTEM "development.xml"> -<!ENTITY reports SYSTEM "reports.xml"> -]> -<book id="bcfg2"> - <bookinfo> - <title>Bcfg2 Manual</title> - <authorgroup> - <author> - <firstname>Narayan</firstname> - <surname>Desai</surname> - <email>desai@mcs.anl.gov</email> - <affiliation> - <orgname>Argonne National Laboratory</orgname> - <orgdiv>MCS Division</orgdiv> - </affiliation> - </author> - <author> - <firstname>Rick</firstname> - <surname>Bradshaw</surname> - <email>bradshaw@mcs.anl.gov</email> - <affiliation> - <orgname>Argonne National Laboratory</orgname> - <orgdiv>MCS Division</orgdiv> - </affiliation> - </author> - <author> - <firstname>Joey</firstname> - <surname>Hagedorn</surname> - <email>hagedorn@mcs.anl.gov</email> - <affiliation> - <orgname>Argonne National Laboratory</orgname> - <orgdiv>MCS Division</orgdiv> - </affiliation> - </author> - </authorgroup> - <date>September 15, 2006</date> - <edition>Manual for version 0.8.4</edition> - <releaseinfo>$Revision$</releaseinfo> - <pubdate>July 2006</pubdate> - <copyright> - <year>2005-2006</year> - <holder>Argonne National Laboratory</holder> - </copyright> - - <legalnotice> - <para> - This manual is free software; you can redistribute it and/or - modify it under the terms of the GNU General Public License as - published by the Free Software Foundation; either version 2 of - the License, or (at your option) any later version. - </para> - <para> - This is distributed in the hope that it will be useful, - but <emphasis>without any warranty</emphasis>; without even the - implied warranty of <emphasis>merchantability</emphasis> or - <emphasis>fitness for a particular purpose</emphasis>. See the - GNU General Public License for more details. - </para> - <para> - Required software packages may require additional - terms. Please refer to these individual packages for more - information. - </para> - </legalnotice> - - <revhistory> - <revision> - <revnumber>0.8.4</revnumber> - <date>$Date$</date> - <revremark>$Id$</revremark> - </revision> - </revhistory> - </bookinfo> - -&architecture; -&install; -&specification; -&deployment; -&development; -&reports; -</book> diff --git a/doc/reports.xml b/doc/reports.xml deleted file mode 100644 index 1587c88d4..000000000 --- a/doc/reports.xml +++ /dev/null @@ -1,246 +0,0 @@ -<chapter id='chap:reports'> - <title>BCFG2 Reports</title> - - <para> - Reports play an important role in effectively managing systems - with BCFG. There are two primary functions they fulfill; providing - otherwise unobtainable information, and presenting common - information in a compact, effective format that allows for easier - admistration. Reports can contain system statistics, discrepancies - between specified and actual configuration, invalid configuration - notices, and auditing information, among other things. </para> - - <para> - The flexible XML configuration file allows reports to be configured - to deliver only the information that is important. Additional reports - can easily be created, providing site-specific capability to manage - at record effiency. The capability to harvest information regarding - statistics, configuration, and problems in a single location should - prove to be powerful. - </para> - - <section id='sec:reports-how'> - <title>How it works</title> - - <para> - The BCFG2 Reporting System consists of a number of elements - including the <command>bcfg2-build-reports</command> Executable, a - configuration file, and XSLT transform - files. <command>bcfg2-build-reports</command> reads a default - configuration file (or a config file specified on the command - line) then prepares and delivers the reports according to the - format defined in the transform files. It is expected that this - executable will be run by the adminstrator periodically via - <command>cron</command> or similar facility. The executable can - also be run manually on demand for a special sort of report that - needs to be generated immediately. </para> - - <para> - <command>bcfg2-build-reports</command> gets the data it reports from a - number of sources. <filename>Metadata/clients.xml</filename> - contains information about if a host is currently pingable or - not. <command>bcfg2-ping-sweep</command> will be run - automatically by <command>bcfg2-build-reports</command> if the - <filename>Metadata/clients.xml</filename> file is out of date - with pingability information. - </para> - - <para> - The next place <command>bcfg2-build-reports</command> gets data from is - the <filename>statistics.xml</filename> file. This file is - maintained by bcfgd, and is updated whenever a client updates, - therefore is always up to date and no maintainance is required - on this file. Most of the information in the predefined reports - come from this file. - </para> - - <para> - Finally <command>bcfg2-build-reports</command> is able to pull information - from the <filename>Metadata/groups.xml</filename> file as well. - This allows reports to describe the configured profile for each client. - </para> - </section> - - <section id='sec:report-types'> - <title>Report Types</title> - - <para> - There are a number of report types and delivery styles to - present and transmit the reported data. The reporting structure - lends itself best to structuring reports around groups of - machines. For any group of machines any number of reports are - generated. Each report may be delivered via Mail, WWW, or RSS - (or any combination of the three.) In the future additional - report types will be added, and if necessary, additional types - of deliveries will be created. It is easy to create your own - custom report using XSLT. Tables describing report types and - report delivery mechanisms follow: </para> - - <table id='table:report-types'> - <title>Bcfg2 Report Types</title> - <tgroup cols='2'> - <colspec colnum='1' colwidth='1*'/> - <colspec colnum='2' colwidth='4*'/> - <thead> - <row><entry>Report Type</entry><entry>Description</entry></row> - </thead> - <tbody> - <row><entry>Overview-Stats</entry> - <entry> - <para> - This report provides information about a large number - of machines and their states. It is often found to be - useful when the constituent machines are simply - specified as All Nodes, which gives an overall outlook - on your network's health. It makes sense to get this - report via any mechanism. - </para> - </entry></row> - <row><entry>Nodes-Digest</entry> - <entry> - <para> - This report includes details about each node, - specifically what packages, files, etc are broken, and - other node specific info. It makes sense to recieve - this via any mechanism. - </para> - </entry></row> - <row><entry>Nodes-Individual</entry> - <entry> - <para> - This report includes details about each node, but - information is separated in to separate sections (such - as separate e-mails or RSS articles) for delivery. - This works well with e-mail (using filters on the - client side) and for error detection (getting e-mail - when there is a problem. Currently WWW is not a supported - delivery mechanism for this type of report, because it is - not completely clear how such a report could be used. - </para> - </entry></row> - </tbody> - </tgroup> - </table> - - <table id='table:report-delivery'> - <title>Bcfg2 Report Delivery Mechanisms</title> - <tgroup cols='2'> - <colspec colnum='1' colwidth='1*'/> - <colspec colnum='2' colwidth='3*'/> - <thead> - <row><entry>Name</entry><entry>Description</entry></row> - </thead> - <tbody> - <row><entry>www</entry><entry>an XHTML file</entry></row> - <row><entry>rss</entry><entry>an RSS file <comment>(links do - not point at real web links, since they may not exist)</comment></entry></row> - <row><entry>mail</entry><entry>A plaintext e-mail - message</entry></row> - </tbody> - </tgroup> - </table> - - </section> - - <section id='sec:reports-configuration'> - <title>Configuration</title> - - <para>The <filename>report-configuration.xml</filename> file is - the standard file that the <command>bcfg2-build-reports</command> - executable uses when it is run without any command line - arguments. Alternate configuration files, formatted identically, - can be used by specifing -c flag. This can be useful for - running different types of reports at different intervals. For - example:</para> - - <programlisting> - Run this hourly: bcfg2-build-reports -c WebAndRssReport-config.xml - Run this daily: bcfg2-build-reports -c emailReports-config.xml - </programlisting> - - <para>The <filename>report-configuration.xml</filename> - file is structured with a root - <![CDATA[<Reports/>]]> tag at the top level. Within this tag any number - of - <![CDATA[<Report/>]]> tags can be inserted. Each report is structured - around a group of machines. <![CDATA[<Machine/>]]> tags may individually - reference a machine by hostname, or by a Python Regular - Expression describing a group of hostnames. ".*" is especially helpful - to describe all hosts. More information can be found about such Regexes - at: <ulink url="http://docs.python.org/lib/re-syntax.html"/>.</para> - - <para>Any number of <![CDATA[<Delivery/>]]> elements can be - defined for a given report. A delivery consists of a mechanism - and type. The mechanism would be something like Mail or Web, - and the type would describe the intended content of the - report. Some are tailored to overall machine health, while - others could be best fit for auditing purposes</para> - - <para>Finally, each <![CDATA[<Delivery/>]]> element contains one - or more <![CDATA[<Destination/>]]> elements. In the case of an - RSS or WWW report, the destination should be a complete path to - the output file including the file's name. In e-mail based - reports the destination should be a valid e-mail address.</para> - - </section> - - <section id='sec:reports-quick-start'> - <title>Reporting Quick Start</title> - - <para> - The following configuration will generate two separate reports - and deliver them in a number of different ways. For more - information on exactly what each section does, please refer to - the Configuration section above. </para> - - <example id='ex:report-config'> - <title>etc/report-configuration.xml</title> - <programlisting><![CDATA[<Reports> - <Report name='core_stats' good='Y' modified='Y'> - <Delivery mechanism='mail' type='nodes-digest'> - <Destination address='user@domain.tld'/> - <Destination address='user@otherdomain.tld'/> - </Delivery> - <Delivery mechanism='www' type='nodes-digest'> - <Destination address='/var/www/core_stats.html'/> - </Delivery> - <Machine name='.*'/> - </Report> - - <Report name='stats_for_a_machines' good='N' modified='Y'> - <Delivery mechanism='mail' type='nodes-digest'> - <Destination address='user@domain.tld'/> - </Delivery> - <Delivery mechanism='mail' type='overview-stats'> - <Destination address='user@otherdomain.tld'/> - </Delivery> - <Machine name='a.*'/> - <Machine name='x-aim'/> - </Report> - </Reports>]]> - </programlisting> - </example> - - <para> - Once configured correctly, just wait for the e-mail or view the - outputed html files with a web browser. - <command>bcfg2-build-reports</command> to recieve your reports - immediately, or configure cron to run it perodically. E-mail - reports will deliver the appropriate content directly to your - mail client, and the html files should be viewable with any web - browser. It is suggested those files be accessable via a - webserver for convenience to other interested parties. - - </para> - - <mediaobject> - <imageobject> - <imagedata fileref='./images/composed.tif'/> - </imageobject> - - <caption> - <para>Examples of the performance and overview reports.</para> - </caption> - </mediaobject> - </section> -</chapter> diff --git a/doc/specs.xml b/doc/specs.xml deleted file mode 100644 index 13619296a..000000000 --- a/doc/specs.xml +++ /dev/null @@ -1,540 +0,0 @@ -<chapter> - <title>Writing Bcfg2 Specifications</title> - - <para> - The Bcfg2 specification is a set of directives that describe how - hosts should be configured. This information is used to generate - client configurations. This section describes the steps taken - during the composition of bcfg2 specifications. - </para> - - <orderedlist> - <listitem> - <para> - All parts of the specification will correspond to some - subset of the clients that bcfg2 has record of. Find or create a - group corresponding to the target of this specification. - </para> - </listitem> - <listitem> - <para> - Add each new configuration entry to the "abstract - configuration" for the target group. This can be done in one - of two ways. If the new configuration has to do with a - service, or some other piece of inter-dependent configuration, - write a bundle. If not, add the extra configuration entries to - the base. - </para> - </listitem> - <listitem> - <para> - Add configuration data for the above entries that apply only - to the target group. - </para> - </listitem> - <listitem> - <para> - Verify that clients not in the target group remain unaffected - by these changes. - </para> - </listitem> - <listitem> - <para> - Re-validate the bcfg2 repository by running - <command>bcfg2-repo-validate</command>. - </para> - </listitem> - </orderedlist> - - <section> - <title>Interacting with Client Groups in Bcfg2</title> - - <para> - Bcfg2 uses an aspect-based classing mechanism to describe - configuration patterns in its specifications. Each class - describes particular aspects of client configurations. The Bcfg2 metadata - mechanism has two types of information, client metadata and - group metadata. Client metadata describes what top level group a - client is associated with. Its configuration is derived from a - combination of its host information and this group. Group - definitions describe groups in terms of what bundles they - include and other groups they include. Groups have a set of - properties that describe how they can be used. (Starred values - are defaults) - </para> - - <table> - <title>Bcfg2 Group Parameters</title> - <tgroup cols='3'> - <colspec colnum='1' colwidth='1*'/> - <colspec colnum='2' colwidth='7*'/> - <colspec colnum='3' colwidth='2*'/> - <thead> - <row><entry>Name</entry><entry>Description</entry> - <entry>Values</entry></row> - </thead> - <tbody> - <row><entry>profile</entry><entry>If a client can be - directly associated with this group</entry><entry>(True|False*)</entry></row> - <row><entry>public</entry><entry>If a client can freely - associate itself with this group</entry><entry>(True|False*)</entry></row> - <row><entry>toolset</entry> - <entry>Describes which client-side logic should be used to - make configuration - changes</entry><entry>(rh|debian|solaris|aix|auto)</entry></row> - <row><entry>category</entry> - <entry>A group can only contain one instance of a group in - any category. This provides the basis for representing - groups which are conjugates of one another in a rigorous - way. It also provides the basis for negation.</entry><entry>string</entry></row> - </tbody> - </tgroup> - </table> - </section> - - <para> - When a client's configuration is generated, its metadata is - fetched. This includes a list of all groups recursively - dereferenced, and all bundles included by those groups. This - collection has already been processed using the group category - rules, so only one instance from each group category is - included. This metadata is used throughout the rest of the - configuration generation process; it defines the client's abstract - configuration and specifies all literal contents of all - configuration entities. - </para> - - <section> - <title>Adding to the Abstract Configuration</title> - - <para> - When writing bcfg2 specification, administrators primarily - perform one of two operations: addition of new configuration - entities or the modification of existing entries. If new - entities need to be added, then they must be added to the - abstract configuration. This is the inventory of configuration - entities that should be installed on a client. Two plugins - provide the basis for the abstract configuration, the bundler - and base. The bundler builds descriptions of interrelated - configuration entities. These are typically used for the - representation of services, or other complex groups of - entities. Base provides a laundry list of configuration entities - that need to be installed on hosts. These entities are - independent from one another, and can be installed individually - without worrying about the impact on other entities. - </para> - - <para> - Entities in the abstract configuration (and correspondingly in - the literal configuration) can have one of several types. In the - abstract configuration, each of these entities only has a tag - and the name attribute set. - </para> - - <table> - <title>Bcfg2 Configuration Entity Types</title> - <tgroup cols='2'> - <colspec colnum='1'/> - <colspec colnum='2'/> - <thead> - <row><entry>Name</entry><entry>Description</entry></row> - </thead> - <tbody> - <row><entry>Package</entry><entry>Software Package</entry></row> - <row><entry>ConfigFile</entry><entry>Configuration File</entry></row> - <row><entry>Service</entry> - <entry>Persistent system services and daemons</entry></row> - <row><entry>Directory</entry><entry>Filesystem Directories</entry></row> - <row><entry>SymLink</entry><entry>Symbolic links</entry></row> - <row><entry>Permissions</entry> - <entry>The permissions (not contents) of a POSIX path</entry></row> - </tbody> - </tgroup> - </table> - - <section> - <title>Writing Bundles</title> - - <para> - Bundles consist of a set of configuration entities. These - entities are grouped together due to a configuration-time - interdependency. Basic services tend to be the simplest - example of these: they consist of some software package(s) - some configuration files and an indication that some service - should be activated. If any of these pieces are installed or - updated, all should be re-checked and any associated services - should be restarted. - </para> - - <para> - Bundles can also contain conditonal entries that are only used - for hosts in some particular groups. This is useful when a - service name varies from platform to platform. A group is - defined for each platform, hence a different service can be - associated with the bundle for clients in the different - groups. Conditional additions can also add extra functionality - to services as needed. This has proven useful in the case of - configuring webservers; php and ssl can be configured as extra - features that some webservers have and others do not. At the - same time, all configuration-time interdependencies are - maintained. - </para> - </section> - - <section> - <title>Using Base</title> - - <para> - The Base plugin provides a mechanism to add independent - configuration entities to a client's abstract - configuration. All files in the Base/ subdirectory of the - respository are processed, and all entries that fall within - the scope of the client metadata are included in its abstract - configuration. These files are similar to those used by the - Bundler, Svcmgr, and Pkgmgr, without the need for - prioritization used by the later two. - </para> - </section> - </section> - - <section> - <title>Adding to the Literal Configuration</title> - - <para> - During the construction of the literal configuration, first the - abstract configuration is built, and then explicit data is bound - in to each entity. The previous section describes how the first - stage works, and this section describes the second stage. Each - entity will be served by one plugin. This plugin will decide - what explicit data should be bound in to a particular entity for - a given client. Each of these plugins has a specific area of the - configuration repository, corresponding to its name. This - section describes how several of the basic plugins works. - </para> - - <section> - <title>Cfg</title> - <para> - The Cfg plugin provides a configuration file repository that - uses literal file contents to provide client-tailored - configuration file entries. It chooses which data to provide - for a given client based on the aspect-based metadata system - used for high-level client configuration. - </para> - <para> - The Cfg repository is structured much like the filesystem - hierarchy being configured. Each configuration file being - served has a corresponding directory in the configuration - repository. These directories have the same relative path as - the absolute path of the configuration file on the target - system. For example, if Cfg was serving data for the - configuration file <filename>/etc/services</filename>, then - its directory would be in the relative path - <filename>./etc/services</filename> inside of the Cfg - repository. - </para> - <para> - Inside of this file-specific directory, three types of files - may exist. Base files are complete instances of configuration - file. Deltas are differences between a base file and the - target file contents. Base files and deltas are tagged with - metadata specifiers, which describe which groups of clients - the fragment pertains to. Configuration files are constructed - by finding the most specific base file and applying any more - specific deltas. - </para> - <para> - Specifiers are embedded in fragment filenames. For example, in - the fragment <filename>services.G99_webserver</filename>, - "G99_webserver" is the specifier. This specifier applies to - the group (G) webserver with a priority of 99. Files can also - be tagged with a host-specific (H) specifier.Global files are - the least specific. Priorities are used as to break ties. - </para> - <para> - Info files, named <filename>:info</filename> are used to - specify target configuration file metadata, such as owner, - group and permissions. If no <filename>:info</filename> is - provided, targets are installed with default - information. Default metadata is root ownership, root group - memberships, and 0644 file permissions. This file can also - contain an encoding parameter (ascii|base64) and a paranoid - flag that causes diffs to be logged on clients. - </para> - <example> - <title>Cfg/filepath/:info</title> - <programlisting> owner:root - group:root - perms:0755</programlisting> - </example> - - <example> - <title>Cfg/etc/passwd/</title> - <programlisting> $ ls - passwd.H_adenine passwd passwd.G99_chiba - passwd.H_bio-debian passwd.H_cvstest passwd.H_foxtrot - passwd.H_reboot passwd.H_rudy2 passwd.G98_netserv - passwd.G99_tacacs-server.cat :info</programlisting> - </example> - - <para> - In the previous example, there exists files with each of the - characteristics mentioned above. All files ending in ".cat" - are deltas; ones with ".H_" are host specific files. There - exists a base file, a <filename>:info</filename> file, two - class-specified base files, and a bundle-specified base file. - </para> - </section> - - <section> - <title>Pkgmgr</title> - - <para> - The Pkgmgr plugin is responsible for providing package version - and installation information. In the case of each "Package" - entity in the configuration, it binds in information needed to - detect, verify and install the package. It has a similar - format to the files used by Base and Bundler, but with a few - differences. First, each file has a priority. This allows the - same entity to be served by multiple files. The priorities can - be used to break ties in the case that multiple files serve - data for the same package. The other difference is that - automatic deriviation of package information from the file - attribute. The Pkgmgr has a set of regular expressions that - can split package names for several formats. The filenames are - used to construct installation URLs, and set several important - fields like package name and version. - </para> - - <table> - <title>Package Entity Attributes</title> - <tgroup cols='2'> - <colspec colnum='1' colwidth='1*'/> - <colspec colnum='2' colwidth='3*'/> - <thead> - <row><entry>Name</entry><entry>Description</entry></row> - </thead> - <tbody> - <row><entry>name</entry><entry>Package Name</entry></row> - <row><entry>version</entry><entry>Package Version</entry></row> - <row><entry>uri</entry> - <entry>URL-style location of file repository (typically http)</entry></row> - <row><entry>file</entry><entry>Package file name. Several - other attributes (name, version, url) can be automatically - defined based on regular expressions definied in the - Pkgmgr plugin.</entry></row> - <row><entry>simplefile</entry><entry>Package file name. No - name parsing is performed, so no extra fields get - set</entry></row> - </tbody> - </tgroup> - </table> - </section> - <section> - <title>Svcmgr</title> - - <para> - The Svcmgr plugin describes where services should be active - and inactive. Its files have a similar form to those used by - the Pkgmgr. Several files in the Svcmgr repository can contain - overlapping definitions, and a per-file priority is used to - determine precedence, the highest priority file serving data - for a particular service wins, on a service by service basis. - </para> - - <para> - These files also have a similar set of semantics to those used - by the Pkgmgr. Entries in the top level element (Services) are - global definitions. Group elements describe additional - conditions that must be matched for that definition to - supercede less specific ones. Deeply nested definitions must - have their parent condition matched, plus all parent - conditions as well. For example, the following declaration - turns ssh on by default, disables it if the client is a part - of group a, and reenables it if the client is a part of both - groups a and b. Group nesting provides a conjunctive - function. - </para> - - <example> - <title>Svcmgr/ssh.xml</title> - <programlisting><![CDATA[<Services priority='0'> - <Service name='ssh' status='on'/> - <Group name='a'> - <Service name='ssh' status='off'/> - <Group name='b'> - <Service name='ssh' status='on'/> - </Group> - </Group> -</Services>]]></programlisting> - </example> - - <para> - The files used by this plugin can be structured in a number of - ways. The most common method is to use one large file, but - this can be inconvenience due to the large size of the - file. The data can also be split up using any convenient - mechanism: per-service, per-administrator, etc. - </para> - </section> - <section> - <title>Rules</title> - - <para> - The Rules plugin works like Pkgmgr and Svcmgr, but can be used - for any entries, including literal packages and services, - directories, permissions and symlinks. - </para> - </section> - <section> - <title>SSHbase</title> - - <para> - The SSHbase plugin implements ssh public and private key - management functionality. This means that a central record of - ssh host keys is maintained. Also, a correct ssh_known_hosts - file is maintained. This means that the keys for new hosts are - added to this configuration, and also that a correct line for - localhost is created. SSHbase will generate a new key for any - hosts that doesn't already have a key stored in the - repository, so it should be pre-seeded with the keys (public - and private) of pre-existing clients. - </para> - </section> - </section> - - <section> - <title>Checking Group-External Clients for Unintended - Changes</title> - - <para> - Any configuration change will apply to some set of clients. - Often, repository changes can have unintended consequences to - clients not included in the target group. To address this issue, - consider the changes performed, and if they can affect clients - in unexpected ways. - </para> - - </section> - - <section> - <title>Validating the Bcfg2 Repository</title> - - <para> - Bcfg2 includes a repository validation tool that will check all - XML files in the repository against included XML schemas. It is - critical to run this command, - <command>bcfg2-repo-valdate</command> after any modifications to - XML files. If all files validate properly, then no output will - be returned. It takes a "-v" option that prints out a line for - each file that is validated. This can be used to ensure that all - files are checked. - </para> - </section> - - <section> - <title>Annotated Configuration Examples</title> - - <para> - In addition to the description of the abstract process - above, we present several examples of the thought process and - actions taken to achieve a particular configuration goal. These - will start simple, but become more complex. - </para> - - <section> - <title>Configuring /etc/motd on all hosts</title> - - <para>The goal for this example is to install a uniform copy of - a specified <filename>/etc/motd</filename> on all hosts.</para> - - <orderedlist> - <listitem><para> - In this case, the target group is all clients, since we want - this version of <filename>/etc/motd</filename>. As - mentioned earlier, the global group is handled specially, - so that all new clients, even newly created ones, are in it. - </para></listitem> - <listitem><para> - Since <filename>/etc/motd</filename> is not interdependent - with any other configuration entities, it can be installed - using Base instead of using Bundler. The ConfigFile - entity should be placed in the globally scoped section of - a base file. This adds the configuration file to the - abstract configuration. - </para></listitem> - <listitem><para> - A file with the correct contents for - <filename>/etc/motd</filename> should be installed in the - Cfg repository area as a global file. This will provide - the right literal configuration specification for each - client. - </para></listitem> - <listitem><para> - Since this change is globally scoped, there are not any - clients that should not be affected. - </para></listitem> - <listitem><para> - Finally, <command>bcfg2-repo-validate</command> should be - run to catch typos. - </para></listitem> - </orderedlist> - </section> - - <section> - <title>Configuring NTP for a network</title> - - <para> - The goal for this example is to configure NTP for an entire - network. This implies several things. All clients should run - NTP as clients. Some hosts should run NTP as a server, and - other hosts should use local NTP service instead of an - external server. - </para> - - <orderedlist> - <listitem><para> - Two discrete groups are used for the different parts of - the desired configuration state. The first is the global - group, handling the "all clients run ntp" part of the - configuration specification. The other part corresponds to - a new group "ntp-server", which contains only clients that - should function as an ntp server. - </para></listitem> - <listitem><para> - This configuration specification describes configuration - entities that have interdependencies, so a bundle should - be used. This bundle should contain the ntp software, the - ntp configuration file, and the ntpd service. - </para></listitem> - <listitem><para> - The literal portions of three different configuration - entities need to be represented. First, the Pkgmgr needs - to be configured to bind a package entity named ntp. Next, - the Svcmgr needs to have a global declaration that the - service ntpd should be on. Finally, two different - configuration files should be added to Cfg. The global - version of <filename>/etc/ntp.conf</filename> should have - an ntp configuration that points hosts at the local ntp - server. The group "ntp-server" specific version of this - file should have the proper configuration for local ntp - servers. - </para></listitem> - <listitem><para> - A quick inspection of these configuration changes show - only minor possibilities for bad interactions. All - machines should run ntp, and the only scope where bad - interactions can occur is on ntp servers. - </para></listitem> - <listitem><para> - Finally, run <command>bcfg2-repo-validate</command>. It - will validate the new bundle that has been added, and - changes to the Svcmgr and Pkgmgr indices. - </para></listitem> - </orderedlist> - </section> - - <!-- <para>Need more examples here</para> --> - - </section> -</chapter>
\ No newline at end of file diff --git a/tools/export.sh b/tools/export.sh index 4f552f75e..eec548f99 100755 --- a/tools/export.sh +++ b/tools/export.sh @@ -15,7 +15,6 @@ tagstr=`echo ${version} | sed -e 's/\./_/g'` svn copy "$url" "${repo}/tags/${name}_${tagstr}" -m "tagged ${version} release" svn export . "${expath}" svn log -v "${repo}/tags/${name}_${tagstr}" > "${expath}/ChangeLog" -cd "${expath}"/doc && make all cd /tmp tar czf "${tarname}" "${name}-${version}" |