Ldap: Complete renew of the Ldap plugin

3 files changed, 107 insertions, 89 deletions

diff --git a/doc/man/bcfg2.conf.txt b/doc/man/bcfg2.conf.txt
index 2f014812e..6c801ff1e 100644
--- a/doc/man/bcfg2.conf.txt
+++ b/doc/man/bcfg2.conf.txt
@@ -267,7 +267,7 @@ revision information out of your repository for reporting purposes.
 Ldap Plugin
 +++++++++++
 
-The Ldap plugin makes it possible to fetch data from an LDAP directory,
+The Ldap plugin makes it possible to fetch data from a LDAP directory,
 process it and attach it to your metadata.
 
 Metadata Plugin
diff --git a/doc/releases/1.4.0pre2.txt b/doc/releases/1.4.0pre2.txt
index 195f81df0..1dcdf237b 100644
--- a/doc/releases/1.4.0pre2.txt
+++ b/doc/releases/1.4.0pre2.txt
@@ -36,6 +36,15 @@ backwards-incompatible user-facing changes
 
   This allows to set arbitrary options with nested settings.
 
+* The Ldap plugin changed significantly. The configuration interface was
+  simplified and new configuration options for the number of retries and the
+  delay in between were added.
+
+  You have to register your ldap queries in the global list, there is no
+  distinction between LdapQueries and LdapSubQueries anymore, the names of
+  your queries default to the class names and the Ldap plugin expires
+  the metadata caches if the config file changes.
+
 Thanks
 ------
 
diff --git a/doc/server/plugins/grouping/ldap.txt b/doc/server/plugins/grouping/ldap.txt
index 90590a272..96e224761 100644
--- a/doc/server/plugins/grouping/ldap.txt
+++ b/doc/server/plugins/grouping/ldap.txt
@@ -33,39 +33,38 @@ next section.
 Configuration
 -------------
 
-As processing LDAP search results can get pretty complex, the configuration has 
+As processing LDAP search results can get pretty complex, the configuration has
 to be written in Python.
 
 Here is a minimal example to get you started::
 
-    from Bcfg2.Server.Plugins.Ldap import LdapConnection, LdapQuery, LdapSubQuery, register_query
-    
-    conn_default = LdapConnection()
-    conn_default.binddn = "uid=example,ou=People,dc=example,dc=com"
-    conn_default.bindpw = "foobat"
-    
-    @register_query
+    from Bcfg2.Server.Plugins.Ldap import LdapConnection, LdapQuery
+
+    __queries__ = ['ExampleQuery']
+
+    conn_default = LdapConnection(
+        binddn="uid=example,ou=People,dc=example,dc=com",
+        bindpw = "foobat")
+
     class ExampleQuery(LdapQuery):
-        name = "example"
         base = "ou=People,dc=example,dc=com"
         scope = "one"
         attrs = ["cn", "uid"]
         connection = conn_default
-    
+
         def prepare_query(self, metadata):
             self.filter = "(personalServer=" + metadata.hostname + ")"
-    
+
         def process_result(self, metadata):
             if not self.result:
                 admin_uid = None
                 admin_name = "This server has no admin."
-            return { 
+            return {
                 "admin_uid" : self.result[0][1]["uid"],
                 "admin_name" : self.result[0][1]["cn"]
             }
 
-The first line provides three classes for dealing with connections and queries
-(details below) and a decorator function for registering your queries with the plugin.
+The first line provides the two required classes for dealing with connections and queries.
 
 In this example our LDAP directory has a number of user objects in it. Each of those
 may have a personal server they administer. Whenever metadata for this machine is being
@@ -73,7 +72,20 @@ generated by the Bcfg2 server, the UID and name of the admin are retrieved from
 
 In your bundles and config templates, you can access this data via the metadata object::
 
-    ${metadata.Ldap["example"]["admin_name"]}
+    ${metadata.Ldap["ExampleQuery"]["admin_name"]}
+
+Connection retry
+++++++++++++++++
+
+If the LDAP server is down during a request, the LDAP plugin tries to reconnect after a
+short delay. By default, it waits 3 seconds during the retries and tries to reconnect
+up to three times.
+
+If you wish, you could customize these values in your ``bcfg2.conf``::
+
+    [ldap]
+    retries = 3
+    retry_delay = 3.0
 
 Class reference
 ---------------
@@ -83,23 +95,23 @@ LdapConnection
 
 .. class:: LdapConnection
 
-   This class represents an LDAP connection. Every query must be associated with exactly 
+   This class represents an LDAP connection. Every query must be associated with exactly
    one connection.
-   
-.. attribute:: LdapConnection.binddn 
-   
+
+.. attribute:: LdapConnection.binddn
+
    DN used to authenticate against LDAP (required).
-   
+
 .. attribute:: LdapConnection.bindpw
-   
+
    Password for the previously mentioned **binddn** (required).
-  
+
 .. attribute:: LdapConnection.host
-   
+
    Hostname of host running the LDAP server (defaults to "localhost").
 
 .. attribute:: LdapConnection.port
-   
+
    Port where LDAP server is listening (defaults to 389).
 
 You may pass any of these attributes as keyword arguments when creating the connection object.
@@ -108,143 +120,140 @@ LdapQuery
 +++++++++
 
 .. class:: LdapQuery
-   
+
    This class defines a single query that may adapt itself depending on the current metadata.
 
 .. attribute:: LdapQuery.attrs
-   
+
    Can be used to retrieve only a certain subset of attributes. May either be a list of
    strings (attribute names) or ``None``, meaning all attributes (defaults to ``None``).
 
 .. attribute:: LdapQuery.base
-   
-   This is the search base. Only LDAP entries below this DN will be included in your 
+
+   This is the search base. Only LDAP entries below this DN will be included in your
    search results (required).
-   
+
 .. attribute:: LdapQuery.connection
-   
+
    Set this to an instance of the LdapConnection class (required).
 
 .. attribute:: LdapQuery.filter
-   
+
    LDAP search filter used to narrow down search results (defaults to ``(objectClass=*)``).
 
 .. attribute:: LdapQuery.name
-   
+
    This will be used as the dictionary key that provides access to the query results from
-   the metadata object (``metadata.Ldap["NAMEGOESHERE"]``) (required).
+   the metadata object: ``metadata.Ldap["NAMEGOESHERE"]`` (defaults to the class name).
 
 .. attribute:: LdapQuery.scope
-   
-   Set this to one of "base", "one" or "sub" to specify LDAP search depth (defaults to "sub"). 
+
+   Set this to one of "base", "one" or "sub" to specify LDAP search depth (defaults to "sub").
 
 .. method:: LdapQuery.is_applicable(self, metadata)
-   
+
    You can override this method to indicate whether this query makes sense for a given
    set of metadata (e.g. you need a query only for a certain bundle or group).
-   
+
    (defaults to returning True)
-   
-.. method:: LdapQuery.prepare_query(self, metadata)
-   
+
+.. method:: LdapQuery.prepare_query(self, metadata, \**kwargs)
+
    Override this method to alter the query prior to execution. This is useful if your filter
    depends on the current metadata, e.g.::
-   
+
        self.filter = "(cn=" + metadata.hostname + ")"
-   
+
    (defaults to doing nothing)
 
-.. method:: LdapQuery.process_result(self, metadata)
-   
+.. method:: LdapQuery.process_result(self, metadata, \**kwargs)
+
    You will probably override this method in every query to reformat the results from LDAP.
    The raw result is stored in ``self.result``, you must return the altered data. Note that LDAP
    search results are presented in this structure::
-   
+
       (
           ("DN of first entry returned",
               {
                   "firstAttribute" : 1,
                   "secondAttribute" : 2,
-              } 
+              }
           ),
           ("DN of second entry returned",
               {
                   "firstAttribute" : 1,
                   "secondAttribute" : 2,
-              } 
+              }
           ),
       )
-   
+
    Therefore, to return just the value of the firstAttribute of the second object returned,
    you'd write::
-   
+
       return self.result[1][1][0]
-   
+
    (defaults to returning ``self.result`` unaltered)
 
-LdapSubQuery
-++++++++++++
-
-.. class:: LdapSubQuery
-
-   Sometimes you need more than one query to obtain the data you need (e.g. use the first
-   query to return all websites running on metadata.hostname and another query to find all
-   customers that should have access to those sites).
-   
-   LdapSubQueries are the same as LdapQueries, except for that the methods
-   
-   * ``get_result()``
-   * ``prepare_query()``
-   * ``process_result()``
-     
-   allow any additional keyword arguments that may contain additional data as needed. Note 
-   that ``get_result()`` will call ``prepare_query()`` and ``process_result()`` for you,
-   so you shouldn't ever need to invoke these yourself, just override them.
-   
-Here is another example that uses LdapSubQuery::
-
-	class WebSitesQuery(LdapSubQuery):
-	    name = "web_sites"
+.. method:: LdapQuery.get_result(self, metadata, \**kwargs)
+
+   This executes the query. First it will call ``prepare_query() for you, then it will try
+   to execute the query with the specified connection and last it will call ``process_result()``
+   and return that return value.
+
+If you use a LdapQuery class by yourself, you could pass additional keyword arguments to
+``get_result()``. It will call ``prepare_query()`` and ``process_result()`` for you and
+also supply this additional arguments to this methods.
+
+Here is an example::
+
+	__queries__ = ['WebPackageQuery']
+
+	class WebSitesQuery(LdapQuery):
 	    filter = "(objectClass=webHostingSite)"
 	    attrs = ["dc"]
 	    connection = conn_default
-	    
+
 	    def prepare_query(self, metadata, base_dn):
 	        self.base = base_dn
-	    
-	    def process_result(self, metadata):
+
+	    def process_result(self, metadata, **kwargs):
 	        [...] # build sites dict from returned dc attributes
 	        return sites
-	
-	@register_query
+
 	class WebPackagesQuery(LdapQuery):
-	    name = "web_packages"
 	    base = "dc=example,dc=com"
 	    attrs = ["customerId"]
 	    connection = conn_default
-	    
+
 	    def prepare_query(self, metadata):
 	        self.filter = "(&(objectClass=webHostingPackage)(cn:dn:=" + metadata.hostname + "))"
-	    
+
 	    def process_result(self, metadata):
 	        customers = {}
 	        for customer in self.result:
 	            dn = customer[0]
 	            cid = customer[1]["customerId"][0]
-	            customers[cid]["sites"] = WebSitesQuery().get_result(metadata, base_dn = dn)
+	            customers[cid]["sites"] = WebSitesQuery().get_result(metadata, base_dn=dn)
 	        return customers
 
 This example assumes that we have a number of webhosting packages that contain various
-sites. We need a first query ("web_packages") to get a list of the packages our customers
-have and another query for each of those to find out what sites are contained in each 
-package. The magic happens in the second class where ``WebSitesQuery.get_result()`` is 
-called with the additional ``base_dn`` parameter that allows our LdapSubQuery to only 
+sites. We need the ``WebPackagesQuery`` to get a list of the packages our customers
+have and another query for each of those to find out what sites are contained in each
+package. The magic happens in the second class where ``WebSitesQuery.get_result()`` is
+called with the additional ``base_dn`` parameter that allows our LdapQuery to only
 search below that DN.
 
-.. warning::
-    Do NOT apply the ``register_query`` decorator to LdapSubQueries.
+You do not need to add all LdapQueries to the ``__queries__`` list. Only add those to
+that list, that should be called automatically and whose results should be added to the
+client metadata.
 
 Known Issues
 ------------
 
 * At this point there is no support for SSL/TLS.
+* This module could not know, if a value changed on the LDAP server. So it could not
+  expire the client metadata cache sanely.
+  If you are using aggressive caching mode, this plugin will expire the metadata cache
+  for a single client at the start of a client run. If you are using LDAP data from
+  another client in a template, you will probably get the cached values from the last
+  client run of that other client.