1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
|
.. -*- mode: rst -*-
.. _help-troubleshooting:
===============
Troubleshooting
===============
From time to time, Bcfg2 produces results that the user finds surprising.
This can happen either due to bugs or user error. This page describes
several techniques to gain visibility into the bcfg2 client and server
and understand what is going on.
Figure out if error is client or server side
============================================
* Cache a copy of the client configuration using ``bcfg2 -qnc /tmp/config.xml``
* Look in the file and search for the entry of interest
* If it looks correct, then there is a client issue
* If not, it is time to inspect things on the server
This file contains all aspects of client configuration. It is structured
as a series of bundles and base entries.
.. note::
Most often the entry is not correct and the issue lies in
the specification.
Review server log messages
==========================
The bcfg2-server process logs to syslog facility LOG_DAEMON. The server
produces a series of messages upon a variety of events and errors.
The server also supports two XML-RPC methods that can be used
to turn up the debug level in a live server:
* ``toggle_debug``: Turn debug on or off, depending on the current
setting.
* ``set_debug``: Turn debug explicitly on or off.
These can be called with :ref:`bcfg2-admin xcmd <server-admin-xcmd>`,
e.g.::
bcfg2-admin xcmd toggle_debug
bcfg2-admin xcmd set_debug true
Each plugin also supports these two methods, which can be used to set
the debug level individually on a given plugin, e.g.::
bcfg2-admin xcmd Packages.set_debug true
bcfg2-admin xcmd Probes.toggle_debug
Finally, the File Activity Monitor has its own analogue to these two
methods, for setting the debug level of the FAM:
bcfg2-admin xcmd Inotify.toggle_debug
bcfg2-admin xcmd Inotify.set_debug false
Check if all repository XML files conform to schemas
====================================================
Bcfg2 comes with XML schemas describing all of the XML formats used in
the server repository. A validation command ``bcfg2-lint`` is
included with the source distribution and all packages.
If the bcfg2 server is not reflecting recent changes, try restarting the bcfg2-server process
=============================================================================================
If this fixes the problem, it is either a bug in the
underlying file monitoring system (fam or gamin) or a bug in
Bcfg2's file monitoring code. In either case, file a `ticket
<https://trac.mcs.anl.gov/projects/bcfg2/newticket>`_ in the tracking
system. In the ticket, include:
* filesystem monitoring system (fam or gamin)
* kernel version (if on linux)
* if any messages of the form "Handled N events in M
seconds" appeared between the modification event and the client
configuration generation request appeared in the server log
* which plugin handled the file in the repostiory (Cfg, Rules, Packages,
SSHbase, Metadata, etc.)
* if a touch of the file after the modification causes the problem to
go away
bcfg2-info
==========
Bcfg2 server operations can be simulated using the ``bcfg2-info`` command.
The command is interactive, and has commands to allow several useful
operations
* clients - Current client metadata (profile and group) settings
* groups - Current group metadata values
* mappings - Configuration entries provided by plugins
* buildfile [--altsrc=<altsrc>] <filename> <hostname> - Build a config
file for a client
* buildbundle <bundle> <hostname> - Render a templated bundle for a client
* showentries <client> <type> - Build the abstract configuration (list
of entries) for a client
* build <hostname> <output-file> - Build the complete configuration
for a client
Type `help` in bcfg2-info for more information.
Error Messages
==============
The tables below describe error messages produced by Bcfg2 and steps that can
be taken to remedy them.
Client Errors
-------------
+------------------------------+----------------------------+--------------+
| Error | Meaning | Repair |
+==============================+============================+==============+
| Incomplete information for | The described entry is not | [c1]_ |
| entry <EntryTag>:<EntryName> | fully specified by the | |
| cannot verify | server, so no verification | |
| | can be performed. | |
+------------------------------+----------------------------+--------------+
| Incomplete information for | The described entry is not | [c1]_ |
| entry <EntryTag>:<EntryName> | fully specified by the | |
| cannot install | server, so no verification | |
| | can be performed. | |
+------------------------------+----------------------------+--------------+
| The following entries are | The client cannot figure | [c2]_ |
| not handled by any tool: | out how to handle this | |
| <EntryTag>:<EntryName> | entry. | |
+------------------------------+----------------------------+--------------+
| No ca is specified. Cannot | The client is unable to | [c3]_ |
| authenticate the server with | verify the server. | |
| SSL. | | |
+------------------------------+----------------------------+--------------+
| GID normalization failed for | The client is unable to | [c4]_ |
| FILENAME. Does group GROUP | convert the group GROUP to | |
| exist? | a usable GID. | |
+------------------------------+----------------------------+--------------+
| UID normalization failed for | The client is unable to | [c5]_ |
| FILENAME. Does owner OWNER | convert the owner OWNER to | |
| exist? | a usable UID. | |
+------------------------------+----------------------------+--------------+
| SSL CA error | The CA certificate | [c6]_ |
| | specified in bcfg2.conf is | |
| | incorrect | |
+------------------------------+----------------------------+--------------+
.. [c1] This entry is not being bound. Ensure that a version of this
entry applies to this client.
.. [c2] It is possible that the type attribute for this generator entry
is undefined. You may need to add the appropriate type attribute
in the literal entry specification.
.. [c3] Copy the Bcfg2 server's CA certificate to the client and specify it
using the **ca** option in the [communication] section of
``bcfg2.conf``
.. [c4] If the group doesn't exist, you need to specify the correct one
in an :ref:`info.xml <server-info>` file or set the default group
appropriately.
.. [c5] If the owner doesn't exist, you need to specify the correct one
in an :ref:`info.xml <server-info>` file or set the default owner
appropriately.
.. [c6] Check that the CA specified in bcfg2.conf is appropriate for the
server you are attempting to access.
Server Errors
-------------
+------------------------------+---------------------+--------------+
| Error | Meaning | Repair |
+==============================+=====================+==============+
| Failed to bind entry: | The server was | [s1]_ |
| <EntryTag> <EntryName> | unable to find a | |
| | suitable version of | |
| | entry for client. | |
+------------------------------+---------------------+--------------+
| Failed to bind to socket | The server was | [s2]_ |
| | unable to bind to | |
| | the tcp server | |
| | socket. | |
+------------------------------+---------------------+--------------+
| Failed to load | The server was | [s3]_ |
| ssl key <path> | unable to read and | |
| | process the ssl key.| |
+------------------------------+---------------------+--------------+
| Failed to read file <path> | The server failed | [s4]_ |
| | to read the | |
| | specified file | |
+------------------------------+---------------------+--------------+
| Failed to parse file <path> | The server failed | [s5]_ |
| | to parse the | |
| | specified XML file | |
+------------------------------+---------------------+--------------+
| Client metadata resolution | The server cannot | [s6]_ |
| error for <IP> | resolve the client | |
| | hostname or the | |
| | client is | |
| | associated with a | |
| | non-profile group. | |
+------------------------------+---------------------+--------------+
| Failed to decode <filename> | The encoding being | [s7]_ |
| Please verify you are using | used is unable to | |
| the proper encoding | decode the | |
| | character present | |
| | in this file. | |
+------------------------------+---------------------+--------------+
| Got unknown entries | The Packages plugin | [s8]_ |
| [list of unknown entries] | has no knowledge of | |
| | the listed entries | |
+------------------------------+---------------------+--------------+
| Failed to import lxml | The server cannot | [s9]_ |
| dependency. Shutting | import lxml | |
| down server. | | |
+------------------------------+---------------------+--------------+
| You need to specify base64 | The server cannot | [s10]_ |
| encoding for <path> | send the file as | |
| | ascii text | |
+------------------------------+---------------------+--------------+
| ERROR: Error reading file | The server cannot | [s11]_ |
| '/path/to/schema': failed to | find the schema | |
| load external entity | file | |
| "/path/to/schema" | | |
+------------------------------+---------------------+--------------+
| Packages: No matching | None of the sources | [s12]_ |
| sources for client | defined in the | |
| <clientname>; improper group | Package plugin's | |
| memberships? | ``sources.xml`` | |
| | apply to the client | |
+------------------------------+---------------------+--------------+
.. [s1] This entry is not being bound. Ensure that a version of this
entry applies to this client.
.. [s2] Ensure that another instance of the daemon (or any other process)
is not listening on the same port.
.. [s3] Ensure that the key is readable by the user running the daemon
and that it is well-formed.
.. [s4] Ensure that this file still exists; a frequent cause is the
deletion of a temp file.
.. [s5] Ensure that the file is properly formed XML.
.. [s6] Fix hostname resolution for the client or ensure that the profile
group is properly setup.
.. [s7] Ensure the correct encoding is specified in the [components]
section of ``bcfg2.conf``.
.. [s8] For packages listed other than **gpg-pubkey**, this error means
that the Packages plugin is unable to find the package in any of
the sources listed in ``Packages/sources.xml``. The issue often
arises when the client is not in one of the groups necessary for
the Source listed. In the case of gpg-pubkey, you can safely
ignore the message as the Packages plugin has no knowledge of
these packages (however, note that this package is most often
specified as a BoundPackage entry).
.. [s9] Ensure that you have installed all the necessary
:ref:`installation-prerequisites`.
.. [s10] You likely need to specify a base64 encoding using an
:ref:`server-info` file for this entry.
.. [s11] Verify that you have the proper prefix set in bcfg2.conf.
.. [s12] Ensure that the client is a member of all the appropriate
:ref:`server-plugins-generators-packages-magic-groups` as
well as any additional groups you may have defined in your
:ref:`server-plugins-generators-packages` configuration.
FAQs
====
Why won't bcfg2-server start?
-----------------------------
If your server doesn't seem to be starting and you see no error
messages in your server logs, try running it in the foreground to see
why.
Why am I getting a traceback?
-----------------------------
If you get a traceback, please let us know by reporting it
on `Trac ticket tracker`_, via the mailing list, or on IRC. Your best bet
to get a quick response will be to jump on IRC during the daytime (CST).
.. _Trac ticket tracker: http://bcfg2.org
What is the most common cause of "The following entries are not handled by any tool"?
-------------------------------------------------------------------------------------
Often it corresponds to entries that aren't bound by the server (for which
you'll get error messages on the server). You should try inspecting the
logs on the server to see what may be the cause.
|
