1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
|
.. -*- mode: rst -*-
ganglia
=======
Another interesting example of Genshi templating is to automatically
generate ``gmond``/``gmetad`` configuration files. The idea is that
each cluster is headless: it communicates with the rest of the cluster
members on an isolated multicast IP address and port. Any of the
cluster members is therefore isolated on that particular ip/port
pair. Additionally, each ``gmond`` instance **also** listens on
UDP. This allows for any of the cluster members to be polled for
information on the entire cluster!
The second part of the trick is in ``gmetad.conf``. Here, we
dynamically generate a list of clusters (based on profiles names) and
a list of members to poll (based on the clients in said profiles). As
the number of profiles and client grows, this list will grow
automatically as well. When a new host is added, ``gmetad`` will
receive an updated configuration and act accordingly.
There **is** one caveat though. The ``gmetad.conf`` parser is hard
coded to read 16 arguments per ``data_source`` line. If you have more
than 15 nodes in a cluster, you will see a warning in the logs. You
can either ignore it, or truncate the list to the first 15 members.
In our environment, a profile is a one to one match with the role of
that particular host. You can also do this based on groups, or any
other client property.
Bundler/ganglia.xml
-------------------
.. code-block:: xml
<Bundle>
<Package name='ganglia-gmond' />
<Package name='ganglia-gmond-modules-python' />
<Path name='/etc/ganglia/gmond.conf' />
<Service name='gmond' />
<Action name='gmond-reload' />
<Group name='gmetad-server'>
<Package name='ganglia-gmetad'/>
<Package name='ganglia-web'/>
<Package name='rrdtool'/>
<Path name='/etc/ganglia/gmetad.conf' />
<Service name='gmetad' />
</Group>
</Bundle>
Rules/services-ganglia.xml
--------------------------
.. code-block:: xml
<Rules priority='10'>
<Service name='gmond' type='chkconfig' status='on' />
<Group name='gmetad-server'>
<Service name='gmetad' type='chkconfig' status='on' />
</Group>
</Rules>
Cfg/etc/ganglia/gmetad.conf/gmetad.conf.genshi
----------------------------------------------
.. code-block:: none
{% python
client_metadata = metadata.query.all()
profile_array = {}
seen = []
for item in client_metadata:
if item.profile not in seen:
seen.append(item.profile)
profile_array[item.profile]=[]
profile_array[item.profile].append(item.hostname)
seen.sort()
%}\
gridname "Our Grid"
{% for profile in seen %}
data_source "${profile}" \
{% for host in profile_array[profile] %}\
${host} \
{% end %}\
{% end %}
rrd_rootdir "/var/lib/ganglia/rrds"
Cfg/etc/ganglia/gmond.conf/gmod.conf.genshi
-------------------------------------------
.. code-block:: none
{% python
import random
random.seed(metadata.profile)
last_octet=random.randint(2,254)
%}\
/*
$$Id$$
$$HeadURL$$
*/
/* This configuration is as close to 2.5.x default behavior as possible
The values closely match ./gmond/metric.h definitions in 2.5.x */
globals {
daemonize = yes
setuid = yes
user = nobody
debug_level = 0
max_udp_msg_len = 1472
mute = no
deaf = no
host_dmax = 1800 /* 30 minutes */
cleanup_threshold = 604800 /*secs=1 week */
gexec = no
send_metadata_interval = 0
}
/* If a cluster attribute is specified, then all gmond hosts are wrapped inside
* of a <CLUSTER> tag. If you do not specify a cluster tag, then all <HOSTS> will
* NOT be wrapped inside of a <CLUSTER> tag. */
cluster {
name = "${metadata.profile}"
owner = "user@company.net"
latlong = "unspecified"
url = "unspecified"
}
/* The host section describes attributes of the host, like the location */
host {
location = "unspecified"
}
/* Feel free to specify as many udp_send_channels as you like. Gmond
used to only support having a single channel */
udp_send_channel {
host = ${metadata.hostname}
port = 8649
}
udp_send_channel {
mcast_join = 239.2.11.${last_octet}
port = 8649
ttl = 1
}
/* You can specify as many udp_recv_channels as you like as well. */
udp_recv_channel {
port = 8649
bind = ${metadata.hostname}
}
udp_recv_channel {
mcast_join = 239.2.11.${last_octet}
bind = 239.2.11.${last_octet}
port = 8649
}
/* You can specify as many tcp_accept_channels as you like to share
an xml description of the state of the cluster */
tcp_accept_channel {
port = 8649
}
/* Each metrics module that is referenced by gmond must be specified and
loaded. If the module has been statically linked with gmond, it does not
require a load path. However all dynamically loadable modules must include
a load path. */
modules {
/* [snip] */
|
