summaryrefslogtreecommitdiffstats
path: root/src/lib/Bcfg2/Reporting
Commit message (Collapse)AuthorAgeFilesLines
* Reporting/Storage: Fix handling of extra packages.Alexander Sulfrian2015-02-191-6/+5
| | | | | | | The code before for extra packages only worked for rpms because it only evaluated the version information of the contained <Instance> tag and droped the package if no one was avilable. Now the handling of <Instance> is optional and the package is imported everytime.
* Merge branch 'maint'Alexander Sulfrian2014-11-126-64/+79
|\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Conflicts: debian/changelog doc/conf.py doc/releases/index.txt doc/server/database.txt doc/server/plugins/generators/cfg.txt doc/server/plugins/generators/sslca.txt man/bcfg2.conf.5 misc/bcfg2-selinux.spec misc/bcfg2.spec osx/Makefile osx/macports/Portfile schemas/packages.xsd solaris-ips/MANIFEST.bcfg2-server.header solaris-ips/MANIFEST.bcfg2.header solaris-ips/Makefile solaris-ips/pkginfo.bcfg2 solaris-ips/pkginfo.bcfg2-server solaris/Makefile solaris/pkginfo.bcfg2 solaris/pkginfo.bcfg2-server src/lib/Bcfg2/Client/Client.py src/lib/Bcfg2/Client/Proxy.py src/lib/Bcfg2/Client/Tools/APT.py src/lib/Bcfg2/Client/Tools/Action.py src/lib/Bcfg2/Client/Tools/SYSV.py src/lib/Bcfg2/Client/Tools/YUM.py src/lib/Bcfg2/Client/Tools/__init__.py src/lib/Bcfg2/Options.py src/lib/Bcfg2/Reporting/Collector.py src/lib/Bcfg2/Reporting/templates/base.html src/lib/Bcfg2/Server/Admin/__init__.py src/lib/Bcfg2/Server/BuiltinCore.py src/lib/Bcfg2/Server/Core.py src/lib/Bcfg2/Server/Plugins/Packages/Apt.py src/lib/Bcfg2/Server/Plugins/Probes.py src/lib/Bcfg2/settings.py src/lib/Bcfg2/version.py testsuite/Testsrc/Testlib/TestClient/TestTools/Test_init.py testsuite/Testsrc/test_code_checks.py
| * Merge branch 'cleanup_pidfile' of https://github.com/mattikus/bcfg2 into maintSol Jerome2014-10-231-16/+22
| |\
| | * Remove PIDFileError as it does not always exist in the package and is rarely ↵Matt Kemp2014-10-021-5/+0
| | | | | | | | | | | | used.
| | * Catch possible typeerror resulting from None being returned when reading the ↵Matt Kemp2014-10-021-1/+1
| | | | | | | | | | | | pid.
| | * Fixes to ensure pidfile can be opened or broken if stale.Matt Kemp2014-10-011-18/+11
| | |
| | * Attempt to break the pid lock during startup.Matt Kemp2014-10-011-8/+26
| | | | | | | | | | | | | | | | | | | | | | | | This commit attempts to break the pidfilelock during startup in cases where the process may have exited without successfully cleaning up the lockfile. It also attempts to grab the lock before opening the context. Also applied to the Collector module, which may have been looking for the wrong exception since it does not rely on a timeout.
| * | Reporting: better exception handlingAlexander Sulfrian2014-10-201-33/+44
| | | | | | | | | | | | Try to keep the try-except-blocks as small as possible.
| * | Reporting: fix filter urlsAlexander Sulfrian2014-10-121-6/+6
| | | | | | | | | | | | | | | | | | The regex match for the filter urls were to strict. They disallowed some charaters, that are valid in group names and so the django reverse mechanism for building urls failed.
| * | Use older nested try syntax for finally for python 2.4 compatMichael Fenn2014-09-101-4/+5
| | |
| * | Version bump to 1.3.5v1.3.5Sol Jerome2014-09-051-1/+1
| | | | | | | | | | | | Signed-off-by: Sol Jerome <sol.jerome@gmail.com>
| * | Reporting: Remove duplicate methodSol Jerome2014-08-141-3/+0
| | | | | | | | | | | | Signed-off-by: Sol Jerome <sol.jerome@gmail.com>
| * | Reporting: Fix prune items import and displaySol Jerome2014-07-152-2/+2
| |/ | | | | | | Signed-off-by: Sol Jerome <sol.jerome@gmail.com>
* | Reporting/Collector: fix semaphore initializationAlexander Sulfrian2014-11-111-1/+2
| |
* | testsuite: Added unit tests for new option parsingChris St. Pierre2014-11-101-1/+2
| |
* | Reporting/Collector: fix python2.4 compatibilityAlexander Sulfrian2014-10-271-9/+11
| |
* | Reporting/Collector: add max-children argumentAlexander Sulfrian2014-10-271-3/+19
| | | | | | | | | | | | Add option to limit the count of child threads to import the transactions. If the number is exceeded the next import will block until one thread is ready.
* | Reporting: do not duplicate isstale functionalityAlexander Sulfrian2014-08-012-15/+2
| | | | | | | | | | | | | | | | | | | | | | The interaction entries have a isstale() method, we do not need a template tag, that tries to figure out this on its own. The isstale() method works even better: It knows, if the interaction is the current interaction and compares the timestamp to the current time or (if it is not the current one) it compares the timestamp to the timestamp of the next interaction. So using the method of the model, you can browse the interaction history and see, if the host was stale some time in the past. Previously all interactions more than 24h ago were marked stale.
* | Merge branch 'more-highlights-in-gridview' of ↵Sol Jerome2014-06-191-0/+4
|\ \ | | | | | | | | | https://github.com/AlexanderS/bcfg2
| * | Reports/templatetags: mark special hosts in grid viewAlexander Sulfrian2014-06-181-0/+4
| | | | | | | | | | | | | | | Highligh additional special hosts in the grid view. Stale hosts get a grey background and clean hosts with extra packages get a blue background.
* | | Merge branch 'reports-add-stale-col' of https://github.com/AlexanderS/bcfg2Sol Jerome2014-06-181-0/+2
|\ \ \
| * | | Reporting: add column staleAlexander Sulfrian2014-06-181-0/+2
| |/ / | | | | | | | | | | | | | | | Add a new column "stale", that displays if a host is stale or not. It could be determined by parsing the date column, but it would be more convenient to have a separate column for that.
* / / Version bump to 1.4.0pre1v1.4.0pre1Sol Jerome2014-06-161-1/+1
|/ / | | | | | | Signed-off-by: Sol Jerome <sol.jerome@gmail.com>
* | DjangoORM: Add missing importSol Jerome2014-04-251-0/+1
| | | | | | | | Signed-off-by: Sol Jerome <sol.jerome@gmail.com>
* | Merge branch 'maint'Chris St. Pierre2014-04-251-2/+2
|\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Conflicts: misc/bcfg2.spec src/lib/Bcfg2/Client/Client.py src/lib/Bcfg2/Client/Tools/APK.py src/lib/Bcfg2/Client/Tools/MacPorts.py src/lib/Bcfg2/Client/Tools/Pacman.py src/lib/Bcfg2/Client/Tools/YUM.py src/lib/Bcfg2/Server/Admin/Minestruct.py src/lib/Bcfg2/Server/Admin/Pull.py src/lib/Bcfg2/Server/Admin/Viz.py src/lib/Bcfg2/Server/Core.py src/lib/Bcfg2/Server/Plugins/Cfg/CfgEncryptedGenerator.py src/lib/Bcfg2/Server/Plugins/Cfg/CfgPrivateKeyCreator.py src/lib/Bcfg2/Server/Plugins/Properties.py src/lib/Bcfg2/settings.py src/sbin/bcfg2-crypt src/sbin/bcfg2-info src/sbin/bcfg2-lint src/sbin/bcfg2-yum-helper testsuite/Testsrc/Testlib/TestServer/TestPlugins/TestCfg/TestCfgEncryptedGenerator.py testsuite/Testsrc/Testlib/TestServer/TestPlugins/TestProperties.py
| * Reporting: Log to debug to prevent spam in logsSol Jerome2014-04-161-2/+2
| | | | | | | | Signed-off-by: Sol Jerome <sol.jerome@gmail.com>
* | Merge branch 'maint' into masterSol Jerome2014-04-163-2/+43
|\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Signed-off-by: Sol Jerome <sol.jerome@gmail.com> Conflicts: doc/appendix/guides/import-existing-ssh-keys.txt misc/bcfg2.spec src/lib/Bcfg2/Client/Tools/VCS.py src/lib/Bcfg2/Client/Tools/YUM.py src/lib/Bcfg2/Encryption.py src/lib/Bcfg2/Reporting/Collector.py src/lib/Bcfg2/Reporting/Storage/DjangoORM.py src/lib/Bcfg2/Server/Core.py src/lib/Bcfg2/Server/FileMonitor/__init__.py src/lib/Bcfg2/Server/Lint/RequiredAttrs.py src/lib/Bcfg2/Server/Plugin/helpers.py src/lib/Bcfg2/Server/Plugins/Metadata.py src/lib/Bcfg2/Server/Plugins/Packages/Yum.py src/lib/Bcfg2/Server/Plugins/Packages/__init__.py src/lib/Bcfg2/settings.py src/sbin/bcfg2-crypt src/sbin/bcfg2-reports src/sbin/bcfg2-yum-helper testsuite/Testsrc/Testlib/TestClient/TestTools/TestPOSIX/TestAugeas.py
| * bcfg2-report-collector: better error messages when failing to daemonizeChris St. Pierre2014-03-041-0/+13
| |
| * Reporting: properly close db connectionChris St. Pierre2014-02-282-4/+6
| | | | | | | | | | | | | | | | | | Close the db connection at the end of each DjangoORM import, not when the reporting collector shuts down. The collector may not have even opened a connection, in the case of a storage backend other than DjangoORM. Fixes #157
| * Version bump to 1.3.4Sol Jerome2014-02-251-1/+1
| | | | | | | | Signed-off-by: Sol Jerome <sol.jerome@gmail.com>
| * ensure that DB connections are always closed at thread/process exitChris St. Pierre2014-02-181-3/+6
| |
| * Reporting: occasionally reap child threadsMichael Fenn2014-01-101-0/+23
| | | | | | | | | | | | | | | | I noticed that, at least on Python 2.4, memory for threads doesn't get freed until the threads are joined. This patch causes the collector to periodically go through and reap those threads. Tested in production for ~1 month, no reported issues.
* | Reporting: update non-ORM bits to know about separate databaseMichael Fenn2014-04-091-4/+6
| | | | | | | | | | | | | | | | | | There is a little bit of code in the reporting web interface that uses raw SQL rather than the django ORM. This bypassed the database router functionality (since there is no model to bind to). Django supports keeping track of multiple connections in the raw SQL interface, so this commit does the refactoring necessary to support the multiple databases.
* | Remove unused importMichael Fenn2014-04-091-1/+1
| |
* | Preliminary support for seperate reporting databaseMichael Fenn2014-03-211-3/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | This commit implements a Django database router which routes each Django application to a database whose name matches a key in the database dict, falling back to the default database if no matching key is found. This support is plumbed through to the config file via database.reporting_* database connection config options. These options mirror ones available for the default database config. If database.reporting_engine is not specified in the config, then the configuration falls back to the traditional single-database way of doing things with the database router becoming a no-op.
* | Merge branch 'maint'Chris St. Pierre2013-12-093-6/+3
|\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Conflicts: doc/appendix/guides/fedora.txt misc/bcfg2.spec schemas/types.xsd src/lib/Bcfg2/Encryption.py src/lib/Bcfg2/Options.py src/lib/Bcfg2/Server/Admin/Client.py src/lib/Bcfg2/Server/Core.py src/lib/Bcfg2/Server/Lint/Validate.py src/lib/Bcfg2/Server/Plugin/helpers.py src/lib/Bcfg2/Server/Plugins/Bundler.py src/lib/Bcfg2/Server/Plugins/Cfg/CfgEncryptedGenerator.py src/lib/Bcfg2/Server/Plugins/Probes.py src/sbin/bcfg2-crypt testsuite/Testsrc/Testlib/TestServer/TestPlugin/Testhelpers.py testsuite/Testsrc/Testlib/TestServer/TestPlugins/TestCfg/TestCfgEncryptedGenerator.py testsuite/Testsrc/Testlib/TestServer/TestPlugins/TestProbes.py testsuite/common.py testsuite/install.sh
| * Reporting: Remove wildcard importsSol Jerome2013-11-163-6/+3
| | | | | | | | Signed-off-by: Sol Jerome <sol.jerome@gmail.com>
* | Merge branch 'maint'Chris St. Pierre2013-11-124-4/+22
|\| | | | | | | | | | | | | | | | | Conflicts: src/lib/Bcfg2/Server/Admin/Reports.py src/lib/Bcfg2/Server/Hostbase/hostbase/urls.py src/lib/Bcfg2/Server/Hostbase/urls.py src/sbin/bcfg2-crypt tools/upgrade/1.3/migrate_dbstats.py
| * Reports: Add missing ImportErrorSol Jerome2013-11-111-1/+1
| | | | | | | | Signed-off-by: Sol Jerome <sol.jerome@gmail.com>
| * Fix compatibility with Django 1.6.Simon Ruderich2013-11-114-4/+22
| |
* | Merge branch 'maint'Chris St. Pierre2013-11-111-1/+1
|\| | | | | | | | | | | | | | | | | | | | | | | Conflicts: src/lib/Bcfg2/Server/Admin/Compare.py src/lib/Bcfg2/Server/Admin/Snapshots.py src/lib/Bcfg2/Server/MultiprocessingCore.py src/lib/Bcfg2/Server/Plugins/Probes.py src/sbin/bcfg2-crypt src/sbin/bcfg2-reports tools/upgrade/1.3/migrate_configs.py tools/upgrade/1.3/migrate_perms_to_mode.py
| * Version bump to 1.3.3v1.3.3Sol Jerome2013-11-071-1/+1
| | | | | | | | Signed-off-by: Sol Jerome <sol.jerome@gmail.com>
* | Reporting: fixed error reportingChris St. Pierre2013-10-311-1/+1
| |
* | bcfg2-reports: rewrote with new option parserChris St. Pierre2013-10-302-1/+277
| |
* | DB: fixed how Django settings are loadedChris St. Pierre2013-10-302-14/+5
| |
* | Merge branch 'maint'Chris St. Pierre2013-10-281-9/+29
|\| | | | | | | | | | | | | | | | | | | | | Conflicts: doc/development/lint.txt misc/bcfg2.spec src/lib/Bcfg2/Reporting/Collector.py src/lib/Bcfg2/Server/Core.py src/lib/Bcfg2/Server/Plugins/Metadata.py src/lib/Bcfg2/Server/models.py testsuite/install.sh
| * Revert "Reporting: Simple sanity check to avoid creating too many threads"Michael Fenn2013-10-141-3/+0
| | | | | | | | | | | | | | | | This reverts commit f813f86f8ac2bc7b55f4eb6a0d936f1ce4f68ba7. Premature optimization is the root of all evil, etc. Conflicts: src/lib/Bcfg2/Reporting/Collector.py
| * Reporting: misc improvements to collector threadingMichael Fenn2013-10-101-4/+6
| | | | | | | | | | | | 1. Use a better convention for calling the threading.Thread constructor 2. Add docstring to ReportingStoreThread.run 3. Give the storage thread variable a better name
| * Reporting: Simple sanity check to avoid creating too many threadsMichael Fenn2013-10-091-0/+3
| |
| * Reporting: start a new thread for each importMichael Fenn2013-10-091-10/+29
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When dealing with a high-latency database connection (eg. across a WAN), the bcfg2-report-collector process can fall behind in its import queue. The imports are very much bound by the response latency of the database server and not processing throughput. This patch fires off a new thread for each database interaction. The thread itself simply falls out of scope when the interaction is finished processing. The interaction object is still read from the disk serially in order avoid having to create a locking mechanism for that part of the process. This change does potentially create greater load on the database server, but ultimately the load is limited by rate at which the bcfg2 server can generate work.
generated by cgit v1.2.3-1-g7c22 (git 2.25.1) at 2024-07-01 13:21:33 +0000