summaryrefslogtreecommitdiffstats
path: root/jobs/workers.go
Commit message (Collapse)AuthorAgeFilesLines
* DeleteAll for KV (#9431)Daniel Schalla2018-10-101-0/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Expire K/V Values Regenerate Code pathfix Update Expiry on Update Check for Exit Signal gofmt Rewrote Go Routine Remove tempoarily cleanup loop fix expiretime TEST: Expired Watchdog as GoRoutine Check if Srv is nil Use Scheduler/Worker for Expired Key CleanUp add license fix scheduler job type; DoJob Restructuring Remove unused imports and constants move db migration from 5.4 to 5.5
* Move HTTPService and ConfigService into services package (#9422)Harrison Healey2018-09-261-1/+2
| | | | | | | | * Move HTTPService and ConfigService into utils package * Re-add StaticConfigService * Move config and http services into their own packages
* MM-10269: Make jobserver work without restart. (#9005)George Goldberg2018-06-261-0/+2
|
* MM-9728: Online migration for advanced permissions phase 2 (#8744)George Goldberg2018-05-141-0/+13
| | | | | | | | | | | | | | | | | | | | | | | | * MM-9728: Online migration for advanced permissions phase 2 * Add unit tests for new store functions. * Move migration specific code to own file. * Add migration state function test. * Style fixes. * Add i18n strings. * Fix mocks. * Add TestMain to migrations package tests. * Fix typo. * Fix review comments. * Fix up the "Check if migration is done" check to actually work.
* Structured logging (#8673)Christopher Speller2018-04-271-3/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Implementing structured logging * Changes to en.json to allow refactor to run. * Fixing global logger * Structured logger initalization. * Add caller. * Do some log redirection. * Auto refactor * Cleaning up l4g reference and removing dependancy. * Removing junk. * Copyright headers. * Fixing tests * Revert "Changes to en.json to allow refactor to run." This reverts commit fd8249e99bcad0231e6ea65cd77c32aae9a54026. * Fixing some auto refactor strangeness and typo. * Making keys more human readable.
* Remove global cfg vars (#8099)Chris2018-01-121-17/+16
| | | | | | * remove global cfg vars * enterprise update
* PLT-7503: Create Message Export Scheduled Task and CLI Command (#7612)Jonathan2017-11-301-0/+21
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Created message export scheduled task * Added CLI command to immediately kick off an export job * Added email addresses for users joining and leaving the channel to the export * Added support for both MySQL and PostgreSQL * Fixing gofmt error * Added a new ChannelMemberHistory store and associated tests * Updating the ChannelMemberHistory channel as users create/join/leave channels * Added user email to the message export object so it can be included in the actiance export xml * Don't fail to log a leave event if a corresponding join event wasn't logged * Adding copyright notices * Adding message export settings to daily diagnostics report * Added System Console integration for message export * Cleaned up TODOs * Made batch size configurable * Added export from timestamp to CLI command * Made ChannelMemberHistory table updates best effort * Added a context-based timeout option to the message export CLI * Minor PR updates/improvements * Removed unnecessary fields from MessageExport object to reduce query overhead * Removed JSON functions from the message export query in an effort to optimize performance * Changed the way that channel member history queries and purges work to better account for edge cases * Fixing a test I missed with the last refactor * Added file copy functionality to file backend, improved config validation, added default config values * Fixed file copy tests * More concise use of the testing libraries * Fixed context leak error * Changed default export path to correctly place an 'export' directory under the 'data' directory * Can't delete records from a read replica * Fixed copy file tests * Start job workers when license is applied, if configured to do so * Suggestions from the PR * Moar unit tests * Fixed test imports
* PLT-5813 support SAML sync via LDAP (#7668)Corey Hulen2017-10-311-4/+4
| | | | | | * PLT-5813 support SAML sync via LDAP * Cleaning up based on review
* Remove more global refs / state (#7723)Chris2017-10-261-9/+12
| | | | | | | | | | * remove more global refs / state * fix job enterprise initialization * fix api4 test compilation * saml api endpoints fix
* PLT-7705: API to get data retention policy. (#7539)George Goldberg2017-10-021-2/+2
| | | | | | * PLT-7705: API to get data retention policy. * Fix review comments.
* remove jobs.Srv and other jobs-related globals (#7535)Chris2017-09-291-7/+6
|
* PLT-7542 Converting LDAP sync to the job server (#7452)Corey Hulen2017-09-251-4/+25
| | | | | | | | | | | | | | | | | | | | * PLT-7542 Converting LDAP sync to the job server * Fixing minor issues * Fixing build failure * Translate error message * Translate error message * Translate error message * Translate error message * Fixing merge * Fixing bad merge
* PLT-6558: Basic data retention job scheduler/worker implementation. (#7449)George Goldberg2017-09-151-4/+4
| | | | | | * PLT-7639: Batch delete methods for data retention. * PLT-6558: Basic data retention job worker/scheduler implementation.
* Renaming repoChristopher Speller2017-09-061-3/+3
|
* PLT-7302: Aggregate Elasticsearch indexes over a certain age. (#7224)George Goldberg2017-08-171-5/+24
| | | | | | | | | * PLT-7302: Aggregate Elasticsearch indexes over a certain age. This is done by a scheduled daily job, in order to keep the shard count to a sensible level in Elasticsearch. * Use map[string]string instead of StringMap
* PLT-7040: Fix capitalisation of Elasticsearch in Config. (#7020)George Goldberg2017-07-271-4/+4
|
* PLT-6475: Elasticsearch Indexing Worker. (#6879)George Goldberg2017-07-111-7/+24
|
* PLT-6916: Redesign the jobs package and Jobserver. (#6733)George Goldberg2017-07-071-0/+79
This commit redesigns the jobserver to be based around an architecture of "workers", which carry out jobs of a particular type, and "jobs" which are a unit of work carried by a particular worker. It also introduces "schedulers" which are responsible for scheduling jobs of a particular type automatically (jobs can also be scheduled manually when apropriate). Workers may be run many times, either in instances of the platform binary, or the standalone jobserver binary. In any mattermost cluster, only one instance of platform OR jobserver must run the schedulers. At the moment this is controlled by a config variable, but in future will be controlled through the cluster leader election process.