- Apr 29, 2019
-
-
Mykhailenko authored
* Make log4j loggers asynchronous - Encapsulation of the log4j root logger configured appenders inside AsyncAppender(s) - Encapsulate all FileAppenders (Task and Job loggers) using AsyncAppenders - Cache already created FileAppender inside a static ConcurrentHashMap - make AsyncAppender buffer size configurable - close appenders at the end of tasks/jobs * Close job logger before deleting log files JobLogger can sometimes be used after the job is terminated. In order to delete log files, the job logger must be closed systematically * fix test java security policy file * Improve asynchronous logging - improve ServerJobAndTaskLogs to limit the number of retries when trying to remove a job log folder, display the files which cannot be deleted, and automatically close associated loggers - improve LiveJobs and SchedulerFrontend to not use job or task loggers when not appropriate (especially when outside the job lifecycle) - added two new properties in PAResourceManagerProperties to configure asynchronous logging (enable or disable, use a cache). - improve asynchronous logging in FileAppender : do not use log4j AsyncAppender as it creates 1 thread per appender and implement instead a single-thread asynchronous logging mechanism using LinkedBlockingQueue - added new tests in JobLoggerTest * Improve asynchronous logging - increase test sleep
-
- Oct 10, 2018
-
-
Mykhailenko authored
Add REST end point which returns job XML (as it was submitted) by job id .
-
- Mar 05, 2018
-
-
Justine Rochas authored
* Separate createNodeSource into two steps - deprecate createNodeSource (RMCore, RMRest, CLI) - create defineNodeSource and deployNodeSource (RMCore, RMRest, CLI) - policy and NodeSource are instantiated at node source definition but they are not active until the node source is deployed. The NodeSource has now a reference towards its active policy and a reference towards its non active policy - initialize node source permissions when activity starts - maintain list of defined and deployed node sources - add node source defined event - add attribute in RMNodeSourceEvent for the node source status - delete test that no longer apply with the node source changes * Adapt to recovery - lighten RMCore class by moving recovery logic - fix nodes recovery and create two different lists for deployed and undeployed node sources * Refactor node source parameters - use Serializable type for node source parameters - refactor parameter merging for the infrastructure and the policy
-
- Jan 25, 2018
-
-
Justine Rochas authored
* Specify nodes recovery per node source - add nodesRecoverable attribute of the node source, persisted in database - duplicate the REST endpoint createNodeSource, add boolean nodesRecoverable at the end of parameters - check this attribute along with the global nodes recovery property, but the global property is considered in priority - when task runs, check whether task runs on recoverable node, if not, do not let it rebind to a new task terminate notification - when the scheduler is started, reset to pending the tasks that are running on nodes that cannot be recovered - remove specific cases for the default DefaultInfrastructureManagaer created and for the default LocalInfrastrucure * Update createns CLI command - add optional String parameter to specify if the node source nodes are recoverable upon scheduler crash. User should now pass any case sensitive falvours of 'true' to have recoverable nodes, or any other string to disable nodes recoverability of the new node source. - add new test class for CLI createns * Apply node source recovery when nodes recovery is disabled - when nodes recovery is disabled, node source recovery should still be applied, to enable the nodes of the node source to be recreated. Before this commit the nodes recovery property was checked before recovering node sources, hence node sources could not be recovered. - add four test that consider whether nodes recovery is globally activated + whether nodes recovery per node source is activated. The nodes must not be recreated during recovery only in the case where the global nodes recovery and the nodes recovery of the node source are both activated (assuming that the nodes are still alive). The other combination should lead to a recreation of the nodes. - revert SchedulerTasksStateRecoverIntegrationTest and TestLoadSchedulerClientState to the verification they did before the task recovery was introduced, because the nodes recovery is now disabled for the node source used in standard tests. - apply sonar fixes
-
- Jan 22, 2018
-
-
Justine Rochas authored
* Prevent setNodesAvailable to be called for not-yet-available nodes - factorize allNodes.put calls in a single method RMCore#makeNodeAvailable that is accessible from the node source and whose returned future is waited on on the node source side, to wait for the node to be added in the map. - make allNodes map a ConcurrentHashMap because of its access in ImmediateService (in setNodesAvailable) - in case of recovery, wait for the RM to have its initActivity finished before letting 'setNodesAvailable' (immediate service) proceed
-
- Sep 28, 2016
-
-
Marco Castigliego authored
Adding HandleOnlyMyJobsPermission as generic permission to execute methods of the SchedulerFrontend on other user's jobs
-
- May 02, 2014
-
-
Vladimir Bodnartchouk authored
Change-Id: I5640560712cd412ade4da0cc5842a60822620ddc
-
- Mar 31, 2014
-
-
Youri Bonnaffé authored
Change-Id: Iedd40c4b552cd9fe4fd284836d99bec0c9bb1b41
-
- Aug 30, 2013
-
-
Youri Bonnaffé authored
add config files (to be moved somewhere else someday) set proper classpath for test helper move resources file to proper folder Change-Id: I4f270bd200ecc6c37d054d8c6442688b4ccd870e
-
- Aug 28, 2013
-
-
Youri Bonnaffé authored
Change-Id: I77baf97d409526381d0b04ad45573b9104a000ce
-
- Dec 14, 2012
-
-
Sanka Samaranayake authored
Added REST API functional tests. Change-Id: Ie21fbb17e0bdc4845e2e93ef2fe3c19fb7d6fff1
-