Skip to content
Unverified Commit b6eb458d authored by Mykhailenko's avatar Mykhailenko Committed by GitHub
Browse files

Async & cached loggers (#3531)

* Make log4j loggers asynchronous

 - Encapsulation of the log4j root logger configured appenders inside AsyncAppender(s)
 - Encapsulate all FileAppenders (Task and Job loggers) using AsyncAppenders
 - Cache already created FileAppender inside a static ConcurrentHashMap
 - make AsyncAppender buffer size configurable
 - close appenders at the end of tasks/jobs

* Close job logger before deleting log files

JobLogger can sometimes be used after the job is terminated.

In order to delete log files, the job logger must be closed systematically

* fix test java security policy file

* Improve asynchronous logging
 - improve ServerJobAndTaskLogs to limit the number of retries when trying to remove a job log folder, display the files which cannot be deleted, and automatically close associated loggers
 - improve LiveJobs and SchedulerFrontend to not use job or task loggers when not appropriate (especially when outside the job lifecycle)
 - added two new properties in PAResourceManagerProperties to configure asynchronous logging (enable or disable, use a cache).
 - improve asynchronous logging in FileAppender : do not use log4j AsyncAppender as it creates 1 thread per appender and implement instead a single-thread asynchronous logging mechanism using LinkedBlockingQueue
 - added new tests in JobLoggerTest

* Improve asynchronous logging
 - increase test sleep 
parent a57a62ad
Loading
Loading
Loading
Loading
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment