Skip to content
  • Mykhailenko's avatar
    Async & cached loggers (#3531) · b6eb458d
    Mykhailenko authored
    * Make log4j loggers asynchronous
    
     - Encapsulation of the log4j root logger configured appenders inside AsyncAppender(s)
     - Encapsulate all FileAppenders (Task and Job loggers) using AsyncAppenders
     - Cache already created FileAppender inside a static ConcurrentHashMap
     - make AsyncAppender buffer size configurable
     - close appenders at the end of tasks/jobs
    
    * Close job logger before deleting log files
    
    JobLogger can sometimes be used after the job is terminated.
    
    In order to delete log files, the job logger must be closed systematically
    
    * fix test java security policy file
    
    * Improve asynchronous logging
     - improve ServerJobAndTaskLogs to limit the number of retries when trying to remove a job log folder, display the files which cannot be deleted, and automatically close associated loggers
     - improve LiveJobs and SchedulerFrontend to not use job or task loggers when not appropriate (especially when outside the job lifecycle)
     - added two new properties in PAResourceManagerProperties to configure asynchronous logging (enable or disable, use a cache).
     - improve asynchronous logging in FileAppender : do not use log4j AsyncAppender as it creates 1 thread per appender and implement instead a single-thread asynchronous logging mechanism using LinkedBlockingQueue
     - added new tests in JobLoggerTest
    
    * Improve asynchronous logging
     - increase test sleep 
    b6eb458d