Understanding the Talend Remote Engine clean-up cycle - Cloud

Talend Cloud Installation and Upgrade Guide

Version
Cloud
Language
English
EnrichDitaval
Windows
Product
Talend Cloud
Module
Talend Artifact Repository
Talend Data Stewardship
Talend Management Console
Talend Remote Engine
Talend SAP RFC Server
Talend Studio
Content
Installation and Upgrade
Available in...

Cloud API Services Platform

Cloud Big Data

Cloud Big Data Platform

Cloud Data Fabric

Cloud Data Integration

Cloud Data Management Platform

The Talend Remote Engine cleans up Job artifacts on a schedule, based on data cleaning parameters. You can customize the Talend Remote Engine clean-up schedule by changing the parameters in the org.talend.remote.jobserver.server.cfg file, listed under Temporary data cleaning parameters. Configuring these values is optional.

Note: Make sure each Remote Engine instance has its separate TalendJobServersFiles folder, as explained in Deploying multiple Remote Engines.

General clean-up frequency

The general clean-up schedule is defined by the FREQUENCY_CLEAN_ACTION parameter. You can disable the general clean-up by setting this parameter to 0.

Job repository and archive clean-up

Job artifacts are cleaned up in the next clean-up cycle when the following conditions are met:
  • The Job is not running.
  • Either the MAX_DURATION_BEFORE_CLEANING_OLD_JOBS or MAX_OLD_JOBS parameter is met.

Job execution log clean-up

Job logs are cleaned up in the next cycle when the following conditions are met:

  • Either the MAX_DURATION_BEFORE_CLEANING_OLD_EXECUTIONS_LOGS or MAX_OLD_EXECUTIONS_LOGS parameter is met.
  • The Job execution is released. A Job is released after 50 Job executions. You can change the Job release frequency by changing the MIN_NUMBER_JOB_EXECUTIONS_BEFORE_RELEASE parameter.

A log without errors is released if the elapsed time since the start of the log is greater than the time defined in the MAX_DURATION_BEFORE_JOB_EXECUTION_RELEASE_NORMAL_CASE and MAX_DURATION_BEFORE_JOB_EXECUTION_RELEASE_ABNORMAL_CASE parameters.

Note: For each MAX_OLD_* and MAX_DURATION_* parameter pair, whichever is reached first will trigger a cleaning action.