Enable your Remote Engine to regularly and automatically delete useless artifacts to free up storage space.
This cleanup does not impact any ongoing task runs, because a task uses artifacts from the local Remote Engine repository only at the moment of deployment. Task-related artifacts are downloaded to this local repository when task is run for the first time. After that, these downloaded artifacts are reused for the following runs of the same task.
The highest security level of your antivirus system may impede the automatic cleanup process, for example, by preventing the deletion of temporary files. To address this issue, lowering the security level of the antivirus software can be a potential solution.
- Open the <RemoteEngineInstallationDirectory>/etc/org.talend.ipaas.rt.deployment.agent.cfg file to edit it.
Set the following parameters:
Artifacts are deleted only when both of the following conditions are satisfied:
- local.maven.clean.interval.days: the automatic cleanup is
enabled once you put a value greater than 0. For example,
Then the cleanup happens every two days.
local.maven.clean.interval.days=2Note: Ensure that there are no trailing spaces in the parameter value, as this results in engine pairing failure.
- local.maven.clean.max.gb.size: the maximum size (in GB)
allocated for artifacts to be stored in
in Remote Engine. For example,
If all the stored artifacts together exceed 8GB, when a scheduled cleanup service starts, the service deletes the oldest artifacts for which the last execution times are beyond the retention period, until the total size is reduced to be equal to or less than 8GB.
- local.maven.clean.older.than.days: the retention period
of an artifact, for example,
When you have an ongoing task expected to run for 4 days, the 3-day retention period in this example does not impact this task.
- The total size of the stored artifacts exceeds the specified limit.
- The old artifacts have their last execution times beyond specified
An artifact is downloaded to the local Remote Engine repository only when a related task is run for the first time; after that, this artifact is reused in the following runs of the same task, with its last execution time updated.
- local.maven.clean.interval.days: the automatic cleanup is enabled once you put a value greater than 0.
- Save the file.
If this message tells you that no artifact has been deleted, you may need to consider to decrease the size limit or shorten the retention period.
Remote Engine m2 artifacts cleanup performed:
- Additional dependency files such as different log4j versions are stored in <RemoteEngineInstallationDirectory>/TalendJobServersFiles/archiveJobs/.
- Temporary files related to Karaf is stored in \data\tmp\ and are automatically deleted when Karaf, along with the Remote Engine, stops running.