Monitoring the resource consumption on Remote Engine - Cloud

Talend Remote Engine User Guide for Linux

Version
Cloud
Language
English
Operating system
Linux
Product
Talend Cloud
Module
Talend Remote Engine
Content
Design and Development
Installation and Upgrade
Last publication date
2024-02-23

Resource consumption of your engine machine is logged in RE_installation/data/log/statistics.csv.

This gives you the visibility about the resource consumption level of each Remote Engine.

Set the monitoring thresholds on your Remote Engine to trigger errors and warnings when a given resource limitation is reached.

Before you begin

Your Remote Engine must be v2.12.4 and onwards.

Procedure

  1. Open the RE_installation/etc/org.talend.ipaas.rt.deployment.agent.cfg file.
  2. Check the following parameters and their default values, or adapt the values.

    Example

    env.stat.log.enabled=true
    env.stat.log.warn.cpu.level.pct=80
    env.stat.log.error.cpu.level.pct=95
    env.stat.log.warn.memory.free.mb=1024
    env.stat.log.error.memory.free.mb=512
    env.stat.log.warn.disk.free.mb=1024
    env.stat.log.error.disk.free.mb=512
  3. Save the file.

Results

The log file contains the following information about your engine machines:
  • CPU
  • Memory
  • Disk usage
It contains the following fields:
  • date: log entry date time
  • level: cumulative log entry criticality (INFO, WARN or ERROR) based on the thresholds configured as explained above for metrics
  • task-count: number of Talend Management Console tasks executed on the engine
  • task-max: maximum number of task allowed to be executed on the engine (0 or lower means no limitation)
  • cpu: CPU load
  • cpu-status: CPU load criticality based on the thresholds configured as explained above (WARN, ERROR or empty)
  • memory-used: used memory on an engine machine
  • memory-max: total available memory
  • memory-status: available memory criticality based on the thresholds configured as explained above (WARN, ERROR or empty)
  • disk-engine: the size of the engine installation directory
  • disk-used: used size of the disk
  • disk-max: total disk size
  • disk-status: available disk space criticality based on the thresholds configured as explained above (WARN, ERROR or empty)