The following metrics are provided by Talend Management Console through Talend Remote Engine and can be integrated in your Application Performance Monitoring (APM) tool to observe your Jobs.
The metrics below are captured only when a Job run is started in Talend Management Console.
Process metrics are sent by Prometheus by default.
Operating system metrics
| Parameter | Description |
|---|---|
| os_cpu_cores_total | Number of logical CPUs available |
| os_cpu_usage | Operating system's real-time CPU usage |
| os_system_load_average | Average number of processes that are executed by the CPU calculated over a given period of time |
| os_memory_bytes_total | Amount of actual physical memory (in bytes) |
| os_memory_bytes_available | Amount of physical memory currently available (in bytes) |
| os_filestore_bytes_total | Amount of actual physical filestore (in bytes) |
| os_filestore_bytes_available | Amount of actual physical filestore currently available (in bytes) |
Business metrics
| Parameter | Category | Description |
|---|---|---|
| component_error | Components | Error information of a component when a Job fails on this component Note: Minimum requirement is Talend Studio 8.0.1-2022-04 patch and Talend Remote Engine v2.12.1 onwards.
|
| component_connection_rows_total | Components | Total number of rows treated by a Job's component (generated for components that are managing rows) |
| component_start_time_seconds | Components | Time stamp indicating the component start time (any component in the
Job) Note: Minimum requirement is Talend Studio 7.3 R2020-11 patch and Talend Remote Engine v2.9.4 (or later versions).
|
| component_execution_duration_milliseconds | Components | Amount of time it takes to execute a Job component (generated for components that are managing rows) |
| job_execution_duration_seconds | Job | Amount of time it takes to execute a Job |
| job_start_time_seconds | Job | Time stamp indicating the execution start time |
| re_execution_count_max | Remote Engine | Maximum number of tasks a Remote Engine can execute in parallel |
| re_execution_count_current | Remote Engine | Number of tasks currently deployed and executed by the Remote Engine |
| re_logs_directory_bytes_used | Remote Engine | Size of the log directory of the Remote Engine |
| re_jobserver_directory_bytes_used | Remote Engine | Size of the JobServer directory in the Remote Engine |
To determine when the Job execution ends, add the values of the job_execution_duration_seconds and job_start_time_seconds parameters. Similarly, to determine when the component execution ends, add the values of the component_start_time_seconds and component_execution_duration_milleseconds.
JMX metrics
| Parameter | Description |
|---|---|
| jvm_memory_bytes_used | Used bytes of a given JVM memory area |
| jvm_memory_bytes_committed | Committed (bytes) of a given JVM memory area |
| jvm_memory_bytes_max | Max (bytes) of a given JVM memory area |
| jvm_memory_bytes_init | Initial bytes of a given JVM memory area |
| jvm_start_time_milliseconds | Time stamp indicating the process start time |
| jvm_process_cpu_time | Total user and system CPU time spent (in seconds) |
| jvm_process_cpu_load | Recent CPU usage |
JVM arguments not set by default can be added manually to the <RemoteEngineInstallationDirectory>/etc/org.talend.ipaas.rt.observability.cfg with the omc.jvm.allowlist= parameter, separated by commas.
| Parameter | Description |
|---|---|
| jvm_memory_pool_allocated_bytes_total | Total bytes allocated in a given JVM memory pool (only updated after GC, not continuously) |
| jvm_buffer_pool_used_bytes | Used bytes of a given JVM buffer pool |
| jvm_buffer_pool_capacity_bytes | Bytes capacity of a given JVM buffer pool |
| jvm_buffer_pool_used_buffers | Used buffers of a given JVM buffer pool |
| jvm_memory_pool_bytes_used | Used bytes of a given JVM memory pool |
| jvm_memory_pool_bytes_committed | Committed bytes of a given JVM memory pool |
| jvm_memory_pool_bytes_max | Max bytes of a given JVM memory pool |
| jvm_memory_pool_bytes_init | Initial bytes of a given JVM memory pool |
| jvm_threads_current | Current thread count of a JVM |
| jvm_threads_daemon | Daemon thread count of a JVM |
| jvm_threads_peak | Peak thread count of a JVM |
| jvm_threads_started_total | Started thread count of a JVM |
| jvm_threads_deadlocked | Cycles of JVM-threads that are in deadlock waiting to acquire object monitors or ownable synchronizers |
| jvm_threads_deadlocked_monitor | Cycles of JVM-threads that are in deadlock waiting to acquire object monitors |
| jvm_threads_state | Current count of threads by state |
| jvm_info | JVM version info |
| jvm_classes_loaded | The number of classes that are currently loaded in the JVM |
| jvm_classes_loaded_total | The total number of classes that have been loaded since the JVM has started execution |
| jvm_classes_unloaded_total | The total number of classes that have been unloaded since the JVM has started execution |
| jvm_gc_collection_seconds | Time spent in a given JVM garbage collector (in seconds) |