Enabling Job monitoring on Remote Engines - Cloud

Talend Remote Engine User Guide for Linux
EnrichVersion
Cloud
EnrichProdName
Talend Cloud
EnrichPlatform
Talend Remote Engine
task
Design and Development
Installation and Upgrade

Before you begin

  • Talend Studio version 7.3 R2020-07 or later is installed but not running.
  • Talend Remote Engine version 2.9.2 is created, installed, and paired in your Talend Cloud Management Console account.

Procedure

  1. Open the <RemoteEngineInstallationDirectory>/etc/org.talend.ipaas.rt.observability.cfg file to edit it.
  2. Enable the following parameters by setting their value to true:
    • omc.enabled=true
    • omc.re.enabled=true
    • omc.jobs.enabled=true
    • omc.oshi.enabled=true
    • omc.apm.enabled=true
    • omc.prometheus.enabled=true (for a Prometheus format)
    • omc.rest.enabled=true (for a JSON format)
  3. Set the omc.push.frequency_ms parameter to the desired value.
    The push frequency parameter defines the frequency at which Remote Engine components produce their metrics. Increase its value to send metrics to the APM less often, or decrease its value for more precise monitoring.
  4. Save the file.
  5. Open the <RemoteEngineInstallationDirectory>/etc/org.talend.observability.omc.cache.apm.cfg file to edit it.
  6. Set the timeToLiveMinMilliseconds parameter to the desired value (in milliseconds).

    The time to live limit of the metrics in the Observability Metrics Collector is based on the push frequency.

    The default value is 1000. Increasing this value increases the time the metrics are available for, though it must always be higher than the push frequency.
  7. If needed, set the timeToLiveMargin to ensure there are no gaps when a component encounters an unexpected error or lag.
    The default parameter value, 5 milliseconds, can be edited but it is recommended not to edit it for security reasons.
  8. Save the file.
  9. Optional: You can also activate contextual information about the Jobs in the <RemoteEngineInstallationDirectory>/etc/org.talend.ipaas.rt.observability.jobContextConfig.cfg file by setting parameters to true.

    Example

    karaf/workspaceName=true
karaf/environmentId=false
karaf/environmentName=true
karaf/environmentVersion=true
karaf/clusterId=false
karaf/clusterName=false
karaf/planExecutionId=false
karaf/planId=false
karaf/planName=false
karaf/planStepExecutionId=false
karaf/planStepId=false
karaf/planStepName=false
karaf/triggerTimestamp=false
karaf/artifactName=true
karaf/artifactVersion=true
karaf/taskName=true
karaf/taskVersion=true
karaf/compatibilityVersion=true
karaf/runType=false
karaf/countOfAttempts=false
  10. Save the file.
  11. Start the Remote Engine.
    You can also configure remote access to the engine by following the instructions in Enabling remote access using HTTP (optional).

What to do next

To validate that the monitoring feature is enabled, enter bundle:list in the console. All Observability Metrics Collector entries must be in Active status.