Enabling Job monitoring on Remote Engines - Cloud

Talend Remote Engine User Guide for Linux
Version
Cloud
Language
English
EnrichDitaval
Linux
Product
Talend Cloud
Module
Talend Remote Engine
Content
Design and Development
Installation and Upgrade

Before you begin

  • Talend Studio version 7.3 R2020-07 or later is installed but not running.
  • Talend Remote Engine version 2.9.2 is created, installed, and paired in your Talend Cloud Management Console account.

Procedure

  1. Open the <RemoteEngineInstallationDirectory>/etc/org.talend.ipaas.rt.observability.cfg file to edit it.
  2. Enable the following parameters by setting their value to true:

    Note that omc in these parameters means Observability Metrics Collector.

    • omc.re.enabled=true: Remote Engine metrics are activated
    • omc.oshi.enabled=true: entire OS metrics are activated
    • omc.apm.enabled=true: APM appender is activated
    • omc.appender.prometheus.enabled=true: metrics appender for the Prometheus format is activated
    The following three parameters are read only:
    • omc.enabled=true: the main property activated to ensure that the whole metrics mechanism works
    • omc.jobs.enabled=true: the property activated to collect Job metrics
    • omc.appender.rest.enabled=true: metrics appender for HTTP REST format is activated
  3. Set the omc.push.frequency_ms parameter to the desired value.
    The push frequency parameter defines the frequency at which Remote Engine components produce their metrics. Increase its value to send metrics to the APM less often, or decrease its value for more precise monitoring.
  4. Optional: Depending on your license, the metrics could be displayed by default on the Run overview page in Talend Cloud Management Console.
  5. Save the file.
  6. Open the <RemoteEngineInstallationDirectory>/etc/org.talend.observability.omc.cache.apm.cfg file to edit it.
  7. Set the timeToLiveMinMilliseconds parameter to the desired value (in milliseconds).

    The time to live limit of the metrics in the Observability Metrics Collector (omc) is based on the push frequency.

    The default value is 1000. Increasing this value increases the time the metrics are available for, though it must always be higher than the push frequency.
  8. If needed, set the timeToLiveMargin to ensure there are no gaps when a component encounters an unexpected error or lag.
    The default parameter value, 5 milliseconds, can be edited but it is recommended not to edit it for security reasons.
  9. Save the file.
  10. Optional: You can also activate contextual information about the Jobs in the <RemoteEngineInstallationDirectory>/etc/org.talend.ipaas.rt.observability.jobContextConfig.cfg file by setting parameters to true.

    Example

    karaf/workspaceName=true
karaf/environmentId=false
karaf/environmentName=true
karaf/environmentVersion=true
karaf/clusterId=false
karaf/clusterName=false
karaf/planExecutionId=false
karaf/planId=false
karaf/planName=false
karaf/planStepExecutionId=false
karaf/planStepId=false
karaf/planStepName=false
karaf/triggerTimestamp=false
karaf/artifactName=true
karaf/artifactVersion=true
karaf/taskName=true
karaf/taskVersion=true
karaf/compatibilityVersion=true
karaf/runType=false
karaf/countOfAttempts=false
  11. Save the file.
  12. Start the Remote Engine.
    You can also configure remote access to the engine by following the instructions in Enabling remote access using HTTP (optional).

    Depending on your license, the metrics could be displayed by default on the Run overview page in Talend Cloud Management Console.

What to do next

To validate that the monitoring feature is enabled, enter bundle:list in the console. All Observability Metrics Collector entries must be in Active status.