Job execution management: registration and release - 8.0

Job server rate computation

Version
8.0
Language
English
Product
Talend Big Data
Talend Big Data Platform
Talend Data Fabric
Talend Data Integration
Talend Data Management Platform
Talend Data Services Platform
Talend ESB
Talend MDM Platform
Talend Real-Time Big Data Platform
Module
Talend Administration Center
Content
Last publication date
2024-02-06
This document describes how Talend JobServer internally registers Job executions. An internal component JobsManager maintains information about jobs and job executions.

The Talend JobServer is a probe running on the execution server. It will measure some features of the execution server:

  • The free disk space in kilobytes
  • The free physical memory in bytes
  • The free swap memory in bytes
  • The idle CPU usage as a percentage between 0.0 and 1.0
  • The nice CPU usage as a percentage between 0.0 and 1.0
  • The total CPU usage as a percentage between 0.0 and 1.0
  • The minimum required number of CPUs

This information is sent to Talend Administration Center which then computes a rate value for this server.

The server rating ranges from 0 to 100:
  • 100: server availability is optimal
  • >100: one or several indicators are out of the defined bounds
    In Talend Administration Center, the rating is represented by stars (from 0 to 5) in the Rank item of the Talend JobServer.

Some features are more important than the other. You can weight these features in the monitoring_client.properties file.

Let's call weight{i} the weight of the ith feature.

You can choose the range in which the feature is supposed to be good enough. This means you set some limits to be fulfilled by the feature of the server. For instance, a server is not a good server for the execution of the Job if it does not have 1Go of disk space. The lower limit on the disk would therefore be 1Go (to be set in the monitoring_client.properties file). Let's call Min{i} the lower limit defined on feature i and Max{i} its upper limit.

The server has an actual value for each feature. For instance, the server i has only 500 MB of free disk space. Let's call this value the actual value of the feature: value{i}.

The basic assumption is that the server is perfect as long as all of its features have actual values in the range defined by the limits. When some of its features have values outside the defined ranges, the server is not very good.