Deploying a Job or a Route in cluster mode - 6.3

Talend Administration Center User Guide

Talend Big Data
Talend Big Data Platform
Talend Data Fabric
Talend Data Integration
Talend Data Management Platform
Talend Data Services Platform
Talend ESB
Talend Real-Time Big Data Platform
Talend Administration Center
Administration and Monitoring

In order to deploy your Job in cluster mode, you need to have grouped your physical servers in a virtual server via the Virtual Servers page, as explained in Configuring virtual servers.

Note about physical servers: For load balancing, the physical servers of the Virtual Servers page are ordered according to the availability of their resources:

  • if one of the physical servers of a specific virtual server has the best rate, this physical server will be used for the executions of the tasks.

  • if all physical servers have the same rate, the first physical server of the virtual server will be used for the executions of the tasks.

For more information on the server rates (based on disk space, CPU and RAM usage, etc.) which determine which physical server to be used for the executions and how to edit these weight values, see Configuring execution servers and the Talend Installation Guide.

This clustered server will be used to distribute the incoming requests (Job generation, deployment and execution) between the physical servers and to ensure that these requests are processed even if one physical server is down.

How to deploy a Job on a clustered server

  1. On the Job Conductor page, add an execution task as explained in Adding a Normal execution task.

    In the Execution server list, select the virtual server which groups your physical servers.

    In the On unavailable JobServer list of the form, select the action that should be performed in case one server is down. You can wait, reset, restart or recover the task.

  2. Generate, deploy and run the task, either manually using the corresponding buttons on the top toolbar, or automatically via a trigger.

One of the servers grouped in your virtual server, Server_China, was down during the execution, but the load balancing feature allowed you to redirect the request automatically to another available server, Server_France, based on the ranking of parameters in the execution server configuration (CPU, RAM, disk usage).

If no servers are available when the request is sent but you have initiated as "checkpoints" one or several OnSubjobOk trigger connections, you can access the Error recovery Management page in Talend Administration Center and recover the Job execution at a selected checkpoint. For more information, see Recovering the execution of a Job.

How to deploy a Route or a Service on a clustered server

The Virtual servers can be used to deploy ESB artifacts to clustered containers.

  • On the ESB Conductor page, add the Route or Service you want to deploy as explained in Adding an execution task, and deploy it in the virtual server which groups your physical containers.