Talend High Availability

Talend Big Data Platform Installation Guide for Linux

Talend Big Data Platform
Installation and Upgrade
Talend Identity and Access Management
Talend Data Stewardship
Talend SAP RFC Server
Talend DQ Portal
Talend JobServer
Talend Administration Center
Talend Log Server
Talend Studio
Talend Artifact Repository
Talend Installer
Talend CommandLine
Talend Repository Manager
Talend Activity Monitoring Console
Talend Data Preparation
Talend Runtime

You can set up a cluster in your Talend system to provide high availability and failover features for task execution scheduling in Talend Administration Center. You do this by deploying multiple Job Conductors and Job execution servers on different machines.

Note: High availability in this context refers only to the scheduling of task executions.

To summarize:

  • Two application servers (Tomcat or JBoss) holding the Talend Administration Center Job Conductors and Virtual Servers, as well as two Talend CommandLine applications are installed (on different machines) and point to the same SVN/Git shared project.

  • All instances of the application server are connected to the project administration database. This database may be clustered; refer to your corresponding database vendor documentation for more information.

  • (optional) Talend Administration Center users are routed to the same active application instance, for example through an HTTP Proxy (switch). This feature is not provided by Talend and thus needs to be implemented separately.

  • The first Talend CommandLine generates the artifacts to be deployed. The second Talend CommandLine is only used when the first one is down.

  • When an execution server fails, the other execution servers can recover the interrupted tasks.

  • A shared storage is implemented to store and share between active instances all archives and logs generated during each Job execution, for example through a Network-Attached Storage (NAS). This feature is not provided by Talend and thus needs to be implemented separately.

For more information about the failover and the various actions you can perform on a task when a server is unavailable, see the Talend Administration Center User Guide.