Assigning Remote Engines to clusters - Cloud

Talend Remote Engine User Guide for Windows
Version
Cloud
Language
English
Operating system
Windows
Product
Talend Cloud
Module
Talend Remote Engine
Content
Design and Development
Installation and Upgrade
Last publication date
2024-02-23

Before you begin

It is recommended to only add Remote Engines with the same version to a cluster.

About this task

Note: If a Route or data service is deployed on an engine you want to add to the cluster, you must first undeploy the task, then add the engine to the cluster and redeploy the task on the engine cluster.

Procedure

  1. Open the Processing page then click the Engines tab.
  2. Click the name of cluster to which you want to add Remote Engines.
    If this cluster already contains some engines, details about the engine availability is displayed on this list as well as on the Cluster details blade.
  3. Click the Edit icon on the Engines tab.

    Example

    Screenshot showing how to add a Remote Engine in Cluster Details window.
  4. Select the Remote Engines to include in the cluster.

    You can only select Remote Engines from the same environment where your cluster is.

    You can select Remote Engines from any workspace on that environment. However, if the workspaces are different, the selected Remote Engines will be placed into the cluster's workspace.

    Note: When deploying Routes and data services on a cluster, all Remote Engines assigned to it must be of the same version (v2.8.4 or above).
  5. If Job tasks are already running or scheduled on the Remote Engine you selected, choose your preferred option to update the cluster's run profile.
    You can reschedule the tasks running on the engine to run on the cluster:
    • Without a run profile
    • By duplicating the engine run profiles and associating them with the cluster
    • Using the cluster run profile

Results

The number of engines belonging to a cluster appears in brackets next to its name on the Engines tab of the Processing page.

When choosing to run a Job task on the cluster, it is executed in a round-robin manner on the engines belonging to the cluster. The task is executed on the first available Remote Engine, then the next and so on, while putting the ones the task was executed on to the end of the list. If a Remote Engine is not available, the task is executed on the next available engine.

When choosing to run a data service or Route task on the cluster, you must define a deployment strategy.
Two deployment strategies are available when running a data service or Route task on the cluster
Parallel The tasks are deployed simultaneously to all the Remote Engines of the cluster. Before deploying a new version of a task, you will need to undeploy the currently running version, which will result in service interruption.
Rolling If the Remote Engine cluster contains more than one Remote Engine, the tasks are deployed one by one to the Remote Engines of the cluster. This means that for a short time, different Remote Engines will be running different versions of the task, therefore you can avoid service interruption.