Creating data service tasks - Cloud

Talend Cloud Management Console User Guide
Version
Cloud
Language
English
Product
Talend Cloud
Module
Talend Management Console
Content
Administration and Monitoring > Managing projects
Administration and Monitoring > Managing users
Deployment > Deploying > Executing Pipelines
Deployment > Deploying > Executing Tasks
Deployment > Scheduling > Scheduling Tasks
Last publication date
2023-10-04
Data service tasks are executable integration processes that contain a service published from Talend Studio.

Before you begin

Data service tasks can be executed only on remote engines that have a microservice or Talend Runtime (OSGi) run profile configured.

About this task

You can deploy and undeploy data services manually through the user interface. Data services cannot be scheduled or added to plans; this type of service runs until it is explicitly undeployed by the operator.

Procedure

  1. Open the Management page.
  2. Go to the Tasks tab.
  3. Select the environment and the workspace using the faceted search at the top of the page.
  4. Click Add task.
  5. Select the Data service artifact type.
  6. Select the artifact you want to deploy from the drop-down list.
    The details and description of the artifact appears.
  7. Select the version of the artifact to be used in the task.

    You can select Always use the latest available artifact version if you want your task to be updated automatically each time a new version of the artifact is published. You can also select a specific version and update it manually as needed.

    The details and description of the artifact are refreshed automatically.
  8. Select the workspace for the task.
  9. Give a name to the new task.
    By default, the task name is the name of the artifact from which it is created.
    Note: Tasks must have a unique name within a workspace.
  10. Optional: Add a tag to the task.
    1. Select an existing tag from the drop-down list.
    2. Create a new tag by typing in the name and clicking on it in the drop-down list.
  11. Optional: Enter the description of the task.
  12. Click Continue.
  13. In the Parameters step, enter the value for the parameters of the data service.
    In this step, when the Show order as in Studio toggle is disabled, you can only set the values to parameters you have defined with the parameter_ prefix in Studio.
    Note: Use \\ instead of \ when the value could be interpreted as a special character, for example, \t.
  14. Optional: In the Advanced parameters step, enter the values for the parameters you have defined in Studio without prefixes, and connection_ parameters that are not defined in the data service. This section is available when the Show order as in Studio toggle is disabled. Otherwise, all parameters are in the Parameters step.
    Note: Use \\ instead of \ when the value could be interpreted as a special character, for example, \t.
  15. Optional: In the Connections step, select the appropriate connection from the drop-down list or create a new connection based on the parameters already defined in the artifact.
  16. Optional: In the Resources step, select the appropriate resource to use in the task.
  17. In the Engine step, select the execution runtime on which to deploy your task from the Engine drop-down list.
    Option Description
    Remote Engine Data services and Routes can only be executed on Remote Engines that are compatible with the used Studio version. For the compatibility matrix, see Artifact compatibility for Cloud Engine and Talend Remote Engine.

    Depending on the type of the task, the selected Remote Engine must have the corresponding run profile set either when the engine is created, or later in the Edit engine wizard accessible via the Engines > <RemoteEngineName> > path.
    Remote Engine cluster All Remote Engines assigned to the cluster must be of the same version (v2.8.4 or above).

    Depending on the type of the task, the Remote Engines in the cluster must have the appropriate Microservice or Talend Runtime (OSGi) run profile set. The engines must also be compatible with the used artifact's version.
  18. Optional: When deploying a microservice on a Remote Engine or cluster, you can select a run profile.

    You can choose a previously created run profile for the selected Remote Engine or cluster to set the JVM parameters for the task run.

    This feature is supported from Remote Engine 2.12.4 onwards.

  19. In the Metric Path field, enter the metric path to be used at deployment time to retrieve metric data, for example, the number of requests at runtime.
    The default value is the artifact name, and we recommend to keep this value to see the metric data displayed on the Task Details page. If this field is empty, no metric information will be displayed on the page.
    Tip: You can add the project name to the artifact name, if artifacts with the same name exist in more than one project and are configured to run on the same Talend Remote Engine/Talend Runtime.
  20. Optional: When deploying a microservice on a Remote Engine or cluster, you can run your task as another user by entering their username in the Run as impersonated user field.
    This feature is supported only if the Remote Engine is installed on Unix or similar environments.
    This feature is supported from Remote Engine 2.12.4 onwards.
    For more information about user impersonation, see the Talend Remote Engine User Guide.
  21. When deploying a microservice, set the port assignment.
    Option Description
    Automatic The port on which the microservice instance(s) will run is defined by the system.
    Custom You can set a static port number on which to run all instances of the microservice, making external load balancing easier.
  22. If you have chosen to deploy the task on a cluster, select the deployment strategy.
    Option Description
    Parallel Deploy the tasks simultaneously to all the Remote Engines of the cluster.
    Rolling Deploy the tasks one by one to the Remote Engines of the cluster.
  23. Save your changes.
    You can deploy the task from the Task Details page by clicking Deploy.

Results

Your task is ready to run on Talend Cloud Management Console.

Note:

Exponential back-off retry pattern has been adopted to provide not only faster first and second retries (1 minute and 3 minutes respectively after the initially failed attempt), but also better global behaviour on peak workload situation.

For example, when you try to execute a task that is already running, the execution fails. Talend Cloud Management Console retries the execution with incrementally increased intervals, that is to say, the first retry takes place at the end of 1 minute only, the second one 2 minutes, then subsequently 4 minutes and 8 minutes.

If the fourth retry fails, you will get the error message Exceeded the limit of deployment attempts: another instance of the same task was running on the engine at same time. Try to run the task later.

Example