Creating pipeline tasks - Cloud

Talend Cloud Management Console for Pipelines User Guide
Version
Cloud
Language
English
Product
Talend Cloud
Module
Talend Pipeline Designer
Content
Administration and Monitoring > Managing projects
Administration and Monitoring > Managing users
Deployment > Deploying > Executing Tasks
Deployment > Scheduling > Scheduling Tasks
Pipeline tasks are executable integration processes that contain a pipeline artifact published from Talend Cloud Pipeline Designer.

Before you begin

  • You must have Author permission on the workspace where the task is located to create a pipeline task.
  • You must have Execute permission on the workspace to run a pipeline task on a Remote Engine Gen2.

About this task

You have the option of saving the task as a draft at every step of the creation process. If you save an incomplete task, its Run type will be No trigger.

Procedure

  1. Open the Management page.
  2. Go to the Tasks tab.
  3. Select the environment and the workspace using the faceted search at the top of the page.
  4. Click Add task.
  5. In the Artifact step, select the Pipeline artifact type.
  6. Select the artifact you want to execute from the drop-down list.
    The details and description of the artifact appears.
  7. Select the version of the artifact to be used in the task.
    The details and description of the artifact are refreshed automatically.
  8. Select the workspace for the task.
  9. Give a name to the new task.
    By default, the task name is the name of the artifact from which it is created.
    Note: Tasks must have a unique name within a workspace.
  10. Optional: Add a tag to the task.
    1. Select an existing tag from the drop-down list.
    2. Create a new tag by typing in the name and clicking on it in the drop-down list.
  11. Click Continue.
  12. In the Engine section, select the engine on which to run your task from the Engine drop-down list.
    • Cloud Engine for Design
    • A pre-configured Remote Engine Gen2
  13. Select a Run Profile for the execution.
  14. Click Continue.
  15. In the Schedule step, select the execution frequency.
    You can add one or more triggers to schedule your task or you can manually start the task without a trigger.
    For more information, see Scheduling pipeline tasks.
  16. Click Save and close.

Results

Your task is ready to run on Talend Cloud Management Console.

Note:

Exponential back-off retry pattern has been adopted to provide not only faster first and second retries (1 minute and 3 minutes respectively after the initially failed attempt), but also better global behaviour on peak workload situation.

For example, when you try to execute a task that is already running, the execution fails. Talend Cloud Management Console retries the execution with incrementally increased intervals, that is to say, the first retry takes place at the end of 1 minute only, the second one 2 minutes, then subsequently 4 minutes and 8 minutes.

If the fourth retry fails, you will get the error message Exceeded the limit of deployment attempts: another instance of the same task was running on the engine at same time. Try to run the task later.

Example