Pipeline tasks are executable integration processes that contain a pipeline artifact published from Talend Cloud Pipeline Designer.
Before you begin
- You must have Author permission on the workspace where the
task is located to create a pipeline task.
- You must have Execute permission on the workspace to run a
pipeline task on a Remote Engine Gen2.
About this task
You have the option of saving the task as a draft
at every step of the creation process. If you save an incomplete task, its
Run
type will be
No
trigger.
Procedure
-
Open the Management page.
-
Go to the Tasks tab.
-
Select the environment and the workspace using the faceted
search at the top of the page.
-
Click Add task.
-
In the Artifact step, select the
Pipeline artifact type.
-
Select the artifact you want to execute from the drop-down
list.
The details and description of the artifact
appears.
-
Select the version
of the artifact to be used in the task.
The
details and description of the artifact are refreshed
automatically.
-
Select the
workspace for the task.
-
Give a name to the
new task.
By default, the
task name is the name of the artifact from which it is created.
Note: Tasks must have a unique name within a
workspace.
- Optional:
Add a tag to the
task.
-
Select an existing tag from the drop-down list.
-
Create a new tag by typing in the name and clicking on
it in the drop-down list.
-
Click Continue.
-
In the Engine section, select the engine on which to run your
task from the Engine drop-down list.
-
Cloud Engine for Design
- A pre-configured Remote Engine Gen2
-
Select a Run Profile for
the execution.
-
Click Continue.
-
In the Schedule step, select the execution frequency.
You can add one or more triggers to schedule your task or you can manually start
the task without a trigger.
-
Click Save and close.
Results
Your task is ready to run on
Talend Cloud Management Console.
Note:
Exponential back-off retry pattern has
been adopted to provide not only faster first and second
retries (1 minute and 3 minutes respectively after the
initially failed attempt), but also better global behaviour
on peak workload situation.
For example, when you try to
execute a task that is already running, the execution fails.
Talend Cloud Management Console retries
the execution with incrementally increased intervals, that is to
say, the first retry takes place at the end of 1 minute only, the
second one 2 minutes, then subsequently 4 minutes and 8 minutes.
If the fourth retry fails, you will get the error message
Exceeded the limit of deployment attempts: another
instance of the same task was running on the engine
at same time
. Try to run the task
later.