Before you begin
- You must have Operator or Administrator rights for Talend Cloud Pipeline Designer.
- You must set up a Remote Engine Gen2 or
ensure that your subscription allows the use of the Cloud Engine for Design.
For instructions, see the Talend Remote Engine Gen2 Quick Start Guide.
Procedure
-
Open the Processing page then click the
Engines tab.
-
Click the name of
the engine on which you want to configure the run profile.
-
Click the
Run profiles tab on the Engine details page.
-
Click Add
profile.
-
Select the engine
to which you want to apply the run profile.
The current
engine is selected by default.
-
Select the Standard run
profile type.
-
Select the execution framework.
Currently, only Spark local is supported.
-
Enter the name of
the profile.
- Optional:
Enter the
description of the run profile.
-
In the Basic
configuration section, enter the number of micro-batch intervals
in milliseconds.
The default value is 5000.
-
Enter the number of threads the run profile can use.
- Optional:
In the Advanced
configuration section, click Add parameter
to create a parameter.
-
Enter the parameter
key and value for each new parameter.
Example
To set the
amount of memory to use per executor process, enter spark.executor.memory to the parameter key and 16g to the value fields.
-
Click Save.
Results
The created run profile is listed on the page in Talend Cloud Management Console. In Talend Cloud Pipeline Designer, the same run profile appears in
the drop-down list of the pipeline.