Before you begin
- You must have Operator or Administrator rights for Talend Cloud Pipeline Designer.
- You must set up the Remote Engine Gen2 or ensure that your subscription allows the use of the Cloud Engine for Pipelines. For instructions on setting up the remote engine, see the Talend Remote Engine Gen2 Quick Start Guide.
- Go to the Engines tab.
- Click RUN PROFILES.
- Click ADD PROFILE.
- Select the Standard run profile type.
- Select the engine to which you want to apply the run profile.
Select the execution framework.
Currently, only Spark local is supported.
- Enter the name of the profile.
- Optional: Enter the description of the run profile.
In the BASIC
CONFIGURATION section, enter the number of micro-batch intervals
The default value is 5000.
- Enter the number of threads the run profile can use.
- Optional: In the Advanced configuration section, click ADD PARAMETER to create a parameter.
Enter the parameter key and value for each new parameter.
ExampleTo set the amount of memory to use per executor process, enter spark.executor.memory to the parameter key and 16g to the value fields.
- Click SAVE.
The created run profile is listed on the Talend Cloud Management Console. In Talend Cloud Pipeline Designer, the same run profile appears in the drop-down list of the pipeline.page in