Creating standard run profiles - Cloud

Talend Management Console User Guide

Version
Cloud
Language
English
Product
Talend Cloud
Module
Talend Management Console
Content
Administration and Monitoring > Managing projects
Administration and Monitoring > Managing users
Deployment > Deploying > Executing Pipelines
Deployment > Deploying > Executing Tasks
Deployment > Scheduling > Scheduling Tasks
Last publication date
2024-02-29

Before you begin

  • You must have Operator or Administrator rights for Talend Cloud Pipeline Designer.
  • You must set up a Remote Engine Gen2 or ensure that your subscription allows the use of the Cloud Engine for Design. For instructions, see the Talend Remote Engine Gen2 Quick Start Guide.

Procedure

  1. Open the Processing page then click the Engines tab.
  2. Click the name of the engine on which you want to configure the run profile.
  3. Click the Run profiles tab on the Engine details page.
  4. Click Add profile.
  5. Select the engine to which you want to apply the run profile.
    The current engine is selected by default.
  6. Select the Standard run profile type.
  7. Select the execution framework.
    Currently, only Spark local is supported.
  8. Enter the name of the profile.
  9. Optional: Enter the description of the run profile.
  10. In the Basic configuration section, enter the number of micro-batch intervals in milliseconds.
    The default value is 5000.
  11. Enter the number of threads the run profile can use.
  12. Optional: In the Advanced configuration section, click Add parameter to create a parameter.
  13. Enter the parameter key and value for each new parameter.

    Example

    To set the amount of memory to use per executor process, enter spark.executor.memory to the parameter key and 16g to the value fields.
  14. Click Save.

Results

The created run profile is listed on the Engines > Run profiles page in Talend Management Console. In Talend Cloud Pipeline Designer, the same run profile appears in the drop-down list of the pipeline.