Creating Big Data run profiles - Cloud

Talend Cloud Management Console for Pipelines User Guide

EnrichVersion
Cloud
EnrichProdName
Talend Cloud
EnrichPlatform
Talend Management Console
task
Administration and Monitoring > Managing projects
Administration and Monitoring > Managing users
Deployment > Deploying > Executing Tasks
Deployment > Scheduling > Scheduling Tasks

Before you begin

About this task

Note: To be able to use Spark and Hadoop with Talend Cloud Pipeline Designer, you must have a Talend Cloud subscription with Big Data.

Procedure

  1. Go to the Engines tab.
  2. Click the name of the engine on which you want to configure the run profile.
  3. Click the Run profiles tab on the Engine details page.
  4. Click ADD PROFILE.
  5. Select the engine to which you want to apply the run profile.
    The current engine is selected by default.
  6. Select the Big Data run profile type.
  7. Select the Spark on Yarn execution framework.
  8. Select the EMR 5.x (Hadoop 2.7) distribution.
  9. Enter the name of the profile.
  10. Optional: Enter the description of the run profile.
  11. Configure the run profile attributes.
  12. Click SAVE.

Results

The created run profile is listed on the Engines > RUN PROFILES page in Talend Cloud Management Console. In Talend Cloud Pipeline Designer, the same run profile appears in the drop-down list of the pipeline.