Using the Multi-thread Execution feature to run Jobs in parallel - 6.1

Talend Data Fabric Studio User Guide

EnrichVersion
6.1
EnrichProdName
Talend Data Fabric
task
Data Governance
Data Quality and Preparation
Design and Development
EnrichPlatform
Talend Studio

Based on the previous use case Using the Implicit Context Load feature, this use case give an example of how to use the Multi-thread Execution feature to run two Jobs in parallel to display the employees information in both the testing and production environments at the same time. When handling large data volumes, this feature can significantly optimize the Job execution performance of the Talend Studio.

For more information on the multi-thread execution feature, see How to execute multiple Subjobs in parallel.

Preparing Jobs to read employees data in different contexts

  1. In the Repository tree view, right-click the Job created in the use case Using the Implicit Context Load feature and select Duplicate from the context menu. Then, in the [Duplicate] dialog box enter a new name for the Job, employees_testing in this example, and click OK.

  2. Open the new Job, and label the components to better reflect their roles.

  3. Create another Job named employees_production by repeating the steps above.

  4. In the Contexts view of both Jobs, remove the db_connection variable.

  5. On Extra tab of the Job view of the Job employees_testing, fill the Table Name field of database settings with db_testing; on the Extra tab of the Job view of the Job employees_production, fill the Table Name field with db_production.

Set up a parent Job to run the Jobs in parallel

  1. Create a new Job and add two tRunJob components on the design workspace, and label the components to better reflect their roles.

  2. In the Component view of the first tRunJob component, click the [...] button next to the Job field and specify the Job it will run, employees_testing in this example.

    Configure the other tRunJob component to run the other Job, employees_production.

    For more information about tRunJob, see Talend Components Reference Guide.

  3. On the Extra tab of the Job view, select the Multi thread execution check box to activate the Multi-thread Execution feature.

Executing the Jobs

  1. Save each Job by pressing Ctrl+S.

  2. In the parent Job, press F6 of click Run on the Run view to start execution of the child Jobs.

    The child Jobs are executed in parallel, reading employees data from both databases and displaying the data on the console.

An alternative way of implementing parallel execution is using the tParallelize component. For more information, see How to orchestrate parallel executions of Subjobs and the Talend Components Reference Guide.