How to execute multiple Subjobs in parallel - 6.1

Talend Open Studio for Big Data User Guide

EnrichVersion
6.1
EnrichProdName
Talend Open Studio for Big Data
task
Design and Development
EnrichPlatform
Talend Studio

As explained in the previous sections, a Job opened in the workspace can contain several Subjobs and you are able to arrange their execution order using the trigger links such as OnSubjobOK. However, when the Subjobs do not have any dependencies between them, you might want to launch them at the same time. For example, the following image presents four Subjobs within a Job and with no dependencies in between.

The tRunJob component is used in this example to call each Subjob they represent. For further information about tRunJob, see Talend Open Studio for Big Data Components Reference Guide.

Then with the Job opened in the workspace, you need simply proceed as follows to run the Subjobs in parallel:

  1. Click the Job tab, then the Extra tab to display it.

  2. Select the Multi thread execution check box to enable the parallel execution.

    When the Use project settings check box is selected, the Multi thread execution check box could be greyed out and become unavailable . In this situation, clear the Use project settings check box to activate the Multi thread execution check box.

This feature is optimal when the number of threads (in general a Subjob count one thread) do not exceed the number of processors of the machine you use for parallel executions. Otherwise, some of the Subjobs have to wait until any processor is freed up.

For a use case of using this feature to run Jobs in parallel, see Using the Multi-thread Execution feature to run Jobs in parallel.