Parallelizing/synchronizing subJobs execution - Cloud - 8.0

Orchestration (Integration)

Talend Big Data
Talend Big Data Platform
Talend Data Fabric
Talend Data Integration
Talend Data Management Platform
Talend Data Services Platform
Talend ESB
Talend MDM Platform
Talend Open Studio for Big Data
Talend Open Studio for Data Integration
Talend Open Studio for ESB
Talend Real-Time Big Data Platform
Talend Studio
Data Governance > Third-party systems > Orchestration components (Integration)
Data Quality and Preparation > Third-party systems > Orchestration components (Integration)
Design and Development > Third-party systems > Orchestration components (Integration)

This scenario applies only to subscription-based Talend products.

For more technologies supported by Talend, see Talend components.

The following simple scenario creates a five-component main Job that uses one tParallelize component with four tMsgBox single-component subJobs. The tMsgBox_1 component is the trigger subJob. The tParallelize_1 component executes tMsgBox_2 and tMsgBox_3 simultaneously, and then synchronizes tMsgBox_4 to be executed at the end of the simultaneous execution of the subJobs.

  • Drop four tMsgBox components from the Palette to the design workspace.

  • Define their dialog box display properties as desired.

For more information on defining tMsgBox properties, see tmsgbox_c.html.

  • Drop a tParallelize component onto the design workspace.

  • Connect the tMsgBox_1 component to tParallelize_1 using an OnSubjobOk link, available on the right-click menu. This link will trigger the next subJob(s) only if the first subJob has completed without error.

  • Connect tParallelize_1 to tMsgBox_2 and tMsgBox_3 using a Parallelize link for each, available on the right-click menu.These links will simply parallelize the execution of the two connected subJobs.

  • Connect tParallelize_1 to tMsgBox_4 using a Synchronize link to seqeunce the execution of this fourth subJob.

  • Select the tMsgBox_4 and set its Basic settings parameters.

  • On the Basic settings panel of the tParallelize component and from the Wait For list, select either end of first subJob or end of all subJobs. This will sequence your fourth subJob to be executed at the end of the first subJob or at the end of all subJobs respectively.

  • In the Sleep Duration field, set the time interval in seconds between each check of a subJob execution.

  • Save your main Job.

  • Click the F6 key to run it.

The four message boxes are displayed according to the defined sequence.

The above was a very simple scenario of what the tParallelize component can do for you. However, you can parallelize/synchronize far more complex Jobs with this component whereby each of the subJobs that build the main Job can execute any possible task processed in Talend Studio .