Orchestrating parallel executions of subJobs - Cloud - 8.0

Talend Studio User Guide

Version
Cloud
8.0
Language
English
Product
Talend Big Data
Talend Big Data Platform
Talend Cloud
Talend Data Fabric
Talend Data Integration
Talend Data Management Platform
Talend Data Services Platform
Talend ESB
Talend MDM Platform
Talend Real-Time Big Data Platform
Module
Talend Studio
Content
Design and Development
Last publication date
2024-02-29

Talend Studio uses the tParallelize component to orchestrate the parallel executions of subJobs that are active within a Job.

When a Job contains several subJobs, you might want to execute some of the subJobs in parallel and then synchronize the executions of the other subJobs at the end of the parallel executions.

To do this, you can simply use tParallelize to orchestrate all of the subJobs to be executed.

Warning: The tParallelize component is designed for Data Integration Jobs. Do not use it in the context of Data Services Jobs or Routes because it is not thread safe between parallel requests and will cause huge memory and CPU consumption. Use a Route if parallel processing is a requirement.
Job in the design workspace.

In the example presented in the image, tParallelize launches at first the following subJobs: workflow_sales, workflow_rd and workflow_finance; after the executions are completed, it launches workflow_hr.