Writing data in parallel - Cloud - 8.0

Talend Studio User Guide

Version
Cloud
8.0
Language
English
Product
Talend Big Data
Talend Big Data Platform
Talend Cloud
Talend Data Fabric
Talend Data Integration
Talend Data Management Platform
Talend Data Services Platform
Talend ESB
Talend MDM Platform
Talend Real-Time Big Data Platform
Module
Talend Studio
Content
Design and Development
Last publication date
2024-02-29
Parallel data writing refers to the concept of speeding-up the execution of a Job by dividing the data flow into multiple fragments that can be written simultaneously.

About this task

Note that when parallel execution is enabled, it is not possible to use global variables to retrieve return values in a subJob.

The Advanced settings for all database output components include the option Enable Parallel Execution which, if selected, allows to perform high-speed data processing, that is treating multiple data flows simultaneously.

Advanced settings view.

When you select the Enable parallel execution check box, the Number of parallel executions field displays where you can enter the number by which the current processed data is devised to achieve N level of parallel processings.

The current processed data being executed across N fragments might execute N times faster than it would if processed as a single fragment.

You can also set the data flow parallelization parameters from the design workspace of the Integration perspective. To do that:

Procedure

  1. Right-click a DB output component on the design workspace and select Parallelize from the drop-down list to display a dialog box.
  2. Select the Enable parallel execution check box and enter the number of parallel executions in the corresponding field. Alternatively, press Ctrl + Space and select the appropriate context variable from the list.
    Dialog box to enable parallel execution.
  3. Click OK to validate data flow parallelization parameters.
    The number of parallel executions displays next to the DB output component in the design workspace.