Specifying the number of MB used in each streaming chunk by Talend Data Mapper - Cloud - 8.0

Talend Studio User Guide

Version
Cloud
8.0
Language
English
Product
Talend Big Data
Talend Big Data Platform
Talend Cloud
Talend Data Fabric
Talend Data Integration
Talend Data Management Platform
Talend Data Services Platform
Talend ESB
Talend MDM Platform
Talend Real-Time Big Data Platform
Module
Talend Studio
Content
Design and Development
Last publication date
2024-02-29
Available in...

Big Data Platform

Cloud API Services Platform

Cloud Big Data Platform

Cloud Data Fabric

Cloud Data Integration

Cloud Data Management Platform

Data Fabric

Data Management Platform

Data Services Platform

MDM Platform

Real-Time Big Data Platform

When running Jobs which contain maps created using Talend Data Mapper and which stream data, it is possible to specify the number of MB used in each streaming chunk. The default is 10 MB, but you can increase this amount if you have more memory to devote to the transformation.

Procedure

  1. In the Run view, in the Advanced settings tab, select the Use specific JVM arguments checkbox.
  2. Click the New button and then, in the Set the VM Argument dialog box that opens, enter the argument to use.
    For example, use -DTDM_STREAM_MEMORY_LIMIT=20 to stream in chunks of 20 MB.
  3. Click OK to add the argument.