Defining the maximum memory size threshold - Cloud - 7.3

Talend Studio User Guide

Version
Cloud
7.3
Language
English
Product
Talend Big Data
Talend Big Data Platform
Talend Cloud
Talend Data Fabric
Talend Data Integration
Talend Data Management Platform
Talend Data Services Platform
Talend ESB
Talend MDM Platform
Talend Real-Time Big Data Platform
Module
Talend Studio
Content
Design and Development
Last publication date
2024-02-13
Available in...

Big Data Platform

Cloud API Services Platform

Cloud Big Data Platform

Cloud Data Fabric

Cloud Data Management Platform

Data Fabric

Data Management Platform

Data Services Platform

MDM Platform

Real-Time Big Data Platform

From Talend Studio, you can control memory usage when using the Java engine to run two types of analyses: column analysis and the analysis of a set of columns.

If you use column analysis or column set analysis to profile very big sets of data or data with many problems, you may run out of memory and end up with a Java heap error. By defining the maximum memory size threshold for these analyses, Talend Studio will stop the analysis execution when the memory limit size is reached and provide you with the analysis results that were measured on the data before the analysis execution was terminated by the memory limit size.

Procedure

  1. From the Profiling perspective, on the menu bar, select Window > Preferences to display the Preferences window.
  2. Perform one of the following steps:
    • Expand Talend > Profiling and select Analysis tuning, or,
    • Start typing analysis tuning in the dynamic filter field.
    The Analysis tuning view is displayed.
  3. In the Memory area, select the Enable analysis thread memory control check box.
  4. Move the slider to the right to define the memory limit at which the analysis execution will be stopped.

Results

The execution of any column analysis or column set analysis will be stopped if it exceeds the allocated memory size. The analysis results given in Talend Studio will cover the data analyzed before the interruption of the analysis execution.