Reloading data at each row - 7.3

Talend Open Studio User Guide

Version
7.3
Language
English
Product
Talend Open Studio for Big Data
Talend Open Studio for Data Integration
Talend Open Studio for Data Quality
Talend Open Studio for ESB
Module
Talend Studio
Content
Design and Development
Last publication date
2023-10-11
Available in...

Open Studio for Big Data

Open Studio for Data Integration

Open Studio for ESB

About this task

The Reload at each row option is used to load all the records of a lookup flow for each record of the main flow.

When the main flow has much less rows than the lookup flow (for example, with a ratio of 1000 or more) and the lookup input is a database component, the advantage of this approach is that it helps deal with the fact that the amount of lookup data increases over time, since you can run queries against the data from the main flow in the database component to select only the lookup data that is relevant for each record in the main flow, such as in the following example which uses lookup data from a MySQL database.

The schemas of the main flow, the lookup flow and the output flow read as follows:

You can select from the MySQL database only the data that matches the values of the id column of the main flow. To do this, proceed as follows:

Procedure

  1. Double-click tMysqlInput to open its Component view.
  2. In the Query field, enter the query to select the data that matches the id column of the main flow. In this example, this query reads: Select * from person where id="+(Integer)globalMap.get("id")

Results

Refer to the related documentation of the components used in this example for more information.