Reloading data at each row - Cloud

Talend Cloud API Services Platform Studio User Guide

author
Talend Documentation Team
EnrichVersion
Cloud
EnrichProdName
Talend Cloud
task
Design and Development
EnrichPlatform
Talend Management Console
Talend Studio

About this task

The Reload at each row option is used to load all the records of a lookup flow for each record of the main flow.

When the main flow has much less rows than the lookup flow (for example, with a ratio of 1000 or more) and the lookup input is a database component, the advantage of this approach is that it helps deal with the fact that the amount of lookup data increases over time, since you can run queries against the data from the main flow in the database component to select only the lookup data that is relevant for each record in the main flow, such as in the following example which uses lookup data from a MySQL database.

The schemas of the main flow, the lookup flow and the output flow read as follows:

You can select from the MySQL database only the data that matches the values of the id column of the main flow. To do this, proceed as follows:

Procedure

  1. Double-click tSetGlobalVar to open its Component view.
  2. Click the [+] button to add one row and name the Key to id and the Value to row1.id.
  3. Double-click tMysqlInput to open its Component view.
  4. In the Query field, enter the query to select the data that matches the id column of the main flow. In this example, this query reads: Select * from person where id="+(Integer)globalMap.get("id")

Results

Refer to the related documentation of the components used in this example for more information.