Waiting for insertion of rows in a table - Cloud - 8.0

Orchestration (Integration)

Version
Cloud
8.0
Language
English
Product
Talend Big Data
Talend Big Data Platform
Talend Data Fabric
Talend Data Integration
Talend Data Management Platform
Talend Data Services Platform
Talend ESB
Talend MDM Platform
Talend Real-Time Big Data Platform
Module
Talend Studio
Content
Data Governance > Third-party systems > Orchestration components (Integration)
Data Quality and Preparation > Third-party systems > Orchestration components (Integration)
Design and Development > Third-party systems > Orchestration components (Integration)
Last publication date
2024-02-29

This scenario describes a Job reading a DB table and waiting for data to be put in this table in order for a subJob to be executed. When the condition of the data insertion in the table is met, then the subJob performs a Select* on the table and simply displays the content of the inserted data onto the standard console.

For more technologies supported by Talend, see Talend components.

  • Drop the following components from the Palette onto the design workspace: tMysqlConnection, tWaitForSqlData, tMysqlInput, tLogRow.

  • Connect the tMysqlConnection component to the tWaitforSqlData using an OnSubjobOK link, available on the right-click menu.

  • Then connect the tWaitForSqlData component to the subJob using an Iterate link as no actual data is transferred in this part. Indeed, simply a loop is implemented by the tWaitForSqlData until the condition is met.

  • On the subJob to be executed if the condition is met, a tMysqlInput is connected to the standard console component, tLogRow. As the connection passes on data, use a Row main link.

  • Now, set the connection to the table to check at regular intervals. On the Basic Settings view of the tMysqlConnection Component tab, set the DB connection properties.

  • Fill out the Host, Port, Database, Username, Password fields to open the connection to the Database table.

  • Select the relevant Encoding if needed.

  • Then select the tWaitForSqlData component, and on the Basic Setting view of the Component tab, set its properties.

  • In the Wait at each iteration field, set the time in seconds you want to wait before the next iteration starts.

  • In the Max iterations field, fill out the number of iterations max you want to have before the whole Job is forced to end.

  • The tWaitForSqlData component requires a connection to be open in order to loop on the defined number of iteration. Select the relevant connection (if several) in the Component List combo box.

  • In the Table to scan field, type in the name of the table in the DB to scan.In this example: test_datatypes.

  • In the Trigger action when rowcount is and Value fields, select the condition to be met, for the subJob to be triggered. In this use case, the number of rows in the scanned table should be greater or equal to 1.

  • In the Then field, select the action to be carried out when the condition is met before the number of iteration defined is reached. In this use case, as soon as the condition is met, the loop should be ended.

  • Then set the subJob to be executed when the condition set is met. In this use case, the subJob simply selects the data from the scanned table and displays it on the console.

  • Select the tMysqlInput component, and on the Basic Setting view of the Component tab, set the connection to the table.

  • If the connection is set in the Repository, select the relevant entry on the list. Or alternatively, select the Use an existing connection check box and select the relevant connection component on the list.

  • In this use case, the schema corresponding to the table structure is stored in the Repository.

  • Fill out the Table Name field with the table the data is extracted from, Test_datatypes.

  • Then in the Query field, type in the Select statement to extract the content from the table.

  • No particular setting is required in the tLogRow component for this use case.

Then before executing the Job, make sure the table to scan (test_datatypes) is empty, in order for the condition (greater or equal to 1) to be met. Then execute the Job by pressing the F6 key on your keyboard. Before the end of the iterating loop, feed the test_datatypes table with one or more rows in order to meet the condition.

The Job ends when this table insert is detected during the loop, and the table content is thus displayed on the console.