tWaitForSqlData - 6.1

Talend Components Reference Guide

EnrichVersion
6.1
EnrichProdName
Talend Big Data
Talend Big Data Platform
Talend Data Fabric
Talend Data Integration
Talend Data Management Platform
Talend Data Services Platform
Talend ESB
Talend MDM Platform
Talend Open Studio for Big Data
Talend Open Studio for Data Integration
Talend Open Studio for Data Quality
Talend Open Studio for ESB
Talend Open Studio for MDM
Talend Real-Time Big Data Platform
task
Data Governance
Data Quality and Preparation
Design and Development
EnrichPlatform
Talend Studio

tWaitForSqlData properties

Component family

Orchestration

 

Function

tWaitForSqlData component iterates on a given connection for insertion or deletion of rows and triggers a subjob to be executed when the condition is met.

Purpose

This component allows a subjob to be triggered given a condition linked to SQL data presence.

Basic settings

Wait at each iteration (in seconds)

Set the time interval in seconds between each check for the sql data.

 

Max. iterations (infinite if empty)

Number of checks for sql data before the Jobs times out.

 

Use an existing connection/Component List

A connection needs to be open to allow the loop to check for sql data on the defined DB.

Note

When a Job contains the parent Job and the child Job, Component list presents only the connection components in the same Job level, so if you need to use an existing connection from the other level, you can

From the available database connection component in the level where the current component is, select the Use or register a shared DB connection check box. For more information about this check box, see Databases - traditional components, Databases - appliance/datawarehouse components, or Databases - other components for the connection components according to the database you are using.

Otherwise, still in the level of the current component, deactivate the connection components and use Dynamic settings of the component to specify the intended connection manually. In this case, make sure the connection name is unique and distinctive all over through the two Job levels. For more information about Dynamic settings, see Talend Studio User Guide.

 

Table to scan

Name of the table to be checked for insert or deletion

 

Trigger action when rowcount is

Select the condition to be met for the action to be carried out:

Equal to Not Equal to Greater than Lower than Greater or equal to Lower or equal to

 

Value

Define the value to take into account.

 

Then

Select the action to be carried out: either stop the iterations when the condition is met (exit loop) or continue the loop until the end of the max iteration number (continue loop).

Global Variables

ERROR_MESSAGE: the error message generated by the component when an error occurs. This is an After variable and it returns a string. This variable functions only if the Die on error check box is cleared, if the component has this check box.

CURRENT_ITERATION: the sequence number of the current iteration. This is a Flow variable and it returns an integer.

ROW_COUNT: the number of rows detected in the table. This is a Flow variable and it returns an integer.

A Flow variable functions during the execution of a component while an After variable functions after the execution of the component.

To fill up a field or expression with a variable, press Ctrl + Space to access the variable list and choose the variable to use from it.

For further information about variables, see Talend Studio User Guide.

Usage

Although this component requires a Connection component to open the DB access, it plays also the role of the start (or trigger) component of the subjob which gets executed under the condition described. Therefore this component requires a subjob to be connected to via an Iterate link.

Limitation

n/a

Scenario: Waiting for insertion of rows in a table

This scenario describes a Job reading a DB table and waiting for data to be put in this table in order for a subjob to be executed. When the condition of the data insertion in the table is met, then the subjob performs a Select* on the table and simply displays the content of the inserted data onto the standard console.

  • Drop the following components from the Palette onto the design workspace: tMySqlConnection, tWaitForSqlData, tMysqlInput, tLogRow.

  • Connect the tMysqlConnection component to the tWaitforSqlData using an OnSubjobOK link, available on the right-click menu.

  • Then connect the tWaitForSqlData component to the subjob using an Iterate link as no actual data is transferred in this part. Indeed, simply a loop is implemented by the tWaitForSqlData until the condition is met.

  • On the subjob to be executed if the condition is met, a tMysqlInput is connected to the standard console component, tLogRow. As the connection passes on data, use a Row main link.

  • Now, set the connection to the table to check at regular intervals. On the Basic Settings view of the tMySqlConnection Component tab, set the DB connection properties.

  • Fill out the Host, Port, Database, Username, Password fields to open the connection to the Database table.

  • Select the relevant Encoding if needed.

  • Then select the tWaitForSqlData component, and on the Basic Setting view of the Component tab, set its properties.

  • In the Wait at each iteration field, set the time in seconds you want to wait before the next iteration starts.

  • In the Max iterations field, fill out the number of iterations max you want to have before the whole Job is forced to end.

  • The tWaitForSqlData component requires a connection to be open in order to loop on the defined number of iteration. Select the relevant connection (if several) in the Component List combo box.

  • In the Table to scan field, type in the name of the table in the DB to scan.In this example: test_datatypes.

  • In the Trigger action when rowcount is and Value fields, select the condition to be met, for the subjob to be triggered. In this use case, the number of rows in the scanned table should be greater or equal to 1.

  • In the Then field, select the action to be carried out when the condition is met before the number of iteration defined is reached. In this use case, as soon as the condition is met, the loop should be ended.

  • Then set the subjob to be executed when the condition set is met. In this use case, the subjob simply selects the data from the scanned table and displays it on the console.

  • Select the tMySqlInput component, and on the Basic Setting view of the Component tab, set the connection to the table.

  • If the connection is set in the Repository, select the relevant entry on the list. Or alternatively, select the Use an existing connection check box and select the relevant connection component on the list.

  • In this use case, the schema corresponding to the table structure is stored in the Repository.

  • Fill out the Table Name field with the table the data is extracted from, Test_datatypes.

  • Then in the Query field, type in the Select statement to extract the content from the table.

  • No particular setting is required in the tLogRow component for this use case.

Then before executing the Job, make sure the table to scan (test_datatypes) is empty, in order for the condition (greater or equal to 1) to be met. Then execute the Job by pressing the F6 key on your keyboard. Before the end of the iterating loop, feed the test_datatypes table with one or more rows in order to meet the condition.

The Job ends when this table insert is detected during the loop, and the table content is thus displayed on the console.