tFileRowCount - 6.3

Talend Components Reference Guide

EnrichVersion
6.3
EnrichProdName
Talend Big Data
Talend Big Data Platform
Talend Data Fabric
Talend Data Integration
Talend Data Management Platform
Talend Data Services Platform
Talend ESB
Talend MDM Platform
Talend Open Studio for Big Data
Talend Open Studio for Data Integration
Talend Open Studio for Data Quality
Talend Open Studio for ESB
Talend Open Studio for MDM
Talend Real-Time Big Data Platform
task
Data Governance
Data Quality and Preparation
Design and Development
EnrichPlatform
Talend Studio

Function

tFileRowCount counts the number of rows in a file.

Purpose

tFileRowCount opens a file and reads it row by row in order to determine the number of rows inside.

tFileRowCount properties

Component Family

File/Management

 

Basic settings

File Name

Name or path to the file to be processed and/or the variable to be used.

For further information about how to define and use a variable in a Job, see Talend Studio User Guide.

 

Row separator

String (ex: "\n"on Unix) to distinguish rows in the output file.

 

Ignore empty rows

Select this check box to ignore the empty rows while the component is counting the rows in the file.

 

Encoding

Select the encoding type from the list or select Custom and define it manually. This field is compulsory for DB data handling.

Advanced settings

tStatCatcher Statistics

Select this check box to gather the processing metadata at the Job level as well as at each component level.

Global Variables

COUNT: the number of rows in a file. This is a Flow variable and it returns an integer.

ERROR_MESSAGE: the error message generated by the component when an error occurs. This is an After variable and it returns a string. This variable functions only if the Die on error check box is cleared, if the component has this check box.

A Flow variable functions during the execution of a component while an After variable functions after the execution of the component.

To fill up a field or expression with a variable, press Ctrl + Space to access the variable list and choose the variable to use from it.

For further information about variables, see Talend Studio User Guide.

Connections

Outgoing links (from this component to another):

Row: Main; Iterate.

Trigger: On Subjob Ok; On Subjob Error; Run if; On Component Ok; On Component Error.

Incoming links (from one component to this one):

Row: Main; Reject; Iterate.

Trigger: On Subjob Ok; On Subjob Error; Run if; On component Ok; On Component Error; Synchronize; Parallelize.

For further information regarding connections, see Talend Studio User Guide.

Usage

tFileRowCount is a standalone component, it must be used with a OnSubjobOk connection to tJava.

Log4j

If you are using a subscription-based version of the Studio, the activity of this component can be logged using the log4j feature. For more information on this feature, see Talend Studio User Guide.

For more information on the log4j logging levels, see the Apache documentation at http://logging.apache.org/log4j/1.2/apidocs/org/apache/log4j/Level.html.

Limitation

Due to license incompatibility, one or more JARs required to use this component are not provided. You can install the missing JARs for this particular component by clicking the Install button on the Component tab view. You can also find out and add all missing JARs easily on the Modules tab in the Integration perspective of your studio. For details, see the article Installing External Modules on Talend Help Center (https://help.talend.com) how to configure the Studio in the Talend Installation Guide.

Scenario: Writing a file to MySQL if the number of its records matches a reference value

In this scenario, tFileRowCount counts the number of records in a .txt file, which is compared against a reference value through tJava. Once the two values match, the .txt file will be written to a MySQL table.

The .txt file has two records:

1;andy
2;mike

Linking the components

  1. Drop tFileRowCount, tJava, tFlieInputDelimited, and tMysqlOutput from the Palette onto the design workspace.

  2. Link tFileRowCount to tJava using an OnSubjobOk trigger.

  3. Link tJava to tFlieInputDelimited using a Run if trigger.

  4. Link tFlieInputDelimited to tMysqlOutput using a Row > Main connection.

Configuring the components

  1. Double-click tFileRowCount to open its Basic settings view.

  2. In the File Name field, type in the full path of the .txt file. You can also click the [...] button to browse for this file.

    Select the Ignore empty rows check box.

  3. Double-click tJava to open its Basic settings view.

    In the Code box, enter the function to print out the number of rows in the file:

    System.out.println(globalMap.get("tFileRowCount_1_COUNT"));
  4. Click the if trigger connection to open its Basic settings view.

    In the Condition box, enter the statement to judge if the number of rows is 2:

    ((Integer)globalMap.get("tFileRowCount_1_COUNT"))==2

    This if trigger means that if the row count equals 2, the rows of the .txt file will be written to MySQL.

  5. Double-click tFlieInputDelimited to open its Basic settings view.

    In the File name/Stream field, type in the full path of the .txt file. You can also click the [...] button to browse for this file.

  6. Click the Edit schema button open the schema editor.

  7. Click the [+] button to add two columns, namely id and name, respectively of the integer and string type.

  8. Click the Yes button in the pop-up box to propagate the schema setup to the following component.

  9. Double-click tMysqlOutput open its Basic settings view.

  10. In the Host and Port fields, enter the connection details.

    In the Database field, enter the database name.

    In the Username and Password fields, enter the authentication details.

    In the Table field, enter the table name, for instance "staff".

  11. In the Action on table list, select Create table if not exists.

    In the Action on data list, select Insert.

Executing the Job

  1. Press Ctrl+S to save the Job.

  2. Press F6 to run the Job.

    As shown above, the Job has been executed successfully and the number of rows in the .txt file has been printed out.

  3. Go to the MySQL GUI and open the table staff.

    As shown above, the table has been created with the two records inserted.