tWaitForFile - 6.1

Talend Components Reference Guide

EnrichVersion
6.1
EnrichProdName
Talend Big Data
Talend Big Data Platform
Talend Data Fabric
Talend Data Integration
Talend Data Management Platform
Talend Data Services Platform
Talend ESB
Talend MDM Platform
Talend Open Studio for Big Data
Talend Open Studio for Data Integration
Talend Open Studio for Data Quality
Talend Open Studio for ESB
Talend Open Studio for MDM
Talend Real-Time Big Data Platform
task
Data Governance
Data Quality and Preparation
Design and Development
EnrichPlatform
Talend Studio

tWaitForFile properties

Component family

Orchestration

 

Function

tWaitForFile iterates on the specified directory and triggers the next component when the defined condition is met.

Purpose

This component is used to put the component connected with it in waiting state. It then triggers that component when the defined file operation occurs in the specified directory.

Basic settings

Time (in seconds) between iterations

Set the time interval in seconds between each two iterations.

 

Max. number of iterations (infinite loop if empty)

Set the maximum number of iterations on the specified directory.

 

Directory to scan

Specify the directory to iterate on.

 

File mask

Type in the file mask to filter the files to be monitored. To monitor a given file, type in the file name.

 

Include subdirectories

Select this check box to iterate also on the subdirectories.

 

Case sensitive

Select this check box to activate case sensitivity.

 

Include present file

Select this check box to include the file in use.

 

Trigger action when

Select one of the following trigger conditions:

a file is created: triggers the next component when a file is created.

a file is deleted: triggers the next component when a file is removed.

a file is updated: triggers the next component when a file is modified.

a file is created or updated or deleted: triggers the next component when a file is created, or removed, or modified.

 

Then

Select what to do with the iteration loop after the next component is triggered:

continue loop: continues iterating until the number of iterations reaches the defined maximum value.

exit loop: stops iterating immediately.

 

Schema and Edit Schema

A schema is a row description, it defines the number of fields that will be processed and passed on to the next component. The schema of this component is Built-in and cannot be updated.

Advanced Settings

Wait for file to be released Every (in ms)

Select this check box so that the next component is triggered only after the file operation is completed.

Global Variables

ERROR_MESSAGE: the error message generated by the component when an error occurs. This is an After variable and it returns a string. This variable functions only if the Die on error check box is cleared, if the component has this check box.

CURRENT_ITERATION: the sequence number of the current iteration. This is a Flow variable and it returns an integer.

PRESENT_FILE: the path and name of the present file. This is a Flow variable and it returns a string.

DELETED_FILE: the path and name of the deleted file. This is a Flow variable and it returns a string.

CREATED_FILE: the path and name of the created file. This is a Flow variable and it returns a string.

UPDATED_FILE: the path and name of the updated file. This is a Flow variable and it returns a string.

FILENAME: the name of the file being processed. This is a Flow variable and it returns a string.

NOT_UPDATED_FILE: the path and name of the file that has not been updated. This is a Flow variable and it returns a string.

A Flow variable functions during the execution of a component while an After variable functions after the execution of the component.

To fill up a field or expression with a variable, press Ctrl + Space to access the variable list and choose the variable to use from it.

For further information about variables, see Talend Studio User Guide.

Usage

This component plays the role of triggering the next component based on the defined condition. Therefore this component requires another component to be connected to it via a link.

Connections

Outgoing links (from this component to another):

Row: Main; Iterate.

Trigger: On Subjob Ok; Run if; On Component Ok; On Component Error.

Incoming links (from one component to this one):

Row: Iterate.

Trigger: On Subjob Ok; Run if; On Component Ok; On Component Error; Synchronize; Parallelize.

For further information regarding connections, see Talend Studio User Guide.

Log4j

If you are using a subscription-based version of the Studio, the activity of this component can be logged using the log4j feature. For more information on this feature, see Talend Studio User Guide.

For more information on the log4j logging levels, see the Apache documentation at http://logging.apache.org/log4j/1.2/apidocs/org/apache/log4j/Level.html.

Limitation

n/a

Scenario 1: Waiting for a file to be created and stopping the iteration loop after a message is triggered

This scenario describes a Job that scans a directory and triggers a message showing file creation information on the console when a text file is created in that directory. Immediately after this, the scanning is ended, followed by another message showing the number of iterations finished.

Dropping and linking the components

  1. Drop three components from the Palette onto the design workspace: one tWaitForFile and two tJava components.

  2. Connect tWaitForFile to the first tJava using a Row > Iterate connection.

  3. Connect tWaitForFile to the second tJava using a Trigger > On Subjob Ok connection.

Configuring the components

  1. Double-click tWaitForFile to open its Basic settings view.

  2. In the Directory to scan field, enter the directory to be scanned.

  3. In the File mask field, enter "*.txt" to scan only text files.

  4. From the Trigger action when drop-down list, select a file is created to trigger the first tJava when a text file is created.

  5. From the Then drop-down list, select exit loop to stop the iteration loop immediately after the first tJava is triggered.

  6. Double-click the first tJava to open its Basic settings view.

  7. In the code area, type in the following code:

    System.out.println("A file was created at " + TalendDate.getCurrentDate());
    System.out.println("Name of the created file: " + ((String)globalMap.get("tWaitForFile_1_CREATED_FILE")));
  8. Double-click the second tJava to open its Basic settings view.

  9. In the code area, type in the following code:

    System.out.println("\r\nIteration loop ended at " + TalendDate.getCurrentDate());
    System.out.println("Number of iterations finished: " +  ((Integer)globalMap.get("tWaitForFile_1_CURRENT_ITERATION")));

Saving and executing the Job

  1. Press Ctrl+S to save the Job.

  2. Press F6 to execute the Job.

  3. Create a text file exit.txt in the defined directory.

    When a text file is created, a message about file creation is displayed on the Run console. The iteration loop is ended immediately after that, followed by another message showing the time the iteration loop is ended and the number of iterations executed.

Scenario 2: Waiting for a file to be created and continuing the iteration loop after a message is triggered

Based on the previous scenario, this scenario describes a Job triggers a message on the console when a text file is created in the directory being scanned, without ending the scanning process immediately. The Job continues scanning the directory until the number of iterations reaches the defined maximum value.

  1. Double-click tWaitForFile to open its Basic settings view.

  2. In the Time (in seconds) between iterations field, enter the time interval in seconds between each two iterations. In this example, it is set to 5.

  3. In the Max. number of iterations (infinite loop if empty) field, enter the maximum number of iterations. In this example, it is set to 10.

  4. From the Then drop-down list, select continue loop to continue the iteration loop after the message is triggered.

  5. Press Ctrl+S to save the Job.

  6. Press F6 to execute the Job.

  7. Create a text file continue.txt in the defined directory.

    When a text file is created, a message about file creation is displayed on the Run console. The iteration loop is ended until the number of iterations reaches ten, followed by another message showing the time the iteration loop is ended and the number of iterations executed.