How to add a File trigger - 8.0

Talend Administration Center User Guide

Version
8.0
Language
English
Product
Talend Big Data
Talend Big Data Platform
Talend Data Fabric
Talend Data Integration
Talend Data Management Platform
Talend Data Services Platform
Talend ESB
Talend MDM Platform
Talend Real-Time Big Data Platform
Module
Talend Administration Center
Content
Administration and Monitoring
Last publication date
2024-02-20

About this task

You can use a file to trigger Job execution tasks. The presence of this file in a defined location will trigger the Job execution task. This file becomes then a job trigger with no other purpose.

Below is a diagram introducing the way file triggers work in Talend Administration Center.
  • Talend Administration Center registers a file trigger in the Talend JobServer to watch the folder located in the file system of the Talend JobServer.

  • Talend Administration Center requests the Talend JobServer after each polling period to get the file events of a given Talend Administration Center file trigger. Note that each Talend JobServer file trigger has an independent queue in memory.

  • Between each polling and for each file trigger, events are added to the queue at the Talend JobServer side, then when Talend Administration Center polls the server at the end of the polling period, all the events related to this file trigger are retrieved.

    At the same time, the Talend JobServer empties the queue, and Talend Administration Center triggers a task execution for each event that is thus added to the Quartz queue.

  • The Talend JobServer events queue is cleared each time Talend Administration Center polls the server.

    Note that the Talend JobServer events are not supposed to be cleared manually, however you can reset the Talend Administration Center Quartz queue of a given task from the Job Conductor page of the web application.

Note: If a context has been defined and that you intend to use the trigger file in the Job to execute, then make sure to define the following context parameters in the Context view in Talend Studio: folderPath, filePath, and fileName. Thus, the Job Conductor passes these values to the Job during its execution.

To schedule an execution task based on the presence of a file in a defined location:

Procedure

  1. On the Job Conductor page, select the task you want to implement a trigger on.
  2. In the Triggers view, click Add trigger... and then select Add file trigger from the drop-down list. The File trigger configuration panel opens.

    Example

  3. Fill in the following information:

    Fields

    Description

    Label

    Enter a name to the trigger you are setting.

    Description

    Enter a description for the trigger type and usage.

    Pollint

    Polling interval in seconds, corresponding to the time between each checking operation of the file server.

    Folder Path

    Path to the folder to be scanned regularly for the trigger file.

    File mask

    Mask of the trigger file.

     

    In order not to trigger job execution for every file in the polling folder, do the following:

    Exist: select this check box to trigger on file if the file already exists in the polling folder

    Created: select this check box to trigger on file if the file is being created and does not already exist in the polling folder

    Modified: select this check box to trigger on file if the file is modified.

    Note: It is possible to select more than one check box.

    Execution server

    Select the server where the trigger file is put.
    Note: If you select a virtual server, then the trigger file will be deployed in each of the Job servers that are grouped in the virtual server. If the Job servers grouped in the virtual server monitor the same file in the same location, then only one event will be processed by Talend Administration Center, otherwise Talend Administration Center will consider the files different and process each event separately.
  4. Click Save to validate the File trigger configuration or Cancel to cancel the operation.

Results

The Trigger Status icon for the selected task changes from No Trigger to At least one trigger is running.