Scenario: handling a file using SCP - 6.3

Talend Open Studio for Big Data Components Reference Guide

EnrichVersion
6.3
EnrichProdName
Talend Open Studio for Big Data
task
Data Governance
Data Quality and Preparation
Design and Development
EnrichPlatform
Talend Studio

In this scenario, a six-component Job checks if a defined file exists in a specified directory on a remote SCP server, and returns a message to indicate the result of the verification. If the file doesn't exist in that directory, the Job will put the local file to the SCP server, and then get the file from the SCP server to another defined local directory to verify that the file has been uploaded correctly.

Dropping and linking the components

  1. Add the following six components by typing their names on the design workspace or dropping them from the Palette to the design workspace: tSCPConnection, tSCPFileExists, tJava, tSCPPut, tSCPGet, and tSCPClose.

  2. Connect the tSCPConnection component to the tSCPFileExists component using a Trigger > On Subjob Ok connection.

  3. Connect the tSCPFileExists component to the tJava component using a Trigger > On Component Ok connection, and to the tSCPPut component using a Trigger > Run if connection.

  4. Connect the tSCPPut component to the tSCPGet component using a Trigger > On Subjob Ok connection.

  5. Connect the tSCPGet component to the tSCPClose component using a Trigger > On Subjob Ok connection.

Configuring the tSCPConnection component

  1. Double-click the tSCPConnection component to open its Basic settings view.

  2. Enter the host name or IP address of the SCP server and the port number in the corresponding field.

  3. Enter the user name used for connection in the Username field.

  4. Select Password from the Authentication method drop-down list.

  5. Click the [...] button next to the Password text box, then in the pop-up [Enter a new password] dialog box, enter the password between double quotes and click OK to save the change.

Configuring file existence verification

  1. Double-click the tSCPFileExists component to open its Basic settings view.

  2. Select the Use an existing connection check box and then select the connection you have configured from the Component List.

  3. In the Filename field, enter the name of the file to be checked between double quotes. In this example, it is "testscp.txt".

  4. Double-click the tJava component to open its Basic settings view.

  5. In the Code field, enter the following code:

    System.out.println((String)globalMap.get("tSCPFileExists_1_FILENAME"));
    System.out.println((String)globalMap.get("tSCPFileExists_1_STATUS"));
  6. Select the Component tab and click the If connection to open its Basic settings view.

  7. In the Condition field, enter the following code:

    !((String)globalMap.get("tSCPFileExists_1_STATUS")).equals("File exists.")

    This way, if the file doesn't exist in the defined directory, the rest of the Job will be executed.

Configuring the tSCPPut component

  1. Double-click the tSCPPut component to open its Basic settings view.