In this scenario, the two-component Job checks whether a specific file exists in HDFS and returns a message to indicate the result of the verification.
In the real-world practice, you can take further action to process the file checked according to the verification result, using the other HDFS components provided with the Studio.
Launch the Hadoop distribution in which you want to check the existence of a particular file. Then, proceed as follows:
In the Integration perspective of Talend Studio, create an empty Job, named hdfsexist_file for example, from the Job Designs node in the Repository tree view.
For further information about how to create a Job, see the Talend Studio User Guide.
Drop tHDFSExist and tMsgBox onto the workspace.
Connect them using the Trigger > Run if link.
Double-click tHDFSExist to open its Component view.
In the Version area, select the Hadoop distribution you are connecting to and its version.
In the Connection area, enter the values of the parameters required to connect to the HDFS.
In the real-world practice, you may use tHDFSConnection to create a connection and reuse it from the current component. For further information, see tHDFSConnection.
In the HDFS Directory field, browse to, or enter the path to the folder where the file to be checked is. In this example, browse to /user/ychen/data/hdfs/out/dest.
In the File name or relative path field, enter the name of the file you want to check the existence. For example, output.csv.
Double-click tMsgBox to open its Component view.
In the Title field, enter the title to be used for the pop-up message box to be created.
In the Buttons list, select OK. This defines the button to be displayed on the message box.
In the Icon list, select Icon information.
In the Message field, enter the message you want to displayed once the file checking is done. In this example, enter "This file does not exist!".
Click the If link to open the Basic settings view, where you are able to define the condition for checking the existence of this file.
In the Condition box, press Ctrl+Space to access the variable list and select the global variable EXISTS. Type an exclamation mark before the variable to negate the meaning of the variable.
Press F6 to execute this Job.
Once done, a message box pops up to indicate that this file called output.csv does not exist in the directory you defined earlier.
In the HDFS we check the existence of the file, browse to this directory specified, you can see that this file does not exist.
 |