Selecting the files

HDFS

author
Talend Documentation Team
EnrichVersion
6.4
EnrichProdName
Talend Real-Time Big Data Platform
Talend Data Fabric
Talend Big Data Platform
Talend Big Data
Talend Open Studio for Big Data
task
Data Governance > Third-party systems > File components (Integration) > HDFS components
Data Quality and Preparation > Third-party systems > File components (Integration) > HDFS components
Design and Development > Third-party systems > File components (Integration) > HDFS components
EnrichPlatform
Talend Studio

Procedure

  1. Double-click tHDFSGet to open its Component view.
  2. In the Version area, select the Hadoop distribution you are connecting to and its version.
  3. In the Connection area, enter the values of the parameters required to connect to the HDFS.
    In the real-world practice, you may have used tHDFSConnection to create a connection; then you can reuse it from the current component. For further information, see tHDFSConnection.
  4. In the HDFS directory field, enter the path to the folder holding the files to be retrieved.
    To do this with the auto-completion list, place the mouse pointer in this field, then, press Ctrl+Space to display the list and select the tHDFSList_1_CURRENT_FILEDIRECTORY variable to reuse the directory you have defined in tHDFSList. In this variable, tHDFSList_1 is the label of the component. If you label it differently, select the variable accordingly.
    Once selecting this variable, the directory reads, for example, ((String)globalMap.get("tHDFSList_1_CURRENT_FILEDIRECTORY")) in this field.
    For further information about how to label a component, see the Talend Studio User Guide.
  5. In the Local directory field, enter the path, or browse to the folder you want to place the selected files in. This folder will be created if it does not exist. In this example, it is C:/hdfsFiles.
  6. In the Overwrite file field, select always.
  7. In the Files table, click to add one row and enter * between the quotation marks in the Filemask column in order to get any files existing.