How to import delimited file metadata

Talend ESB Studio User Guide

Talend ESB
Design andĀ Development
Talend Studio

You can import the metadata of a delimited file from a predefined CSV file.

Before importing delimited file metadata from a CSV file, make sure that each line of your CSV file complies with the following format:

Name*; Purpose; Description; Version(0.1 by default); Status(DEV|TEST|PROD); FilePath*; FileFormat(UNIX|WINDOWS|MAC); Encoding; FieldSeparatorValue; RowSeparatorValue; EscapeType; EscapeChar; TextEnclosure; FirstLineCaption(true|false); HeaderValue; FooterValue; RemoveEmptyRow(true|false); LimitValue; TableName*; Label*; Comment; DefaultValue; Key*(true|false); Length*; Nullable(true|false); Pattern; Precision; TalendType

Note that:

  • The fields with an asterisk (*) must not be left blank.

  • Name is the file connection name that will be created under the File delimited node. You can create multiple file connections by specifying different connection names.

  • TableName is the name of the file schema, and Label is the column name in the schema.

  • Escape sequences must be used to specify CSV metacharacters or control characters, such as ; or \n.

  • The FirstLineCaption field must be set to true and the HeaderValue field must be filled properly if the delimited file contains a header row and rows to be skipped.

The following example shows how to import the metadata of a delimited file named directors.csv from a predefined CSV file named directors_metadata.csv.

Below is an abstract of the file directors.csv, which has two columns id and name:

1;Gregg Araki
2;P.J. Hogan
3;Alan Rudolph

The CSV file directors_metadata.csv contains two lines to describe the metadata of directors.csv:

directors;Centralize directors metadata;Metadata of directors.csv; 0.1;DEV; E:\Talend\Data\Input\directors.csv;WINDOWS;UTF-8; "\";\""; "\"\\n\""; Delimited;;;true;1;;false;;directors_schema;id;;;false;1;true;;0;id_Integer
directors;Centralize directors metadata;Metadata of directors.csv;0.1;DEV; E:\Talend\Data\Input\directors.csv;WINDOWS;UTF-8; "\";\""; "\"\\n\""; Delimited;;;true;1;;false;;directors_schema;name;;;false;1;true;;0;id_String

To import delimited file connection metadata from the above-mentioned CSV file, do the following:

  1. In the Repository tree view, expand the Metadata node and right-click File delimited.

  2. In the contextual menu, select Import connections from CSV.

    The [Import connections from CSV] dialog box opens.

  3. Click Browse... and browse to the CSV file that describes the metadata of the delimited file metadata, directors_metadata.csv in this example.

  4. Click Finish to close the dialog box.

    The [Show Logs] dialog box opens to list imported and rejected metadata, if any.

  5. Click OK to close the dialog box.

    A new file connection named directors is created under the File delimited node in the Repository tree view, with its properties as defined in the CSV file.