Azure Data Lake Storage Gen2 properties

Properties to configure to establish a connection to a given Azure Data Lake Storage Gen2 file system.

Azure Data Lake Storage Gen2 connection

Select Azure Data Lake Storage Gen2 in the list and configure the connection.

Configuration

Select your engine from the list and set the main and advanced settings.

Main settings
Property	Configuration
Authentication Method	Select the database driver you want to use.
Account Name	If this option is disabled, enter each parameter that identifies the database to be used in the corresponding fields. Click Load default values in order to help you pre-fill the fields with the default values associated with this type of database. Information noteNote: Use the host name of the target system instead of '`localhost`' in the URL as the Remote Engine Gen2 needs to be able to communicate with the target system.
Endpoint suffix	If this option is enabled, enter the JDBC URL that identifies the database to be used. The expected format is the following: `jdbc:<dbtype>://<endpoint>[:<port>]/<database_name>` Information noteNote: Use the host name of the target system instead of '`localhost`' in the URL as the Remote Engine Gen2 needs to be able to communicate with the target system.

Advanced settings
Property	Configuration
Timeout	Sets the maximum number of seconds that a user will wait for a connection to be available. If this time is exceeded and the connection is still unavailable, an exception is thrown.

After configuring the connection, give it a display name (mandatory) and a description (optional).

Azure Data Lake Storage Gen2 dataset

Dataset configuration
Property		Configuration
Dataset name		Enter a display name for the dataset. This name will be used as a unique identifier of the dataset in all Talend Cloud apps.
Connection		Select your connection in the list. If you are creating a dataset based on an existing connection, this field is read-only.
Filesystem		Select or enter the name of your Azure Data Lake Storage file system.
Blob path		Enter the path to the directory containing the file to be retrieved.
Format		Select in the list the format of the file to be retrieved and enter or select the information related to this file format: CSV: Field delimiter: Select the type of field separator used in the file to be retrieved. If you select Other, you will be able to enter a custom record delimiter in the custom field delimiter field. Record separator: Select the type of record separator used in the file to be retrieved. If you select Other, you will be able to enter a custom record delimiter in the custom record delimiter field. Text enclosure character: Enter the character used to enclose the fields. Escape character: Enter the character to be escaped in the records to be retrieved. Header: Enable this option if the file to be retrieved contains header lines and enter or select the number of lines to be skipped in the schema. CSV schema: Enter the schema corresponding to your CSV file. File encoding: Select the type of encoding used in the file to be retrieved. If you select Other, you will be able to enter a custom encoding type in the Custom encoding field. Avro: No specific parameters required for this format. JSON: No specific parameters required for this format. Parquet: No specific parameters required for this format. Delta: No specific parameters required for this format. Information noteImportant: Partitioned Delta tables are not supported. Any partitioned column will not be returned.

Did this page help you?

If you find any issues with this page or its content – a typo, a missing step, or a technical error – let us know how we can improve!

Leave your feedback here