Connecting to an ADLS Databricks Gen2 file - Cloud - 8.0

Talend Studio User Guide

Version
Cloud
8.0
Language
English
Product
Talend Big Data
Talend Big Data Platform
Talend Cloud
Talend Data Fabric
Talend Data Integration
Talend Data Management Platform
Talend Data Services Platform
Talend ESB
Talend MDM Platform
Talend Real-Time Big Data Platform
Module
Talend Studio
Content
Design and Development
Last publication date
2024-02-29
Available in...

Big Data Platform

Cloud API Services Platform

Cloud Big Data Platform

Cloud Data Fabric

Cloud Data Management Platform

Data Fabric

Data Management Platform

Data Services Platform

MDM Platform

Real-Time Big Data Platform

You need to create an external table and set up a parameter to profile an ADLS Databricks Gen2 file.

Before you begin

You have created an external table. For more information, see Microsoft Azure documentation.

To create an external table, you can also use the tAzureFSConfiguration and tJava components.

About this task

This procedure is specific to ADLS Databricks Gen2.

Procedure

  1. On the Configuration tab of your Databricks cluster page, expand the Advanced options.
  2. In the Spark tab, add the following Spark properties: spark.hadoop.fs.azure.account.key.<storage_account>.dfs.core.windows.net <key>

    This key is associated with the storage account to be used. You can find it in the Access keys blade of this storage account. Two keys are available for each account and by default, either of them can be used for this access.

    Ensure that the account to be used has the appropriate read/write rights and permissions.