Configuring the connection to the S3 service to be used by Spark - 7.3

Amazon S3
Version
7.3
Language
English
Product
Talend Big Data
Talend Big Data Platform
Talend Data Fabric
Talend Data Integration
Talend Data Management Platform
Talend Data Services Platform
Talend ESB
Talend MDM Platform
Talend Real-Time Big Data Platform
Module
Talend Studio
Content
Data Governance > Third-party systems > Amazon services (Integration) > Amazon S3 components
Data Quality and Preparation > Third-party systems > Amazon services (Integration) > Amazon S3 components
Design and Development > Third-party systems > Amazon services (Integration) > Amazon S3 components
Last publication date
2024-02-21

Procedure

  1. Double-click tS3Configuration to open its Component view.
    Spark uses this component to connect to the S3 system in which your Job writes the actual business data. If you place neither tS3Configuration nor any other configuration component that supports Databricks on AWS, this business data is written in the Databricks Filesystem (DBFS).

    Example

  2. In the Access key and the Secret key fields, enter the keys to be used to authenticate to S3.
  3. In the Bucket name field, enter the name of the bucket and the folder in this bucket to be used to store the business data, for example, mybucket/myfolder. This folder is created on the fly if it does not exist but the bucket must already exist at runtime.