Linking the components to design the flow of Delta Lake data - Cloud - 8.0

Delta Lake

Version
Cloud
8.0
Language
English
Product
Talend Big Data
Talend Big Data Platform
Talend Data Fabric
Talend Data Integration
Talend Data Management Platform
Talend Data Services Platform
Talend ESB
Talend MDM Platform
Talend Real-Time Big Data Platform
Module
Talend Studio
Content
Data Governance > Third-party systems > Technical components > Delta Lake components
Data Quality and Preparation > Third-party systems > Technical components > Delta Lake components
Design and Development > Third-party systems > Technical components > Delta Lake components
Last publication date
2024-02-20
Drop and link the components to be used to read and process your Delta Lake data.

Procedure

  1. In the Integration perspective of Talend Studio, create an empty Spark Batch Job from the Job Designs node in the Repository tree view.
  2. In the workspace, enter the name of the component to be used and select this component from the list that appears. In this scenario, the components are tS3Configuration (labeled s3_flights), two tDeltaLakeInput components (labeled flights_latest_version and flights_first_version, respectively), two tAggregateRow components (labeled count_per_flights), two tPartition components (labeled repart), one tMap and one tFileOutputDelimited.
  3. Connect these components using the Row > Main link as the image above presented.
  4. Leave the tS3Configuration component alone without any connection.