What's new in R2022-04 - Cloud - 8.0

Talend Release Notes

Version
Cloud
8.0
Language
English
Product
Talend Big Data
Talend Big Data Platform
Talend Cloud API Services Platform
Talend Cloud Big Data
Talend Cloud Big Data Platform
Talend Cloud Data Fabric
Talend Cloud Data Integration
Talend Cloud Data Management Platform
Talend Data Fabric
Talend Data Integration
Talend Data Management Platform
Talend Data Services Platform
Talend ESB
Talend MDM Platform
Talend Real-Time Big Data Platform
Module
Talend Cloud API Designer
Talend Cloud API Tester
Talend Cloud Data Inventory
Talend Cloud Data Preparation
Talend Cloud Data Stewardship
Talend Cloud Pipeline Designer
Talend Data Preparation
Talend Data Stewardship
Talend Management Console
Talend Studio
Content
Installation and Upgrade
Release Notes
Last publication date
2024-02-13

Big Data: new features

Feature

Description

Available in

Support of Cloudera Data Engineering experience with AWS on Spark Universal 3.1.x You can now run your Spark Jobs on Cloudera Data Engineering experience using Spark Universal with Spark 3.1.x. You can configure it in the Spark Configuration view of your Spark Jobs.

Available in:

Big Data

Big Data Platform

Cloud Big Data

Cloud Big Data Platform

Cloud Data Fabric

Data Fabric

Real-Time Big Data Platform

All subscription-based Talend products with Big Data

Support of Databricks runtime 8.x and 9.x with job clusters on Spark Universal 3.1.x You can now run your Spark Jobs on a job Databricks cluster using Spark Universal with Spark 3.1.x. You can configure it either in the Spark Configuration view of your Spark Jobs or in the Hadoop Cluster Connection metadata wizard.

When you select this mode, Talend Studio is compatible from Databricks 8.x to 9.x versions.

Available in:

Big Data

Big Data Platform

Cloud Big Data

Cloud Big Data Platform

Cloud Data Fabric

Data Fabric

Real-Time Big Data Platform

All subscription-based Talend products with Big Data

Support of Spark Universal 3.2.x in Local mode You can now run your Spark Jobs using Spark Universal with Spark 3.2.x in Local mode. You can configure it either in the Spark Configuration view of your Spark Jobs or in the Hadoop Cluster Connection metadata wizard.

When you select this mode, Talend Studio builds the Spark environment in itself at runtime to run the Job locally in the Talend Studio.

Talend Data Mapper Big Data components (tHConvertFile, tHMapFile, tHMapInput, and tHMapRecord) are not compatible with Spark 3.2.x for the moment.

As it is a beta feature only, it is not suitable for production environment.

Available in:

Big Data

Big Data Platform

Cloud Big Data

Cloud Big Data Platform

Cloud Data Fabric

Data Fabric

Real-Time Big Data Platform

All subscription-based Talend products with Big Data

Support of Google Dataproc 2.0.x on Spark Universal 3.1.x You can now run your Spark Jobs on a Google Dataproc cluster using Spark Universal with Spark 3.1.x. You can configure it in the Spark Configuration view of your Spark Jobs.

When you select this mode, Talend Studio is compatible with Dataproc 2.0.x version on Spark Batch and Spark Streaming Jobs only. Standard Jobs with Hive components are not supported for the moment.

Available in:

Big Data

Big Data Platform

Cloud Big Data

Cloud Big Data Platform

Cloud Data Fabric

Data Fabric

Real-Time Big Data Platform

All subscription-based Talend products with Big Data

Support of multiple tGSConfiguration components in Spark Jobs You can now use multiple tGSConfiguration components in one Job to use different Google Storage buckets with different credentials. This feature applies to both Spark Batch and Spark Streaming Jobs.

Available in:

Big Data

Big Data Platform

Cloud Big Data

Cloud Big Data Platform

Cloud Data Fabric

Data Fabric

Real-Time Big Data Platform

All subscription-based Talend products with Big Data

Support of CDP Public Cloud with Knox and Impala in Standard Jobs If you use CDP Public Cloud to run your Standard Jobs, you can now make use of both Knox and Impala.

Available in:

Big Data

Big Data Platform

Cloud Big Data

Cloud Big Data Platform

Cloud Data Fabric

Data Fabric

Real-Time Big Data Platform

All subscription-based Talend products with Big Data

Support of AWS EMR 6.3.x, 6.4.x and 6.5.x distributions with Spark Universal 3.1.x You can now run your Spark Jobs on an AWS EMR cluster using Spark Universal with Spark 3.1.x in Yarn cluster mode. You can configure it either in the Spark Configuration view of your Spark Jobs or in the Hadoop Cluster Connection metadata wizard.

When you select this mode, Talend Studio is compatible with AWS EMR 6.3.x, 6.4.x and 6.5.x versions. Spark Jobs are not working with tHiveConnection component for the moment.

Available in:

Big Data

Big Data Platform

Cloud Big Data

Cloud Big Data Platform

Cloud Data Fabric

Data Fabric

Real-Time Big Data Platform

All subscription-based Talend products with Big Data

Support of Open Source Apache Spark Standalone cluster on Spark Universal 3.2.x You can now run your Spark Jobs in Standalone mode with Spark Universal 3.2.x. You can configure it in the Spark Configuration view of your Spark Jobs.

When you select this mode, Talend Studio connects to a Spark-enabled cluster to run the Job from this cluster.

  • As it is a beta feature only, it is not suitable for production environment.
  • Kerberos is not available yet for beta.

Available in:

Big Data

Big Data Platform

Cloud Big Data

Cloud Big Data Platform

Cloud Data Fabric

Data Fabric

Real-Time Big Data Platform

All subscription-based Talend products with Big Data

Support of Azure Active Directory (AD) Service Principal for Azure Synapse Spark pools in Spark Jobs

When you use Azure Synapse Analytics with Apache Spark pools as a distribution for your Spark Batch and Spark Streaming Jobs, you can now authenticate with Azure AD Service Principal. You can configure it in the Spark Configuration view of your Spark Jobs.

As it is a beta feature only, it is not suitable for production environment.

Available in:

Big Data

Big Data Platform

Cloud Big Data

Cloud Big Data Platform

Cloud Data Fabric

Data Fabric

Real-Time Big Data Platform

All subscription-based Talend products with Big Data

Data Integration: new features

Feature

Description

Available in

Talend Studio can now automatically create test branches to test the migration of remote projects

Talend Studio now provides a new Migrate projects or test wizard with the following two options:
  • Migrate: migrates the project without testing.
  • Test: tests the project migration by creating test branches and migrating the project on test branches to keep the production branch safe.

The wizard is displayed when you open a remote project whose version is lower than the Talend Studio version.

Available in:

Big Data

Big Data Platform

Cloud API Services Platform

Cloud Big Data

Cloud Big Data Platform

Cloud Data Fabric

Cloud Data Integration

Cloud Data Management Platform

Data Fabric

Data Integration

Data Management Platform

Data Services Platform

ESB

MDM Platform

Real-Time Big Data Platform

All subscription-based Talend products with Talend Studio

New component to write data to an active ADSO through SAP Java Connector version 3

This release provides the tSAPADSOOutput component, which allows you to write data to an active Advanced Data Store Object (ADSO) through SAP Java Connector version 3.

Available in:

Big Data

Big Data Platform

Cloud API Services Platform

Cloud Big Data

Cloud Big Data Platform

Cloud Data Fabric

Cloud Data Integration

Cloud Data Management Platform

Data Fabric

Data Integration

Data Management Platform

Data Services Platform

ESB

MDM Platform

Real-Time Big Data Platform

All subscription-based Talend products with Talend Studio

New components to list and delete files respectively on a Samba host

This release provides the following two components.

  • tSambaList, which lists files in a directory on a Samba v2 or v3 host and passes the information about the files to the subsequent component.

  • tSambaDelete: which deletes a file in a directory on a Samba v2 or v3 host or the entire directory.

Available in:

Big Data

Big Data Platform

Cloud API Services Platform

Cloud Big Data

Cloud Big Data Platform

Cloud Data Fabric

Cloud Data Integration

Cloud Data Management Platform

Data Fabric

Data Integration

Data Management Platform

Data Services Platform

ESB

MDM Platform

Real-Time Big Data Platform

All subscription-based Talend products with Talend Studio

Support for Microsoft SQL Server version 2019

This release provides support for Microsoft SQL Server version 2019, which allows Azure Active Directory authentication when connecting a Microsoft SQL server database.

Available in:

Big Data

Big Data Platform

Cloud API Services Platform

Cloud Big Data

Cloud Big Data Platform

Cloud Data Fabric

Cloud Data Integration

Cloud Data Management Platform

Data Fabric

Data Integration

Data Management Platform

Data Services Platform

ESB

MDM Platform

Real-Time Big Data Platform

All subscription-based Talend products with Talend Studio

Support for Redshift driver v2.x

This release provides support for Redshift driver v2.x. A Driver version option is added to these Redshift components: tRedshiftInput, tRedshiftUnload, tRedshiftOutput, tRedshiftConnection, tRedshiftRow, tRedshiftOutputBulkExec, tRedshiftBulkExec.

The way you specify JDBC parameters varies between driver v1 and driver v2. For details, see tRedshiftConnection Standard properties.

Available in:

Big Data

Big Data Platform

Cloud API Services Platform

Cloud Big Data

Cloud Big Data Platform

Cloud Data Fabric

Cloud Data Integration

Cloud Data Management Platform

Data Fabric

Data Integration

Data Management Platform

Data Services Platform

ESB

MDM Platform

Real-Time Big Data Platform

All subscription-based Talend products with Talend Studio

Support for Dynamic Schema for tSAPTableInput The Dynamic Schema functionality is now supported for tSAPTableInput in your Standard Jobs with the use of tSetDynamicSchema. The Dynamic Schema functionality allows you to configure a schema in a non-static way, so you won't have to redesign your Job for future schema alteration while ensuring it will work all the time. You are now able to provide tables, columns and joins dynamically.

Available in:

Big Data

Big Data Platform

Cloud API Services Platform

Cloud Big Data

Cloud Big Data Platform

Cloud Data Fabric

Cloud Data Integration

Cloud Data Management Platform

Data Fabric

Data Integration

Data Management Platform

Data Services Platform

ESB

MDM Platform

Real-Time Big Data Platform

All subscription-based Talend products with Talend Studio

Data Quality: new features

Feature

Description

Available in

New key derivation function The data privacy components have been enhanced with the PBKDF2 key derivation function with 300,000 iterations. This function is the default one and will be used for new Jobs.

For compatibility reasons with Jobs prior to 8.0 R2022-04, PBKDF2 with 65,536 iterations is still supported but deprecated.

Select the key derivation function in the Advanced settings.
The new function is available in:
  • tDataEncrypt
  • tDataDecrypt
  • tDataMasking
  • tDataUnmasking
  • tPatternMasking
  • tPatternUnmasking

Available in:

Big Data Platform

Cloud API Services Platform

Cloud Big Data Platform

Cloud Data Fabric

Cloud Data Management Platform

Data Fabric

Data Management Platform

Data Services Platform

MDM Platform

Real-Time Big Data Platform

All Talend Platform and Data Fabric products

Application Integration: new features

Feature

Description

Available in

The default Microservice monitoring endpoints are now limited to info, health and Jolokia.

Microservices built with Talend Studio provide access to Spring boot actuator endpoints. The default monitoring endpoints are now limited to info, health and Jolokia.

Available in:

Cloud API Services Platform

Cloud Data Fabric

Data Fabric

Data Services Platform

ESB

MDM Platform

Real-Time Big Data Platform

All subscription-based Talend products with ESB

Continuous Integration: new features

Feature

Description

Available in

Talend CI Builder upgraded to version 8.0.4 Talend CI Builder is upgraded from version 8.0.1 to version 8.0.4.

Use Talend CI Builder 8.0.4 in your CI commands or pipeline scripts from this monthly version onwards until a new version of Talend CI Builder is released.

Available in:

Big Data

Big Data Platform

Cloud API Services Platform

Cloud Big Data

Cloud Big Data Platform

Cloud Data Fabric

Cloud Data Integration

Cloud Data Management Platform

Data Fabric

Data Integration

Data Management Platform

Data Services Platform

ESB

MDM Platform

Real-Time Big Data Platform

All subscription-based Talend products with Talend Studio

New command to install Talend CommandLine in a standalone CI step A new mvn org.talend.ci:builder-maven-plugin:8.0.4:install command is now available, which allows you to install Talend CommandLine in a standalone CI step without creating a project or generating POM files.

Available in:

Big Data

Big Data Platform

Cloud API Services Platform

Cloud Big Data

Cloud Big Data Platform

Cloud Data Fabric

Cloud Data Integration

Cloud Data Management Platform

Data Fabric

Data Integration

Data Management Platform

Data Services Platform

ESB

MDM Platform

Real-Time Big Data Platform

All subscription-based Talend products with Talend Studio