What's new in R2022-04 - Cloud

Talend Cloud Release Notes

Version
Cloud
Language
English (United States)
Product
Talend Cloud
Module
Talend API Designer
Talend API Tester
Talend Data Inventory
Talend Data Preparation
Talend Data Stewardship
Talend Management Console
Talend Pipeline Designer
Content
Installation and Upgrade
Release Notes

Big Data: new features

Feature

Description

Support of Cloudera Data Engineering experience with AWS on Spark Universal 3.1.x You can now run your Spark Jobs on Cloudera Data Engineering experience using Spark Universal with Spark 3.1.x. You can configure it in the Spark Configuration view of your Spark Jobs.
Support of Databricks runtime 8.x and 9.x with transient clusters on Spark Universal 3.1.x You can now run your Spark Jobs on a transient Databricks cluster using Spark Universal with Spark 3.1.x. You can configure it either in the Spark Configuration view of your Spark Jobs or in the Hadoop Cluster Connection metadata wizard.

When you select this mode, Talend Studio is compatible from Databricks 8.x to 9.x versions.

Support of Spark Universal 3.2.x in Local mode You can now run your Spark Jobs using Spark Universal with Spark 3.2.x in Local mode. You can configure it either in the Spark Configuration view of your Spark Jobs or in the Hadoop Cluster Connection metadata wizard.

When you select this mode, Talend Studio builds the Spark environment in itself at runtime to run the Job locally within the Studio.

Talend Data Mapper Big Data components (tHConvertFile, tHMapFile, tHMapInput and tHMapRecord) are not compatible with Spark 3.2.x for the moment.

Support of Google Dataproc 2.0.x on Spark Universal 3.1.x You can now run your Spark Jobs on a Google Dataproc cluster using Spark Universal with Spark 3.1.x. You can configure it in the Spark Configuration view of your Spark Jobs.

When you select this mode, Talend Studio is compatible with Dataproc 2.0.x version on Spark Batch and Spark Streaming Jobs only. Standard Jobs with Hive components are not supported for the moment.

Support of multiple tGSConfiguration components in Spark Jobs You can now use multiple tGSConfiguration components in one Job to use different Google Storage buckets with different credentials. This feature applies to both Spark Batch and Spark Streaming Jobs.
Support of CDP Public Cloud with Knox and Impala in Standard Jobs If you use CDP Public Cloud to run your Standard Jobs, you can now make use of both Knox and Impala.
Support of AWS EMR 6.3.x, 6.4.x and 6.5.x distributions with Spark Universal 3.1.x You can now run your Spark Jobs on an AWS EMR cluster using Spark Universal with Spark 3.1.x in Yarn cluster mode. You can configure it either in the Spark Configuration view of your Spark Jobs or in the Hadoop Cluster Connection metadata wizard.

When you select this mode, Talend Studio is compatible with AWS EMR 6.3.x, 6.4.x and 6.5.x versions. Spark Jobs are not working with tHiveConnection component for the moment.

Support of Open Source Apache Spark Standalone cluster on Spark Universal 3.2.x You can now run your Spark Jobs in Standalone mode with Spark Universal 3.2.x. You can configure it in the Spark Configuration view of your Spark Jobs.

When you select this mode, Talend Studio connects to a Spark-enabled cluster to run the Job from this cluster.

  • As it is technical preview only, it is not suitable for production environment.
  • Kerberos is not available yet for the technical preview.
Support of Azure Active Directory (AD) Service Principal for Azure Synapse Spark pools in Spark Jobs

When you use Azure Synapse Analytics with Apache Spark pools as a distribution for your Spark Batch and Spark Streaming Jobs, you can now authenticate with Azure AD Service Principal. You can configure it in the Spark Configuration view of your Spark Jobs.

As it is technical preview only, it is not suitable for production environment.

Data Integration: new features

Feature

Description

Studio can now automatically create test branches to test the migration of remote projects

Talend Studio now provides a new Migrate projects or test wizard with the following two options:
  • Migrate: migrates the project without testing.
  • Test: tests the project migration by creating test branches and migrating the project on test branches to keep the production branch safe.

The wizard is displayed when you open a remote project whose version is lower than the Talend Studio version.

New component to write data to an active ADSO through SAP Java Connector version 3

This release provides the tSAPADSOOutput component, which allows you to write data to an active Advanced Data Store Object (ADSO) through SAP Java Connector version 3.

New components to list and delete files respectively on a Samba host

This release provides the following two components.

  • tSambaList, which lists files in a directory on a Samba v2 or v3 host and passes the information about the files to the subsequent component.

  • tSambaDelete: which deletes a file in a directory on a Samba v2 or v3 host or the entire directory.

Support for Microsoft SQL Server version 2019

This release provides support for Microsoft SQL Server version 2019, which allows Azure Active Directory authentication when connecting a Microsoft SQL server database.

Support for Redshift driver v2.x

This release provides support for Redshift driver v2.x. A Driver version option is added to these Redshift components: tRedshiftInput, tRedshiftUnload, tRedshiftOutput, tRedshiftConnection, tRedshiftRow, tRedshiftOutputBulkExec, tRedshiftBulkExec.

The way you specify JDBC parameters varies between driver v1 and driver v2. For details, see tRedshiftConnection standard properties.

Support of Dynamic Schema for tSAPTableInput The Dynamic Schema functionality is now supported for tSAPTableInput in your Standard Jobs with the use of tSetDynamicSchema. The Dynamic Schema functionality allows you to configure a schema in a non-static way, so you won't have to redesign your Job for future schema alteration while ensuring it will work all the time. You are now able to provide tables, columns and joins dynamically.

Data Quality: new features

Feature

Description

New key derivation function The data privacy components have been enhanced with the PBKDF2 key derivation function with 300,000 iterations. This function is the default one and will be used for new Jobs.

For compatibility reasons with Jobs prior to 8.0 R2022-04, PBKDF2 with 65,536 iterations is still supported but deprecated.

Select the key derivation function in the Advanced settings.
The new function is available in:
  • tDataEncrypt
  • tDataDecrypt
  • tDataMasking
  • tDataUnmasking
  • tPatternMasking
  • tPatternUnmasking

Application Integration: new features

Feature

Description

The default Microservice monitoring endpoints are now limited to info, health and Jolokia.

Microservices built with Talend Studio provide access to Spring boot actuator endpoints. The default monitoring endpoints are now limited to info, health and Jolokia.

Continuous Integration: new features

Feature

Description

Talend CI Builder upgraded to version 8.0.4 Talend CI Builder is upgraded from version 8.0.1 to version 8.0.4.

Use Talend CI Builder 8.0.4 in your CI commands or pipeline scripts from this monthly version onwards until a new version of Talend CI Builder is released.

New command to install Talend CommandLine in a standalone CI step A new mvn org.talend.ci:builder-maven-plugin:8.0.4:install command is now available, which allows you to install Talend CommandLine in a standalone CI step without creating a project or generating POM files.