Big Data: new features
|
Feature |
Description |
Available in |
|---|---|---|
| Support of Cloudera Data Engineering experience with AWS on Spark Universal 3.1.x | You can now run your Spark Jobs on Cloudera Data Engineering experience
using Spark Universal with Spark 3.1.x. You can configure it in the
Spark Configuration view of your Spark Jobs.
|
ⓘ Available in: Big Data Big Data Platform Cloud Big Data Cloud Big Data Platform Cloud Data Fabric Data Fabric Real-Time Big Data Platform All subscription-based Talend products with Big Data |
| Support of Databricks runtime 8.x and 9.x with job clusters on Spark Universal 3.1.x | You can now run your Spark Jobs on a job Databricks cluster using Spark
Universal with Spark 3.1.x. You can configure it either in the Spark
Configuration view of your Spark Jobs or in the Hadoop
Cluster Connection metadata wizard. When you select this mode, Talend Studio is compatible from Databricks 8.x to 9.x versions. |
ⓘ Available in: Big Data Big Data Platform Cloud Big Data Cloud Big Data Platform Cloud Data Fabric Data Fabric Real-Time Big Data Platform All subscription-based Talend products with Big Data |
| Support of Spark Universal 3.2.x in Local mode | You can now run your Spark Jobs using Spark Universal with Spark 3.2.x in
Local mode. You can configure it either in the
Spark Configuration view of your Spark Jobs or in the
Hadoop Cluster Connection metadata wizard. When you select this mode, Talend Studio builds the Spark environment in itself at runtime to run the Job locally in the Talend Studio. Talend Data Mapper Big Data components (tHConvertFile, tHMapFile, tHMapInput, and tHMapRecord) are not compatible with Spark 3.2.x for the moment. As it is a beta feature only, it is not suitable for production environment. 
|
ⓘ Available in: Big Data Big Data Platform Cloud Big Data Cloud Big Data Platform Cloud Data Fabric Data Fabric Real-Time Big Data Platform All subscription-based Talend products with Big Data |
| Support of Google Dataproc 2.0.x on Spark Universal 3.1.x | You can now run your Spark Jobs on a Google Dataproc cluster using Spark
Universal with Spark 3.1.x. You can configure it in the Spark
Configuration view of your Spark Jobs. When you select this mode, Talend Studio is compatible with Dataproc 2.0.x version on Spark Batch and Spark Streaming Jobs only. Standard Jobs with Hive components are not supported for the moment. 
|
ⓘ Available in: Big Data Big Data Platform Cloud Big Data Cloud Big Data Platform Cloud Data Fabric Data Fabric Real-Time Big Data Platform All subscription-based Talend products with Big Data |
| Support of multiple tGSConfiguration components in Spark Jobs | You can now use multiple tGSConfiguration components in one Job to use different Google Storage buckets with different credentials. This feature applies to both Spark Batch and Spark Streaming Jobs. |
ⓘ Available in: Big Data Big Data Platform Cloud Big Data Cloud Big Data Platform Cloud Data Fabric Data Fabric Real-Time Big Data Platform All subscription-based Talend products with Big Data |
| Support of CDP Public Cloud with Knox and Impala in Standard Jobs | If you use CDP Public Cloud to run your Standard Jobs, you can now make use of both Knox and Impala. |
ⓘ Available in: Big Data Big Data Platform Cloud Big Data Cloud Big Data Platform Cloud Data Fabric Data Fabric Real-Time Big Data Platform All subscription-based Talend products with Big Data |
| Support of AWS EMR 6.3.x, 6.4.x and 6.5.x distributions with Spark Universal 3.1.x | You can now run your Spark Jobs on an AWS EMR cluster using Spark Universal
with Spark 3.1.x in Yarn cluster mode. You can configure it
either in the Spark Configuration view of your Spark Jobs
or in the Hadoop Cluster Connection metadata wizard. When you select this mode, Talend Studio is compatible with AWS EMR 6.3.x, 6.4.x and 6.5.x versions. Spark Jobs are not working with tHiveConnection component for the moment. |
ⓘ Available in: Big Data Big Data Platform Cloud Big Data Cloud Big Data Platform Cloud Data Fabric Data Fabric Real-Time Big Data Platform All subscription-based Talend products with Big Data |
| Support of Open Source Apache Spark Standalone cluster on Spark Universal 3.2.x | You can now run your Spark Jobs in Standalone mode
with Spark Universal 3.2.x. You can configure it in the Spark
Configuration view of your Spark Jobs. When you select this mode, Talend Studio connects to a Spark-enabled cluster to run the Job from this cluster.
|
ⓘ Available in: Big Data Big Data Platform Cloud Big Data Cloud Big Data Platform Cloud Data Fabric Data Fabric Real-Time Big Data Platform All subscription-based Talend products with Big Data |
| Support of Azure Active Directory (AD) Service Principal for Azure Synapse Spark pools in Spark Jobs |
When you use Azure Synapse Analytics with Apache Spark pools as a distribution for your Spark Batch and Spark Streaming Jobs, you can now authenticate with Azure AD Service Principal. You can configure it in the Spark Configuration view of your Spark Jobs. As it is a beta feature only, it is not suitable for production environment. 
|
ⓘ Available in: Big Data Big Data Platform Cloud Big Data Cloud Big Data Platform Cloud Data Fabric Data Fabric Real-Time Big Data Platform All subscription-based Talend products with Big Data |
Data Integration: new features
|
Feature |
Description |
Available in |
|---|---|---|
|
Talend Studio can now automatically create test branches to test the migration of remote projects |
Talend Studio
now provides a new Migrate projects or test wizard with the
following two options:
The wizard is displayed when you open a remote project whose version is lower than the Talend Studio version. 
|
ⓘ Available in: Big Data Big Data Platform Cloud API Services Platform Cloud Big Data Cloud Big Data Platform Cloud Data Fabric Cloud Data Integration Cloud Data Management Platform Data Fabric Data Integration Data Management Platform Data Services Platform ESB MDM Platform Real-Time Big Data Platform All subscription-based Talend products with Talend Studio |
|
New component to write data to an active ADSO through SAP Java Connector version 3 |
This release provides the tSAPADSOOutput component, which allows you to write data to an active Advanced Data Store Object (ADSO) through SAP Java Connector version 3. |
ⓘ Available in: Big Data Big Data Platform Cloud API Services Platform Cloud Big Data Cloud Big Data Platform Cloud Data Fabric Cloud Data Integration Cloud Data Management Platform Data Fabric Data Integration Data Management Platform Data Services Platform ESB MDM Platform Real-Time Big Data Platform All subscription-based Talend products with Talend Studio |
|
New components to list and delete files respectively on a Samba host |
This release provides the following two components.
|
ⓘ Available in: Big Data Big Data Platform Cloud API Services Platform Cloud Big Data Cloud Big Data Platform Cloud Data Fabric Cloud Data Integration Cloud Data Management Platform Data Fabric Data Integration Data Management Platform Data Services Platform ESB MDM Platform Real-Time Big Data Platform All subscription-based Talend products with Talend Studio |
|
Support for Microsoft SQL Server version 2019 |
This release provides support for Microsoft SQL Server version 2019, which allows Azure Active Directory authentication when connecting a Microsoft SQL server database. |
ⓘ Available in: Big Data Big Data Platform Cloud API Services Platform Cloud Big Data Cloud Big Data Platform Cloud Data Fabric Cloud Data Integration Cloud Data Management Platform Data Fabric Data Integration Data Management Platform Data Services Platform ESB MDM Platform Real-Time Big Data Platform All subscription-based Talend products with Talend Studio |
|
Support for Redshift driver v2.x |
This release provides support for Redshift driver v2.x. A Driver version option is added to these Redshift components: tRedshiftInput, tRedshiftUnload, tRedshiftOutput, tRedshiftConnection, tRedshiftRow, tRedshiftOutputBulkExec, tRedshiftBulkExec. The way you specify JDBC parameters varies between driver v1 and driver v2. For details, see tRedshiftConnection Standard properties. 
|
ⓘ Available in: Big Data Big Data Platform Cloud API Services Platform Cloud Big Data Cloud Big Data Platform Cloud Data Fabric Cloud Data Integration Cloud Data Management Platform Data Fabric Data Integration Data Management Platform Data Services Platform ESB MDM Platform Real-Time Big Data Platform All subscription-based Talend products with Talend Studio |
| Support for Dynamic Schema for tSAPTableInput | The Dynamic Schema functionality is now supported for tSAPTableInput in your Standard Jobs with the use of tSetDynamicSchema. The Dynamic Schema functionality allows you to configure a schema in a non-static way, so you won't have to redesign your Job for future schema alteration while ensuring it will work all the time. You are now able to provide tables, columns and joins dynamically. |
ⓘ Available in: Big Data Big Data Platform Cloud API Services Platform Cloud Big Data Cloud Big Data Platform Cloud Data Fabric Cloud Data Integration Cloud Data Management Platform Data Fabric Data Integration Data Management Platform Data Services Platform ESB MDM Platform Real-Time Big Data Platform All subscription-based Talend products with Talend Studio |
Data Quality: new features
|
Feature |
Description |
Available in |
|---|---|---|
| New key derivation function | The data privacy components have been
enhanced with the PBKDF2 key derivation function with 300,000 iterations. This
function is the default one and will be used for new Jobs. For compatibility reasons with Jobs prior to 8.0 R2022-04, PBKDF2 with 65,536 iterations is still supported but deprecated. Select the key derivation function in the
Advanced settings.
The new function is available in:
|
ⓘ Available in: Big Data Platform Cloud API Services Platform Cloud Big Data Platform Cloud Data Fabric Cloud Data Management Platform Data Fabric Data Management Platform Data Services Platform MDM Platform Real-Time Big Data Platform All Talend Platform and Data Fabric products |
Application Integration: new features
|
Feature |
Description |
Available in |
|---|---|---|
|
The default Microservice monitoring endpoints are now limited to info, health and Jolokia. |
Microservices built with Talend Studio provide access to Spring boot actuator endpoints. The default monitoring endpoints are now limited to info, health and Jolokia. |
ⓘ Available in: Cloud API Services Platform Cloud Data Fabric Data Fabric Data Services Platform ESB MDM Platform Real-Time Big Data Platform All subscription-based Talend products with ESB |
Continuous Integration: new features
|
Feature |
Description |
Available in |
|---|---|---|
| Talend CI Builder upgraded to version 8.0.4 |
Talend
CI Builder is upgraded from version 8.0.1 to version 8.0.4. Use Talend CI Builder 8.0.4 in your CI commands or pipeline scripts from this monthly version onwards until a new version of Talend CI Builder is released. |
ⓘ Available in: Big Data Big Data Platform Cloud API Services Platform Cloud Big Data Cloud Big Data Platform Cloud Data Fabric Cloud Data Integration Cloud Data Management Platform Data Fabric Data Integration Data Management Platform Data Services Platform ESB MDM Platform Real-Time Big Data Platform All subscription-based Talend products with Talend Studio |
| New command to install Talend CommandLine in a standalone CI step | A new mvn org.talend.ci:builder-maven-plugin:8.0.4:install
command is now available, which allows you to install Talend CommandLine
in a standalone CI step without creating a project or generating POM
files. |
ⓘ Available in: Big Data Big Data Platform Cloud API Services Platform Cloud Big Data Cloud Big Data Platform Cloud Data Fabric Cloud Data Integration Cloud Data Management Platform Data Fabric Data Integration Data Management Platform Data Services Platform ESB MDM Platform Real-Time Big Data Platform All subscription-based Talend products with Talend Studio |