What's new in R2022-03

Big Data: new features

Feature	Description	Available in
Support of Databricks runtime 8.x to 9.x with Spark Universal 3.1.x with interactive clusters	You can now run your Spark Jobs on an interactive Databricks cluster using Spark Universal with Spark 3.1.x. You can configure it either in the Spark Configuration view of your Spark Jobs or in the Hadoop Cluster Connection metadata wizard. When you select this mode, Talend Studio is compatible from Databricks 8.0 to 9.1 versions.	All subscription-based Talend products with Big Data
Support of CDP Public Cloud authentication via Knox	You can now authenticate using Knox when you use CDP Public Cloud 7.x with Hive to run your Spark Jobs.	All subscription-based Talend products with Big Data
Support of Cassandra 4 in components and metadata	Talend Studio now supports Cassandra 4.x and Cassandra 4.0.x: When you create a connection to a Cassandra database through the NoSQL Connections metadata wizard. When you use the Cassandra components in Standard Jobs. The Cassandra driver to be used is now upgraded to version 4.13.	All subscription-based Talend products with Big Data
Support of Cloudera Data Engineer experience with Spark Universal 3.1.x	You can now run your Spark Jobs on Cloudera Data Engineer experience using Spark Universal with Spark 3.1.x. You can configure it in the Spark Configuration view of your Spark Jobs. Information noteImportant: As it is a beta feature only, it is not suitable for production environment.	All subscription-based Talend products with Big Data
Support of AWS EMR 6.3.x, 6.4.x and 6.5.x distributions with Spark Universal 3.1.x	You can now run your Spark Jobs on an AWS EMR cluster using Spark Universal with Spark 3.1.x. You can configure it either in the Spark Configuration view of your Spark Jobs or in the Hadoop Cluster Connection metadata wizard. When you select this mode, Talend Studio is compatible with AWS EMR 6.3.x, 6.4.x and 6.5.x versions. Information noteImportant: As it is a beta feature only, it is not suitable for production environment.	All subscription-based Talend products with Big Data
Support of Databricks runtime 8.x to 9.x with Spark Universal 3.1.x with job clusters	You can now run your Spark Jobs on an job Databricks cluster using Spark Universal with Spark 3.1.x. You can configure it either in the Spark Configuration view of your Spark Jobs or in the Hadoop Cluster Connection metadata wizard. When you select this mode, Talend Studio is compatible from Databricks 8.x to 9.x versions. Information noteImportant: As it is a beta feature only, it is not suitable for production environment.	All subscription-based Talend products with Big Data

Data Integration: new features

Feature	Description	Available in
Snowflake JDBC driver upgraded	Snowflake JDBC driver is upgraded to version 3.13.14.	All subscription-based Talend products with Talend Studio

Data Mapper: new features

Feature	Description	Available in
JSON Schema importer	A new option allows you to create structures based on JSON schemas. With this new option, you can work with any JSON Schema, including FHIR.	All Talend Platform and Data Fabric products
New JSON representation options	You can now omit null values and empty strings, arrays, and objects in JSON outputs.	All Talend Platform and Data Fabric products

Data Quality: new features

Feature	Description	Available in
New validation component	tDataQualityRules lets Talend Studio connect to Talend Cloud Data Stewardship so you can use data quality rules in Talend Studio.	All Talend Platform and Data Fabric products
Profiling of Parquet file supported	You can now create an analysis from a Parquet file on a Hive connection. Only flat-columnar-format files are supported for now.	All Talend Platform and Data Fabric products
Tweak configuration in data masking components	You can now use an input column to provide the tweaks in tDataMasking and tPatternMasking. Select the new option: Use a column containing the tweaks.	All Talend Platform and Data Fabric products
Prompt for context variables from the Profiling perspective	You can now configure a prompt to enter the password every time you run an analysis or report. This prevents from storing the password in the context settings. Information noteRestriction: Testing the connection is currently not supported.	All Talend Platform and Data Fabric products

Continuous Integration: new features

Feature	Description	Available in
Detection of fixed CVEs while building artifacts	When using CI to build artifacts, you are now able to use the 'mvn org.talend.ci:builder-maven-plugin:8.0.3:detectCVE' command to detect which artifacts (Standard Jobs, Big Data Jobs) affected by Common Vulnerabilities and Exposures (CVEs) have been fixed since the latest Talend Studio update has been released. The detection of fixed CVEs for Route artifacts is not supported yet. Learn more about this feature in the Talend Software Development Life Cycle Best Practices Guide.	All subscription-based Talend products with Talend Studio
New CI build scripts available in the online documentation	The CI build scripts provided with the online documentation that allow you to easily build artifacts using Jenkins and/or Azure DevOps have been updated. You can find these scripts in the Downloads tab of the Talend Software Development Life Cycle Best Practices Guide.	All subscription-based Talend products with Talend Studio

Feature

Description

Available in

Detection of fixed CVEs while building artifacts

When using CI to build artifacts, you are now able to use the 'mvn org.talend.ci:builder-maven-plugin:8.0.3:detectCVE' command to detect which artifacts (Standard Jobs, Big Data Jobs) affected by Common Vulnerabilities and Exposures (CVEs) have been fixed since the latest Talend Studio update has been released.

The detection of fixed CVEs for Route artifacts is not supported yet.

Learn more about this feature in the Talend Software Development Life Cycle Best Practices Guide.

All subscription-based Talend products with Talend Studio

New CI build scripts available in the online documentation

The CI build scripts provided with the online documentation that allow you to easily build artifacts using Jenkins and/or Azure DevOps have been updated.

You can find these scripts in the Downloads tab of the Talend Software Development Life Cycle Best Practices Guide.