Big Data: new features
|
Feature |
Description |
Available in |
|---|---|---|
| Support of Databricks runtime 8.x to 9.x with Spark Universal 3.1.x with interactive clusters | You can now run your Spark Jobs on an interactive Databricks cluster using
Spark Universal with Spark 3.1.x. You can configure it either in the
Spark Configuration view of your Spark Jobs or in the
Hadoop Cluster Connection metadata wizard. When you select this mode, Talend Studio is compatible from Databricks 8.0 to 9.1 versions. 
|
ⓘ Available in: Big Data Big Data Platform Cloud Big Data Cloud Big Data Platform Cloud Data Fabric Data Fabric Real-Time Big Data Platform All subscription-based Talend products with Big Data |
| Support of CDP Public Cloud authentication via Knox | You can now authenticate using Knox when you use CDP Public Cloud 7.x with Hive to run your Spark Jobs. |
ⓘ Available in: Big Data Big Data Platform Cloud Big Data Cloud Big Data Platform Cloud Data Fabric Data Fabric Real-Time Big Data Platform All subscription-based Talend products with Big Data |
| Support of Cassandra 4 in components and metadata |
Talend Studio
now supports Cassandra 4.x and Cassandra 4.0.x:
The Cassandra driver to be used is now upgraded to version 4.13. |
ⓘ Available in: Big Data Big Data Platform Cloud Big Data Cloud Big Data Platform Cloud Data Fabric Data Fabric Real-Time Big Data Platform All subscription-based Talend products with Big Data |
|
Support of Cloudera Data Engineer experience with Spark Universal 3.1.x |
You can now run your Spark Jobs on Cloudera Data Engineer experience using
Spark Universal with Spark 3.1.x. You can configure it in the Spark
Configuration view of your Spark Jobs. Important: As it
is a beta feature only, it is not suitable for production
environment.
|
ⓘ Available in: Big Data Big Data Platform Cloud Big Data Cloud Big Data Platform Cloud Data Fabric Data Fabric Real-Time Big Data Platform All subscription-based Talend products with Big Data |
|
Support of AWS EMR 6.3.x, 6.4.x and 6.5.x distributions with Spark Universal 3.1.x |
You can now run your Spark Jobs on an AWS EMR cluster using Spark Universal
with Spark 3.1.x. You can configure it either in the Spark
Configuration view of your Spark Jobs or in the Hadoop
Cluster Connection metadata wizard. When you select this mode, Talend Studio is compatible with AWS EMR 6.3.x, 6.4.x and 6.5.x versions. Important: As it is a beta feature only, it is not suitable for
production environment.
|
ⓘ Available in: Big Data Big Data Platform Cloud Big Data Cloud Big Data Platform Cloud Data Fabric Data Fabric Real-Time Big Data Platform All subscription-based Talend products with Big Data |
|
Support of Databricks runtime 8.x to 9.x with Spark Universal 3.1.x with job clusters |
You can now run your Spark Jobs on an job Databricks cluster using Spark
Universal with Spark 3.1.x. You can configure it either in the Spark
Configuration view of your Spark Jobs or in the Hadoop
Cluster Connection metadata wizard. When you select this mode, Talend Studio is compatible from Databricks 8.x to 9.x versions. Important: As it is a beta feature only, it is not suitable for
production environment.
|
ⓘ Available in: Big Data Big Data Platform Cloud Big Data Cloud Big Data Platform Cloud Data Fabric Data Fabric Real-Time Big Data Platform All subscription-based Talend products with Big Data |
Data Integration: new features
|
Feature |
Description |
Available in |
|---|---|---|
| Snowflake JDBC driver upgraded | Snowflake JDBC driver is upgraded to version 3.13.14. |
ⓘ Available in: Big Data Big Data Platform Cloud API Services Platform Cloud Big Data Cloud Big Data Platform Cloud Data Fabric Cloud Data Integration Cloud Data Management Platform Data Fabric Data Integration Data Management Platform Data Services Platform ESB MDM Platform Real-Time Big Data Platform All subscription-based Talend products with Talend Studio |
Data Mapper: new features
|
Feature |
Description |
Available in |
|---|---|---|
| JSON Schema importer | A new option allows you to create structures based on JSON schemas. With
this new option, you can work with any JSON Schema, including FHIR.
|
ⓘ Available in: Big Data Platform Cloud API Services Platform Cloud Big Data Platform Cloud Data Fabric Cloud Data Management Platform Data Fabric Data Management Platform Data Services Platform MDM Platform Real-Time Big Data Platform All Talend Platform and Data Fabric products |
| New JSON representation options | You can now omit null values and empty strings, arrays, and objects in JSON outputs.
|
ⓘ Available in: Big Data Platform Cloud API Services Platform Cloud Big Data Platform Cloud Data Fabric Cloud Data Management Platform Data Fabric Data Management Platform Data Services Platform MDM Platform Real-Time Big Data Platform All Talend Platform and Data Fabric products |
Data Quality: new features
|
Feature |
Description |
Available in |
|---|---|---|
| New validation component |
tDataQualityRules lets Talend Studio
connect to Talend Cloud Data Stewardship
so you can use data quality rules in Talend Studio.
|
ⓘ Available in: Big Data Platform Cloud API Services Platform Cloud Big Data Platform Cloud Data Fabric Cloud Data Management Platform Data Fabric Data Management Platform Data Services Platform MDM Platform Real-Time Big Data Platform All Talend Platform and Data Fabric products |
| Profiling of Parquet file supported | You can now create an analysis from a Parquet file on a Hive
connection. Only flat-columnar-format files are supported for now. |
ⓘ Available in: Big Data Platform Cloud API Services Platform Cloud Big Data Platform Cloud Data Fabric Cloud Data Management Platform Data Fabric Data Management Platform Data Services Platform MDM Platform Real-Time Big Data Platform All Talend Platform and Data Fabric products |
| Tweak configuration in data masking components | You can now use an input column to provide the tweaks in
tDataMasking and tPatternMasking. Select
the new option: Use a column containing the tweaks.
|
ⓘ Available in: Big Data Platform Cloud API Services Platform Cloud Big Data Platform Cloud Data Fabric Cloud Data Management Platform Data Fabric Data Management Platform Data Services Platform MDM Platform Real-Time Big Data Platform All Talend Platform and Data Fabric products |
| Prompt for context variables from the Profiling perspective | You can now configure a prompt to enter the password every time you run an
analysis or report. This prevents from storing the password in the context
settings.
Restriction: Testing the connection is currently not supported.
|
ⓘ Available in: Big Data Platform Cloud API Services Platform Cloud Big Data Platform Cloud Data Fabric Cloud Data Management Platform Data Fabric Data Management Platform Data Services Platform MDM Platform Real-Time Big Data Platform All Talend Platform and Data Fabric products |
Continuous Integration: new features
|
Feature |
Description |
Available in |
|---|---|---|
|
Detection of fixed CVEs while building artifacts |
When using CI to build artifacts, you are now able to use the 'mvn org.talend.ci:builder-maven-plugin:8.0.3:detectCVE' command to detect which artifacts (Standard Jobs, Big Data Jobs) affected by Common Vulnerabilities and Exposures (CVEs) have been fixed since the latest Talend Studio update has been released. The detection of fixed CVEs for Route artifacts is not supported yet. Learn more about this feature in the Talend Software Development Life Cycle Best Practices Guide. |
ⓘ Available in: Big Data Big Data Platform Cloud API Services Platform Cloud Big Data Cloud Big Data Platform Cloud Data Fabric Cloud Data Integration Cloud Data Management Platform Data Fabric Data Integration Data Management Platform Data Services Platform ESB MDM Platform Real-Time Big Data Platform All subscription-based Talend products with Talend Studio |
|
New CI build scripts available in the online documentation |
The CI build scripts provided with the online documentation that allow you to easily build artifacts using Jenkins and/or Azure DevOps have been updated. You can find these scripts in the Downloads tab of the Talend Software Development Life Cycle Best Practices Guide. |
ⓘ Available in: Big Data Big Data Platform Cloud API Services Platform Cloud Big Data Cloud Big Data Platform Cloud Data Fabric Cloud Data Integration Cloud Data Management Platform Data Fabric Data Integration Data Management Platform Data Services Platform ESB MDM Platform Real-Time Big Data Platform All subscription-based Talend products with Talend Studio |