Big Data: new features
|
Feature |
Description |
Available in |
|---|---|---|
| Support of Spark 3.0 in local mode for Spark Jobs | Talend now support Spark 3.0 in local mode when running Spark Jobs in
Talend Studio.
Note: The
following elements do not support Spark 3.0 in local mode:
|
ⓘ Available in: Big Data Big Data Platform Cloud Big Data Cloud Big Data Platform Cloud Data Fabric Data Fabric Real-Time Big Data Platform All subscription-based Talend products with Big Data |
| Support of Databricks 7.3 LTS with Spark 3.0 components |
You can now run Spark Batch and
Spark Streaming Jobs on Databricks 7.3 LTS distribution, both on AWS and
on Azure for interactive and transient clusters, with Spark 3.0. The
following components are supported:
Important: As it is a beta feature only,
it is not suitable for production environment.
|
ⓘ Available in: Big Data Big Data Platform Cloud Big Data Cloud Big Data Platform Cloud Data Fabric Data Fabric Real-Time Big Data Platform All subscription-based Talend products with Big Data |
| New options available for transient Databricks clusters | You can now fine tune your configuration when you
create a transient Databricks cluster from the Spark configuration view of your Spark Job. The following
properties are now available:
|
ⓘ Available in: Big Data Big Data Platform Cloud Big Data Cloud Big Data Platform Cloud Data Fabric Data Fabric Real-Time Big Data Platform All subscription-based Talend products with Big Data |
| Inherit credentials from AWS role option available for DynamoDB components in Spark Batch Jobs | The following DynamoDB components now support the
ability to obtain AWS security credentials from Amazon EC2 instance metadata
with the new Inherit credentials from AWS
role option:
This allows you not to specify any access key or secret key in Talend Studio. 
|
ⓘ Available in: Big Data Big Data Platform Cloud Big Data Cloud Big Data Platform Cloud Data Fabric Data Fabric Real-Time Big Data Platform All subscription-based Talend products with Big Data |
Data Integration: new features
|
Feature |
Description |
Available in |
|---|---|---|
|
Libraries sharing further enhancement |
Talend Studio now supports configuring whether to share component libraries to the local libraries repository at startup via the Share libraries to artifact repository at startup check box on view in the Preferences dialog box. |
ⓘ Available in: Big Data Big Data Platform Cloud API Services Platform Cloud Big Data Cloud Big Data Platform Cloud Data Fabric Cloud Data Integration Cloud Data Management Platform Data Fabric Data Integration Data Management Platform Data Services Platform ESB MDM Platform Real-Time Big Data Platform All subscription-based Talend products with Talend Studio |
|
Support for Databricks Delta Lake mapping |
The support for Databricks Delta Lake mapping is provided by the following omponents.
|
ⓘ Available in: Big Data Big Data Platform Cloud API Services Platform Cloud Big Data Cloud Big Data Platform Cloud Data Fabric Cloud Data Integration Cloud Data Management Platform Data Fabric Data Integration Data Management Platform Data Services Platform MDM Platform Real-Time Big Data Platform All subscription-based Talend products except Talend ESB |
|
New options for Update and Delete operations provided |
The Use WHERE conditions table option and the Where conditions table field are provided in the Basic settings view. The change improves the productivity. Components involved:
|
ⓘ Available in: Big Data Big Data Platform Cloud API Services Platform Cloud Big Data Cloud Big Data Platform Cloud Data Fabric Cloud Data Integration Cloud Data Management Platform Data Fabric Data Integration Data Management Platform Data Services Platform ESB MDM Platform Real-Time Big Data Platform All subscription-based Talend products with Talend Studio |
|
tRedshiftBulkExec: new file type supported |
The tRedshiftBulkExec component
can load data stored in Apache Parquet files.
|
ⓘ Available in: Big Data Big Data Platform Cloud API Services Platform Cloud Big Data Cloud Big Data Platform Cloud Data Fabric Cloud Data Integration Cloud Data Management Platform Data Fabric Data Integration Data Management Platform Data Services Platform ESB MDM Platform Real-Time Big Data Platform All subscription-based Talend products with Talend Studio |
|
tFileOutputExcel: new option provided for Excel2007 files |
The tFileOutputExcel components
provides the Truncate characters exceeding
max cell length option, which prevents failures that
occur when a string written to an Excel2007 cell exceeds the maximum
length allowed (that is, 32767 characters).
|
ⓘ Available in: Big Data Big Data Platform Cloud API Services Platform Cloud Big Data Cloud Big Data Platform Cloud Data Fabric Cloud Data Integration Cloud Data Management Platform Data Fabric Data Integration Data Management Platform Data Services Platform ESB MDM Platform Real-Time Big Data Platform All subscription-based Talend products with Talend Studio |
|
tChangeFileEncoding: buffer size customizable |
The tChangeFileEncoding component
provides the Buffer Size field,
allowing you to specify the buffer size for changing the file
encoding.
|
ⓘ Available in: Big Data Big Data Platform Cloud API Services Platform Cloud Big Data Cloud Big Data Platform Cloud Data Fabric Cloud Data Integration Cloud Data Management Platform Data Fabric Data Integration Data Management Platform Data Services Platform ESB MDM Platform Real-Time Big Data Platform All subscription-based Talend products with Talend Studio |
|
Safety Switch option available to tSalesforceBulkExec and tSalesforceOutputBulkExec |
The Safety Switch option is now provided for the
tSalesforceBulkExec and tSalesforceOutputBulkExec components to prevent
excessive memory usage. When the database contains columns that are
longer than 100000 characters, do not use this option.
|
ⓘ Available in: Big Data Big Data Platform Cloud API Services Platform Cloud Big Data Cloud Big Data Platform Cloud Data Fabric Cloud Data Integration Cloud Data Management Platform Data Fabric Data Integration Data Management Platform Data Services Platform ESB MDM Platform Real-Time Big Data Platform All subscription-based Talend products with Talend Studio |
Data Mapper: new features
|
Feature |
Description |
Available in |
|---|---|---|
| New options for decimal elements | In the CSV, Flat, JSON, Map and XML representation
properties, two new options have been added to handle decimal elements and
fix an issue related to implied decimals:
|
ⓘ Available in: Big Data Platform Cloud API Services Platform Cloud Big Data Platform Cloud Data Fabric Cloud Data Management Platform Data Fabric Data Management Platform Data Services Platform MDM Platform Real-Time Big Data Platform All Talend Platform and Data Fabric products |
Data Quality: new features
|
Feature |
Description |
Available in |
|---|---|---|
| Support of Spark 3.0 in local mode | Spark components support Apache Spark 3.0 in local mode, except for tMatchIndex, tMatchIndexPredict, tNLPModel, tNLPPredict, and tNLPPreprocessing. |
ⓘ Available in: Big Data Platform Cloud API Services Platform Cloud Big Data Platform Cloud Data Fabric Cloud Data Management Platform Data Fabric Data Management Platform Data Services Platform MDM Platform Real-Time Big Data Platform All Talend Platform and Data Fabric products |