What's new in R2021-01 - Cloud

Talend Cloud Release Notes

EnrichVersion
Cloud
EnrichProdName
Talend Cloud
EnrichPlatform
Talend API Designer
Talend API Tester
Talend Data Inventory
Talend Data Preparation
Talend Data Stewardship
Talend Management Console
Talend Pipeline Designer
task
Installation and Upgrade
Release Notes

Big Data: new features

Feature

Description

Assume Role configuration for Databricks 5.5 LTS and 6.4 distributions

When you are running a Job on Databricks 5.5 LTS or 6.4 and you want to write and read data from S3, you can now make your Job temporarily assume a role and the permissions associated with this role.

This allows you not to specify the secret and access keys to Databricks clusters in the tS3Configuration component. You now only have to specify the Amazon Resource Name (ARN) of the role to assume in the Spark configuration view and enter the bucket name then select the Inherit credentials from AWS check box in the Basic settings view of the tS3Configuration component.

Basic Assume Role configuration in tS3Configuration component When you enable the Assume Role option in the tS3Configuration component, you can now configure the following properties from the Basic settings view to fine tune your configuration:
  • Serial Number
  • Token Code
  • Tags
  • Transitive Tag Keys
  • Policy ARNs
  • Policy

This feature is now available for the CDP Private Cloud Base 7.1 distribution.

Topic, partition, and key options available in Kafka components You can now add information about the key and the partition used for the messages in the tKafkaOutput component. The tKafkaInput component will read these information in its output schema thanks to the following new attributes: topic, partition, and key.

This feature allows you to retrieve and show more information in the Kafka message from the topic.

tKafkaCommit available in Spark Streaming Jobs You can now use the tKafkaCommit component in your Spark Streaming Jobs with Spark v2.0 and onwards in the Local Spark mode. This component allows you to manually control when the offset is commited. It enables to have a commit in one go rather than having an auto-commit at a given time interval.
Deprecated distributions The following distributions are now deprecated:
  • HDP 2.6.0 and backwards
  • Cloudera CDH 5.13 and backwards
  • MapR 5.2.0 and backwards
  • Microsoft HD Insight 3.4 and backwards
  • Databricks 3.5 LTS and backwards
  • Cloudera Altus 1.0
  • Dataproc 1.1

Data Integration: new features

Feature

Description

Shared mode for Talend Studio Talend Studio now supports the shared mode, which allows each user on the machine where Talend Studio is installed to work with different configuration and workspace folders.
Libraries sharing enhancement

Talend Studio now supports:

  • configuring whether to share libraries to the local libraries repository at startup
  • sharing libraries manually after startup

By default, the libraries are not shared at Talend Studio startup to improve the startup performance.

SAP function extraction path customizable

You can specify the path for the SAP function to generate the files that hold the data extracted. Components applied:

  • tELTSAPMap
  • tSAPDSOInput (with Use FTP-Batch Options selected in the Basic settings view)
  • tSAPODPInput (with Use FTP-Batch Options selected in the Basic settings view)
  • tSAPInfoCubeInput (with Use FTP-Batch Options selected in the Basic settings view)

tGPGDecrypt: specifying additional parameters for the GPG decrypt command

The Use extra parameters option is provided, allowing you to specify additional parameters for the GPG decrypt command.

Support for Greenplum 6.x

This release provides support for Greenplum 6.x.

Greenplum components: the default Database driver changed

For Greenplum components, the database driver defaults to Greenplum.

tGreenplumGPLoad improved

Multiple new features/options are added to tGreenplumGPLoad. As listed below.

  • The Populate column list based on the schema option in the Basic settings view, which adds the columns defined in the schema to the YAML file.
  • New parameters provided in the Addition options table: LOG_ERRORS, MAX_LINE_LENGTH, EXTERNAL_SCHEMA (_ext_stg_objects), PRELOAD_TRUNCATE, PRELOAD_REUSE_TABLES, PRELOAD_STAGING_TABLE, PRELOAD_FAST_MATCH, SQL_BEFORE LOAD, and SQL_AFTER LOAD.
  • The Remove datafile on successful execution option and the Gzip compress the datafile option in the Advanced settings view, which removes the datafile when the load operation completes successfully and compresses the datafile using Gzip.
  • New global variables provided: NB_LINE_INSERTED, NB_LINE_UPDATED, NB_DATA_ERRORS, GPLOAD_STATUS, and GPLOAD_RUNTIME.

Data Quality: new features

Feature

Description

Shared mode Talend Studio now supports the shared mode. If you enable it, some paths change:
  • For tBRMS, the path to the Drools folder is C:/Users/user-account/studio-path/Drools/
  • For tDqReportRun, the path to the Generated reports folder is C:/Users/user-account/studio-path/Generated reports/
  • For the synonym indexes, the path to the addons folder is C:/Users/user-account/studio-path/addons/
Supported databases SAP Hana is now supported in the Profiling perspective for Table, View and Calculation view schemas.
tDataMasking

tDataUnmasking

The Dynamic data type is now supported by the Standard component.