What's new in R2021-08 - Cloud

Talend Cloud Release Notes

Version
Cloud
Language
English (United States)
Product
Talend Cloud
Module
Talend API Designer
Talend API Tester
Talend Data Inventory
Talend Data Preparation
Talend Data Stewardship
Talend Management Console
Talend Pipeline Designer
Content
Installation and Upgrade
Release Notes

Big Data: new features

Feature

Description

Support of Azure Synapse distribution with Spark 3.0

You can now use Azure Synapse Analytics with Apache Spark pools as a new distribution for your Spark Batch and Spark Streaming Jobs with Spark 3.0 in YARN cluster mode only. You can configure it in the Spark Configuration view of your Spark Jobs, for more information, see Defining the Azure Synapse Analytics connection parameters.

Azure Synapse Analytics allows you to process your data thanks to various analytics engines. With Apache Spark pools you can have various compute capabilities (such as speed and efficiency) and a compatibility with ADLS Gen2 storage.

Important: As it is a beta feature only, it is not suitable for production environment.
Advanced expressions for tMap join conditions You can now use the Filter expression area in the Map Editor of tMap component to enter an advanced expression for your join conditions in Spark Batch Jobs using Dataset API. This feature allows you to filter your data from the join condition with the following expressions:
  • simple expression with <, >, <=, >= or == operators
  • complex expression combining several simple expressions with || or && operators
For example, you can use a complex expression on dates:

Data Integration: new features

Feature

Description

Support of removing old versions of multiple project items

A Cleanup view has been added under the General > Version Management node in the Project Settings dialog box, which allows you to

  • remove all old versions and keep only the latest one for multiple project items, or
  • remove all old versions lower than a specific version for multiple project items.
Warning: There is no dependency check when removing old versions of Jobs, Joblets, Routes, and Routelets. We recommend you to do the removal and validate the cleanup on a branch.

tCouchbaseInput: support for N1QL for Analytics statement

The tCouchbaseInput component now provides the N1QL for Analytics option in the Query type drop-down list. This option allows you to query semistructured data using N1QL for Analytics statements.

tSQSOutput: retrieving and passing message IDs to the subsequent components

The tSQSOutput component can now retrieve IDs of the received messages and pass the message IDs to the subsequent components. This is achieved by adding the MessageID column in the output schema. This function is accessible when Use batch mode option is cleared.

tAzureStoragePut: the use of the + character

The tAzureStoragePut component now provides the Allow to escape the '+' sign in filemask option, which, when selected, treats the + character as a normal character in the Files field. With this option not selected, the + character is treated as a regular expression operator.

tSAPTableInput: NUMC data mapped as string

The tSAPTableInput component now provides the Read NUMC data as string in the dynamic column option, which, when selected, treats data of the NUMC type in the dynamic column as strings. With this option not selected, data of NUMC type is treated as integers.

MongoDB components: support for the SCRAM-SHA-256 SASL authentication mechanism for MongoDB 4.4.x and later versions

The MongoDB components now support the SCRAM-SHA-256 SASL authentication mechanism when MongoDB 4.4.x or a later version is used. See Authentication mechanisms for related information.

Specifying the SAP connection to be used

The tSAPTableInput and the tELTSAPMap component now provide the Connection id field, through which you can specify the ID of the SAP connection to be used. The SAP connection ID is the name of the SAP connection configuration file.

This field requires that the SAP RFC server patch Patch_20210820_TDI-45536_v1-7.3.1 be applied. The patch allows you to establish multiple SAP connections using an SAP RFC server. For information about configuring SAP RFC server and SAP connections, see Configuring the RFC server.

tSAPDataSourceReceiver and tSAPIDocReceiver: the capability of processing partner host information

The tSAPDataSourceReceiver and tSAPIDocReceiver components now provide the capability of processing partner host information. The information identifies the SAP connection from which a message is received.

  • The tSAPDataSourceReceiver component provides a new pre-defined column named partnerHost in its schema.
  • The tSAPIDocReceiver component will insert partner host information to the headers of the messages it extracts.

This field requires that the SAP RFC server patch Patch_20210820_TDI-45536_v1-7.3.1 be applied. The patch allows you to establish multiple SAP connections using an SAP RFC server. For information about configuring SAP RFC server and SAP connections, see Configuring the RFC server.