What's new in R2021-08 - 7.3

Talend Big Data products Release Notes

Version
7.3
Language
English (United States)
EnrichDitaval
Big Data
Product
Talend Big Data
Talend Big Data Platform
Talend Open Studio for Big Data
Talend Real-Time Big Data Platform
Content
Installation and Upgrade
Release Notes

Big Data: new features

Feature

Description

Product

Support of Azure Synapse distribution with Spark 3.0 (technical preview)

You can now use Azure Synapse Analytics with Apache Spark pools as a new distribution for your Spark Batch and Spark Streaming Jobs with Spark 3.0 in YARN cluster mode only. You can configure it in the Spark Configuration view of your Spark Jobs, for more information, see Defining the Azure Synapse Analytics connection parameters.

Azure Synapse Analytics allows you to process your data thanks to various analytics engines. With Apache Spark pools you can have various compute capabilities (such as speed and efficiency) and a compatibility with ADLS Gen2 storage.

Important: As it is technical preview only, it is not suitable for production environment.

Talend Big Data

Talend Big Data Platform

Talend Real-Time Big Data Platform

Advanced expressions for tMap join conditions
You can now use the Filter expression area in the Map Editor of tMap component to enter an advanced expression for your join conditions in Spark Batch Jobs using Dataset API. This feature allows you to filter your data from the join condition with the following expressions:
  • simple expression with <, >, <=, >= or == operators
  • complex expression combining several simple expressions with || or && operators
For example, you can use a complex expression on dates:

Talend Big Data

Talend Big Data Platform

Talend Real-Time Big Data Platform

Data Integration: new features

Feature

Description

Product

Support of removing old versions of multiple project items

A Cleanup view has been added under the General > Version Management node in the Project Settings dialog box, which allows you to

  • remove all old versions and keep only the latest one for multiple project items, or
  • remove all old versions lower than a specific version for multiple project items.
Warning: There is no dependency check when removing old versions of Jobs, Joblets, Routes, and Routelets. We recommend you to do the removal and validate the cleanup on a branch.

Talend Big Data

Talend Big Data Platform

Talend Real-Time Big Data Platform

tCouchbaseInput: support for N1QL for Analytics statement

The tCouchbaseInput component now provides the N1QL for Analytics option in the Query type drop-down list. This option allows you to query semistructured data using N1QL for Analytics statements.

Talend Big Data

Talend Big Data Platform

Talend Real-Time Big Data Platform

tSQSOutput: retrieving and passing message IDs to the subsequent components

The tSQSOutput component can now retrieve IDs of the received messages and pass the message IDs to the subsequent components. This is achieved by adding the MessageID column in the output schema. This function is accessible when Use batch mode option is cleared.

Talend Big Data

Talend Big Data Platform

Talend Real-Time Big Data Platform

tAzureStoragePut: the use of the + character

The tAzureStoragePut component now provides the Allow to escape the '+' sign in filemask option, which, when selected, treats the + character as a normal character in the Files field. With this option not selected, the + character is treated as a regular expression operator.

Talend Big Data

Talend Big Data Platform

Talend Real-Time Big Data Platform

tSAPTableInput: NUMC data mapped as string

The tSAPTableInput component now provides the Read NUMC data as string in the dynamic column option, which, when selected, treats data of the NUMC type in the dynamic column as strings. With this option not selected, data of NUMC type is treated as integers.

Talend Big Data

Talend Big Data Platform

Talend Real-Time Big Data Platform

MongoDB components: support for the SCRAM-SHA-256 SASL authentication mechanism for MongoDB 4.4.x and later versions

The MongoDB components now support the SCRAM-SHA-256 SASL authentication mechanism when MongoDB 4.4.x or a later version is used. See Authentication mechanisms for related information.

Talend Big Data

Talend Big Data Platform

Talend Real-Time Big Data Platform

Specifying the SAP connection to be used

The tSAPTableInput and the tELTSAPMap component now provide the Connection id field, through which you can specify the ID of the SAP connection to be used. The SAP connection ID is the name of the SAP connection configuration file.

This field requires that the SAP RFC server patch Patch_20210820_TDI-45536_v1-7.3.1 be applied. The patch allows you to establish multiple SAP connections using an SAP RFC server. For information about configuring SAP RFC server and SAP connections, see Configuring the RFC server.

Talend Big Data

Talend Big Data Platform

Talend Real-Time Big Data Platform

tSAPDataSourceReceiver and tSAPIDocReceiver: the capability of processing partner host information

The tSAPDataSourceReceiver and tSAPIDocReceiver components now provide the capability of processing partner host information. The information identifies the SAP connection from which a message is received.

  • The tSAPDataSourceReceiver component provides a new pre-defined column named partnerHost in its schema.
  • The tSAPIDocReceiver component will insert partner host information to the headers of the messages it extracts.

This field requires that the SAP RFC server patch Patch_20210820_TDI-45536_v1-7.3.1 be applied. The patch allows you to establish multiple SAP connections using an SAP RFC server. For information about configuring SAP RFC server and SAP connections, see Configuring the RFC server.

Talend Big Data

Talend Big Data Platform

Talend Real-Time Big Data Platform

New components

This release provides the following new components: tFileInputParquet and tFileOutputParquet.

Talend Big Data

Talend Big Data Platform

Talend Real-Time Big Data Platform