Data Quality: new features - 8.0

Talend Big Data products Release Notes

Version
8.0
Language
English (United States)
EnrichDitaval
Big Data
Product
Talend Big Data
Talend Big Data Platform
Talend Open Studio for Big Data
Talend Real-Time Big Data Platform
Content
Installation and Upgrade
Release Notes

From Talend Studio 8.0, you must use the Feature manager to install DQ features and components.

Feature

Description

Product

New Apache Spark version The DQ components now support Apache Spark 3.0.1 in local mode.

Talend Big Data Platform

Talend Real-Time Big Data Platform

Databricks support The DQ components now support Databricks 7.3 LTS on AWS and Azure.

Talend Big Data Platform

Talend Real-Time Big Data Platform

New component tMatchIndexPredict for Apache Spark Streaming has been added.

Talend Big Data Platform

Talend Real-Time Big Data Platform

tDataMasking/tDataUnmasking

tDataEncrypt/tDataDecrypt

These components now support Dynamic data type as input. The output data type is also Dynamic.

Talend Big Data Platform

Talend Real-Time Big Data Platform

tPersonator
  • The component now supports the Date and Long data types as input.
  • The new IP address input field lets you have details on the IP address such as the connection speed, country...

Talend Big Data Platform

Talend Real-Time Big Data Platform

New tables in reports

Delivered in 7.3 R2021-10 monthly release

Two tables have been added to the reports to get details on the blocking and matching keys.

Talend Big Data Platform

Talend Real-Time Big Data Platform

SAP Hana

Delivered in 7.3 R2021-01 monthly release

SAP Hana is now supported for Table, View and Calculation View schemas.

Talend Big Data Platform

Talend Real-Time Big Data Platform

tDataMasking/tDataUnmasking

Delivered in 7.3 R2020-11 monthly release

New bank account masking functions:
  • Mask account number and keep original country and bank code: This function masks IBAN and US bank account numbers. It keeps the original country and bank/branch code.
  • Unmask account number and keep original country and bank code: This function unmasks IBAN and US bank account numbers masked with the preceding function.
Two bank account masking functions have been renamed:
  • Mask account number and keep original country renamed to Mask bank code and account number and keep original country
  • Unmask account number and keep original country renamed to Unmask bank code and account number and keep original country

The bank account masking functions and Generate account number and keep original country now handle more national check digits. This means that, for the following countries, the output national check digits are conform to the corresponding country: Belgium, Bosnia, Croatia, Czech Republic, Estonia, France, Hungary, Italy, Mauritania, Monaco, Montenegro, Norway, Portugal, San Marino, Serbia, Slovakia, Slovenia, Spain, Timor-Leste, Tunisia.

Talend Big Data Platform

Talend Real-Time Big Data Platform

New phone number statistics indicator

Delivered in 7.3 R2020-10 monthly release

The new Valid Phone Number for Region Count indicator computes the phone numbers that are recognized as valid for a certain region.

Talend Big Data Platform

Talend Real-Time Big Data Platform