Big Data: new features - 7.2

Talend Data Fabric Release Notes

Version
7.2
Language
English (United States)
Product
Talend Data Fabric
Content
Installation and Upgrade
Release Notes

Spark Job designer enhancements

Feature

Description

Delta Lake The tDeltaLakeInput and tDeltaLakeOutput have been created to leverage this open-source storage layer that brings ACID transactions to Big Data workloads on Apache Spark.
Warning: This feature is in technical preview status.
Apache Spark V2.4 The new Aparch Spark version is supported in the Local mode and in the Spark Batch and Spark Streaming Jobs with Cloudera CDH V6.1.
Databricks
  • Transient clusters are supported for Databricks on AWS.
  • Users can now select a check box to keep their cluster running when submitting Jobs.
Snowflake The Snowflake components for Spark Batch have been created.
Warning: This feature is in technical preview status.
Elasticsearch Elasticsearch V5.6.x and V6.4.x are supported.
Cloud security
  • The support for SSE KMS on S3 has been added.
  • The credentials inheritance feature is now available in the Jobs for Apache Spark.
tFileInputDelimited Users can select a check box to enable the Spark cluster to use multiple executors to read large CSV files in parallel.

Support for Big Data platforms

Feature

Description

Cloudera
  • Cloudera CDH V6.1 is now supported.
  • The Cloudera CDH V6.x series is now officially available among the Dynamic Distributions. They are not on technical preview anymore
MapR The support for MapR has been updated to MapR V6.1 with MEP (MapR Ecosystem Pack) V6.1

Other components

Feature

Description

Google BigQuery
  • The Google BigQuery components now support the regional locations in the Standard Jobs.
  • The tBigQuerySQLRow component has been added.
Couchbase API
  • New tCouchbaseInput and tCouchbaseOutput components have been created to better reflect the logic of the Couchbase Java SDK API.
  • The old tCouchbaseInput and tCouchbaseOutput components have been renamed to tCouchbaseDCPInput and tCouchbaseDCPOutput, respectively, as they use the Database Change Protocol to stream data changes to buckets.

Continuous Integration and Deployment

Feature

Description

Continuous Integration and Deployment: light CommandLine, light installation

The installation of Talend CommandLine is no longer required as the repository that contains the application as well as Talend Maven plugins can now be hosted on your local server and automatically installed during the build process.

To increase your performances and resource utilization, the size of Talend CommandLine has been reduced by half.