What's new in R2021-09

Big Data: new features

Feature	Description	Available in
Different run modes available for Databricks Jobs	You can now choose how you want to run your Job on Databricks cluster when running a Spark Job with the new Run mode option. The Run mode option is available either in the Spark configuration view of your Spark Jobs or in the Hadoop Cluster Connection metadata wizard. You can choose between the following two modes: Create and run now, allowing you to create and run immediately a Job on Databricks. With this mode, you can retrieve your Job with its ID in your Databricks workspace. Information noteNote: This mode was the default one automatically used before this release. Runs submit, allowing you to submit a one-time run without creating a Job on Databricks. With this mode, nothing is displayed in the user interface and no Job ID are created in your Databricks clusters. It avoids to create too many Job or to reuse an existing one, and thus avoid to reach the hour limit number of created Jobs in Databricks workspace.	All Talend products with Big Data
Support of Knox for CDP Public Cloud Data Hub	When you use a CDP Public Cloud Data Hub instance for your Spark Jobs, you can now authenticate using Knox with CDP 7.1 and onwards in YARN cluster and HDFS modes. The Use Knox check box is available either in the Spark configuration view of your Spark Jobs or in the Hadoop Cluster Connection metadata wizard. For more information, see Setting up the Knox parameters with CDP Public Cloud Data Hub. Knox allows you to provide a single point of authentication only using SSO. This feature is now generally available.	All Talend products with Big Data

Feature

Description

Available in

Different run modes available for Databricks Jobs

You can now choose how you want to run your Job on Databricks cluster when running a Spark Job with the new Run mode option. The Run mode option is available either in the Spark configuration view of your Spark Jobs or in the Hadoop Cluster Connection metadata wizard.

You can choose between the following two modes:

Create and run now, allowing you to create and run immediately a Job on Databricks. With this mode, you can retrieve your Job with its ID in your Databricks workspace.
Information noteNote: This mode was the default one automatically used before this release.
Runs submit, allowing you to submit a one-time run without creating a Job on Databricks. With this mode, nothing is displayed in the user interface and no Job ID are created in your Databricks clusters. It avoids to create too many Job or to reuse an existing one, and thus avoid to reach the hour limit number of created Jobs in Databricks workspace.

All Talend products with Big Data

Support of Knox for CDP Public Cloud Data Hub

When you use a CDP Public Cloud Data Hub instance for your Spark Jobs, you can now authenticate using Knox with CDP 7.1 and onwards in YARN cluster and HDFS modes. The Use Knox check box is available either in the Spark configuration view of your Spark Jobs or in the Hadoop Cluster Connection metadata wizard. For more information, see Setting up the Knox parameters with CDP Public Cloud Data Hub.

Knox allows you to provide a single point of authentication only using SSO.

This feature is now generally available.

All Talend products with Big Data

Data Integration: new features

Feature	Description	Available in
New component available: tAmazonAuroraRow	This release provides the tAmazonAuroraRow component, which allows you to execute query statements on a specified Amazon Aurora database table.	All Talend products with Talend Studio
SAP transport archive updated	The SAP transport archive TBDK900118 is replaced by TBDK900130. TBDK900130 is for Netweaver 7.3. It provides the custom type ZSTALEND_TABLE_FIELDS_IN and makes sure the transport can be installed successfully.	All Talend products with Talend Studio

Data Mapper: new features

Feature	Description	Available in
New option to wrap the output in an array	A new option in the JSON representation allows you to wrap your map output in an array, even if a single object is produced.	All Talend Platform and Data Fabric products

Did this page help you?

If you find any issues with this page or its content – a typo, a missing step, or a technical error – let us know how we can improve!

Leave your feedback here