Skip to main content

Big Data

Feature

Description

Available in

Support for HPE Ezmeral Runtime Enterprise 5.4 on Kubernetes with Spark 3.1.x You can now run your Spark Batch and Streaming Jobs on Kubernetes with Livy and Datatap using Spark Universal with Spark 3.1.x.

All subscription-based Talend products with Big Data

Support for Databricks 12.x runtime with Spark Universal 3.3.x
Availability-noteBeta contentBeta
You can now run your Spark Batch and Streaming Jobs on all-purpose and job clusters on Google Cloud Platform (GCP), AWS, and Azure using Spark Universal with Spark 3.3.x. You can configure it either in the Spark Configuration view of your Spark Jobs or in the Hadoop Cluster Connection metadata wizard.

When you select this mode, Talend Studio is compatible with Databricks 12.x version.

All subscription-based Talend products with Big Data

Support for Amazon EMR 6.8.0 and 6.9.0 with Spark Universal 3.3.x
Availability-noteBeta contentBeta
You can now run your Spark Jobs on an Amazon EMR cluster using Spark Universal with Spark 3.3.x in Yarn cluster mode. You can configure it either in the Spark Configuration view of your Spark Jobs or in the Hadoop Cluster Connection metadata wizard.

When you select this mode, Talend Studio is compatible with Amazon EMR 6.8.0 and 6.9.0 versions.

With the Beta version for this feature, the following known issues exist with a workaround:
  • Spark Batch Jobs with HBase never end, make sure to use htrace-core4-4.2.0-incubating.jar in the /usr/lib/hbase/lib.
  • Spark Jobs with Redshift components have runtime exception, make sure to use the Hadoop 3.3.1 version.

All subscription-based Talend products with Big Data

Support for MongoDB v4+ for Spark Streaming 3.1 and onwards
Availability-noteBeta contentBeta
Talend Studio now supports MongoDB v4+ with Spark 3.1 and onwards versions for the following components in your Spark Streaming Jobs using Dataset:
  • tMongoDBConfiguration
  • tMongoDBInput
  • tMongoDBLookupInput
  • tMongoDBOutput

With the Beta version for this feature, the MongoDB version to select from the DB Version drop-down list is MongoDB 3.2+.

All subscription-based Talend products with Big Data

Did this page help you?

If you find any issues with this page or its content – a typo, a missing step, or a technical error – let us know how we can improve!