Enhancements of Spark Job designer - 6.3

Talend Real-time Big Data Platform Release Notes

EnrichVersion
6.3
EnrichProdName
Talend Real-Time Big Data Platform
task
Installation and Upgrade
EnrichPlatform
Talend Activity Monitoring Console
Talend Administration Center
Talend Artifact Repository
Talend CommandLine
Talend Data Preparation
Talend Data Stewardship
Talend DQ Portal
Talend ESB
Talend Identity Management
Talend Installer
Talend JobServer
Talend Log Server
Talend Project Audit
Talend Repository Manager
Talend Runtime
Talend SAP RFC Server
Talend Studio
  1. Spark 2.0 is fully supported.

  2. More components are available in Spark Batch and Spark Streaming Jobs regarding the following technologies or domains:

    • MapR-DB

    • MapR-Streams

    • Data preparation

    • Data quality

  3. The Joblet component has been introduced to Spark Jobs in order to factorize recurrent processing or complex transformation steps to ease the reading of a complex Spark Job.

  4. Enhancements of existing components:

    • Some Data Mapper components and Data Quality components

    • The following components for Spark Batch have been enabled to read data in partitions:

      1. tJDBCInput

      2. tMySqlInput

      3. tOracleInput

      4. tTeradataInput

    • The Region list has been added to tS3Configuration.

    • The Cassandra components have been upgraded to support Spark 2.0.

  5. The Spark checkpointing feature has been added to Spark Batch Jobs.