tKafkaCommit properties for Apache Spark Streaming - 7.3

Kafka

Version
7.3
Language
English
Product
Talend Big Data
Talend Big Data Platform
Talend Data Fabric
Talend Real-Time Big Data Platform
Module
Talend Studio
Content
Data Governance > Third-party systems > Messaging components (Integration) > Kafka components
Data Quality and Preparation > Third-party systems > Messaging components (Integration) > Kafka components
Design and Development > Third-party systems > Messaging components (Integration) > Kafka components
Last publication date
2024-02-21

These properties are used to configure tKafkaCommit running in the Spark Streaming Job framework.

The Spark Streaming tKafkaCommit component belongs to the Messaging family.

This component is available in Talend Real Time Big Data Platform and Talend Data Fabric.

Basic settings

Commit offsets from

Select the tKafkaInput component from which the consumed messages are committed.

Usage

Usage rule

To use tKafkaCommit, the Auto-commit offests check box in the Basic settings view of the tKafkaInput component must be cleared.

This component is supported with Spark v2.0 and onwards in the Local Spark mode. It is typically used standalone to commit a batch of offset.

This component, along with the Spark Streaming component Palette it belongs to, appears only when you are creating a Spark Streaming Job.