Configuring a Spark stream for your Apache Spark streaming Job - 8.0

Talend Real-Time Big Data Platform Getting Started Guide

Version
8.0
Language
English
Operating system
Real-Time Big Data Platform
Product
Talend Real-Time Big Data Platform
Module
Talend Administration Center
Talend Installer
Talend Runtime
Talend Studio
Content
Data Quality and Preparation > Cleansing data
Data Quality and Preparation > Profiling data
Design and Development
Installation and Upgrade
Last publication date
2024-03-13
Define how often your Spark Job creates and processes micro batches.

Procedure

  1. In the Batch size field, enter the time interval at the end of which the Job reviews the source data to identify changes and processes the new micro batches.
  2. If needs be, select the Define a streaming timeout check box and in the field that is displayed, enter the time frame at the end of which the streaming Job automatically stops running.