Tracking Talend Data Stewardship events and processing data in real time

All actions in Talend Data Stewardship are published to a central topic named data-history, and are available for subscription for a range of use cases.

Consider a scenario where your enterprise product catalog needs on-going updates to fix all anomalies in product records and make sure the product descriptions are correct at all times. To solve this type of challenge, you need to cure your data and manage the integration of the resolved and validated records in the product catalog in real time.

In this example, a consumer group is used to track only the tasks which have been resolved in Talend Data Stewardship, and next ingest the messages from the data-history topic into Slack to indicate the task transitions to the Resolved status for real-time processing. These messages can be used to upload the clean records to the Master Data Management repository in real time.

Here is an architecture diagram which identifies Talend Data Stewardship internal components and their interaction with the Kafka topic.

For more information about the configuration of Kafka topics and consumers, see https://kafka.apache.org/documentation/.