All actions in Talend Data Stewardship are
published to a central topic named data-history
, and are available for
subscription for a range of use cases.
Consider a scenario where your enterprise product catalog needs on-going updates to fix all anomalies in product records and make sure the product descriptions are correct at all times. To solve this type of challenge, you need to cure your data and manage the integration of the resolved and validated records in the product catalog in real time.
In this example, a consumer group is used to track only the tasks which have been
resolved in Talend Data Stewardship, and next
ingest the messages from the data-history
topic into Slack to indicate
the task transitions to the Resolved status for real-time processing. These messages can
be used to upload the clean records to the Master Data Management repository in real
time.
Here is an architecture diagram which identifies Talend Data Stewardship internal components and their interaction with the Kafka topic.
For more information about the configuration of Kafka topics and consumers, see https://kafka.apache.org/documentation/.