CDC in traditional databases - 6.5

Change Data Capture

Talend Big Data
Talend Big Data Platform
Talend Data Fabric
Talend Data Integration
Talend Data Management Platform
Talend Data Services Platform
Talend ESB
Talend MDM Platform
Talend Real-Time Big Data Platform
Talend Studio
Data Governance > Third-party systems > Database components > Change Data Capture
Data Quality and Preparation > Third-party systems > Database components > Change Data Capture
Design and Development > Third-party systems > Database components > Change Data Capture

Talend provides CDC support for all the traditional relational databases. Talend uses a subscriber/publish architecture wherein publisher captures the change data and makes it available to the subscribers. The two main CDC modes are the Trigger mode and Redo/Archive logs mode. Almost all the relational databases use Trigger mode except for Oracle which uses Redo/Archive log mode for capturing changes in the source data.

For Trigger mode to work, the database needs to be already enabled with CDC. This is typically done by stored procedures within that database. For example, in SQL Server, CDC is enabled by executing sys.sp_cdc_enable_db.

Talend needs its own metadata for CDC to work. These are the SUBSCRIBER table, which tracks the tables for changes and the Change table which tracks the changes to the data in the table itself. This is primarily done by creating CDC Metadata in the repository and using the metadata in the Talend CDC components. Talend provides CDC components such as tOracleCDC, tAS400CDC, tDB2CDC, etc.

For more information about Change Data Capture, see the Talend Data Fabric User Guide.

CDC can also be achieved by storing the date and time of when the record was last updated. Then the filter can be applied to the extraction layer of the source system.