Extracting change data

Change Data Capture

author
Talend Documentation Team
EnrichVersion
6.4
EnrichProdName
Talend Data Services Platform
Talend Data Integration
Talend Data Fabric
Talend Big Data
Talend Big Data Platform
Talend Data Management Platform
Talend ESB
Talend MDM Platform
Talend Real-Time Big Data Platform
task
Data Quality and Preparation > Third-party systems > Database components > Change Data Capture
Data Governance > Third-party systems > Database components > Change Data Capture
Design and Development > Third-party systems > Database components > Change Data Capture
EnrichPlatform
Talend Studio

About this task

After setting up the CDC environment, you can now design a job using the Mysql CDC component to incrementally extract the change data from the Leadfact table. To do that:

Procedure

  1. From the Palette, drop the tMysqlCDC and tLogRow components to the design workspace.
  2. Link the two components using a Row Main link.
  3. Double-click the tMysqlCDC component to define its properties.
  4. Set Property Type to Repository and then select the select the schema corresponding to your Mysql database table, CDC_connection in this scenario. The connection details will display automatically in the corresponding fields
    Note:

    If you have not stored the CRM data warehouse connection details in the Metadata entry in the Repository, select Built-in in the property type list and set the connection details manually.

  5. In the Schema using CDC field, select Repository and then select the schema of the Leadfact table stored in the Metadata entry.
  6. In the Table using CDC field, enter the name of the table captured by the CDC, in this scenario Leadfact.
  7. In the Subscriber field, enter the name of the subscriber that will extract modified data, Sub_Mktg, Sub_Sales, and Sub_Finance for the Marketing, Sales and Treasury Departments respectively.
  8. In the Events to catch field, select the check boxes corresponding to the type of the modified data the subscriber will extract. In this scenario, select the three check boxes for the three subscribers.
  9. Double-click the tLogRow component to set is properties.
  10. Click the Sync columns button to retrieve the schema from the preceding component.
  11. Save your job and press F6 to execute it.

Results

The customer names are converted to upper case and the modification type displays here is U to stand for Update.

Once these modifications are extracted, they are no more available in the modified table. To verify their extraction, right-click the Leadfact table catched by the CDC and then select Views All Changes. The extracted changes do not display anymore.