Improving performances with a Snowflake target - 7.3

Talend Change Data Capture User Guide

Version
7.3
Language
English
Product
Talend Change Data Capture
Module
Talend Change Data Capture
Content
Data Governance > Third-party systems > Database components (Integration) > Change Data Capture
Data Quality and Preparation > Third-party systems > Database components (Integration) > Change Data Capture
Design and Development > Third-party systems > Database components (Integration) > Change Data Capture
Last publication date
2023-12-22

In case of large file, using the SnowFlakeTarget.Put and SnowFlakeTarget.Remove parameters in the DD_USRDATA.DD_PARAM table enables to modify auto-compress mode.

Procedure

  1. Activate the compress mode with the following query:
    UPDATE DD_USRDATA.DD_PARAM SET PARVAL = 'put file://&snowfile @~ auto_compress=true;' WHERE PARNAME='SnowFlakeTarget.Put'
  2. Update removed compressed files with the following query:
    UPDATE DD_USRDATA.DD_PARAM SET PARVAL = 'rm @~ pattern=''&snowfile.*'';' WHERE PARNAME='SnowFlakeTarget.Remove'
    Warning: If other parameters have been added to SnowFlakeTarget.Put and SnowFlakeTarget.Remove, make sure to update the queries with those parameters.