Setting up the Job

Deduplication

author
Talend Documentation Team
EnrichVersion
6.5
EnrichProdName
Talend Big Data Platform
Talend Big Data
Talend Open Studio for Big Data
Talend Data Management Platform
Talend Real-Time Big Data Platform
Talend Data Integration
Talend ESB
Talend Data Services Platform
Talend Open Studio for Data Integration
Talend Open Studio for ESB
Talend MDM Platform
Talend Data Fabric
Talend Open Studio for MDM
task
Design and Development > Third-party systems > Data Quality components > Deduplication components
Data Governance > Third-party systems > Data Quality components > Deduplication components
Data Quality and Preparation > Third-party systems > Data Quality components > Deduplication components
EnrichPlatform
Talend Studio

Procedure

  1. Set up the first subjob:
    1. Drop the following components from the Palette onto the design workspace: tFileInputDelimited, two tFilterRow components and tFileOutputDelimited.

      Use the Main link to connect the components.

    2. Connect tFileInputDelimited to the first tFilterRow component.
    3. Connect the first tFilterRow component to tRuleSurvivorship.
    4. Connect tRuleSurvivorship to the second tFilterRow component.
    5. Connect the second tFilterRow component to tFileOutputDelimited.
  2. Set up the second subjob:
    1. Drop the following components from the Palette onto the design workspace: two tFileInputDelimited components, tFilterColumns, tUnite and tFileOutputDelimited.

      Use the Main link to connect the components.

    2. Connect the first tFileInputDelimited component to tFilterColumn.
    3. Connect tFilterColumn to tUnite.
    4. Connect the second tFileInputDelimited component to tUnite.
    5. Connect tUnite to tFileOutputDelimited.
  3. Connect the tFileInputDelimited from the first subjob to the tFileInputDelimited from the second subjob using a Trigger > OnSubjobOk link.

Results