Configuring the merging process - 6.5

Deduplication

author
Talend Documentation Team
EnrichVersion
6.5
EnrichProdName
Talend Big Data
Talend Big Data Platform
Talend Data Fabric
Talend Data Integration
Talend Data Management Platform
Talend Data Services Platform
Talend ESB
Talend MDM Platform
Talend Open Studio for Big Data
Talend Open Studio for Data Integration
Talend Open Studio for ESB
Talend Open Studio for MDM
Talend Real-Time Big Data Platform
task
Data Governance > Third-party systems > Data Quality components > Deduplication components
Data Quality and Preparation > Third-party systems > Data Quality components > Deduplication components
Design and Development > Third-party systems > Data Quality components > Deduplication components
EnrichPlatform
Talend Studio

Procedure

  1. Double-click tUnite to open its Basic settings view.
  2. Click [...] next to Edit schema to check that the output schema corresponds to the schema from the input tFileInputDelimited components.
  3. Double-click the first tFileOutputDelimited component to display the Basic settings view and define the component properties.
    You have already accepted to propagate the schema to the output components when you defined the input component.
  4. Clear the Define a storage configuration component check box to use the local system as your target file system.
  5. In the Folder field, set the path to the folder which will hold the output data.
  6. From the Action list, select the operation for writing data:
    • Select Create when you run the Job for the first time.

    • Select Overwrite to replace the file every time you run the Job.

  7. Set the row and field separators in the corresponding fields.
  8. Select the Merge results to single file check box, and in the Merge file path field set the path where to output the file of the clean and deduplicated data set.