Deduplicating rows - Cloud

Talend Cloud Data Preparation User Guide

author
Talend Documentation Team
EnrichVersion
Cloud
EnrichProdName
Talend Cloud
task
Data Quality and Preparation > Cleansing data
EnrichPlatform
Talend Data Preparation

You can use the Remove duplicate rows function to easily delete all the rows that are exact duplicates and keep only one in your dataset.

Duplicated information can be introduced in spreadsheets because of human error, like a bad copy and paste for example, as well as automated operations. In this example, you received a dataset containing customer information, where all the rows are systematically duplicated.

You will use the Remove duplicate rows to easily clean your dataset.

Procedure

  1. Click the header of any column from your dataset.
  2. Click the Table tab of the functions panel to display the list of functions that can be applied on the whole table.
  3. Point your mouse over the Remove duplicate rows function to preview its result, and click to apply it.

Results

All the duplicated information has been removed in one simple action, leaving you with only one correct occurrence of each row in your dataset.