Setting up the Job - 7.0

Matching with machine learning

author
Talend Documentation Team
EnrichVersion
7.0
EnrichProdName
Talend Big Data Platform
Talend Data Fabric
Talend Data Management Platform
Talend Data Services Platform
Talend MDM Platform
Talend Real-Time Big Data Platform
task
Data Governance > Third-party systems > Data Quality components > Matching components > Matching with machine learning components
Data Quality and Preparation > Third-party systems > Data Quality components > Matching components > Matching with machine learning components
Design and Development > Third-party systems > Data Quality components > Matching components > Matching with machine learning components
EnrichPlatform
Talend Data Stewardship
Talend Studio

Before you begin

  • You have generated the suspect data pairs by using the tMatchPairing component.

  • You added a label next to the second record in each suspect pair to say whether it is a duplicate record or not or whether it is a possible duplicate as well:

    
    480060609;DFSS_AgencySiteLies_2012.csv;Catholic Charities of the Archdiocese of Chicago St. Joseph;4800 S. Paulina; st. joseph_1;;
    480060609;purple_binder_early_childhood.csv;Catholic Charities Chicago - St. Joseph;4800 S Paulina Street; st. joseph_1;0.8058642705131237;YES
    425760624;chapin_dfss_providers_2011_070212.csv;CHICAGO PUBLIC SCHOOLS GOLDBLATT, NATHAN R.;4257 W ADAMS; r._20;;
    422560653;chapin_dfss_providers_2011_070212.csv;CHICAGO PUBLIC SCHOOLS ROBINSON, JACKIE R.;4225 S LAKE PARK AVE; r._20;0.8219437219200757;NO

    The labels used in this example are YES or NO, but you can use any label you like and more than two.

Procedure

  1. Drop the following components from the Palette onto the design workspace: tFileInputDelimited and tMatchModel.
  2. Connect the components together using the Row > Main link.
  3. Check that you have defined the connection to the Spark cluster in the Run > Spark Configuration view as described in Computing suspect pairs and suspect sample from source data.

Results