This scenario applies only to a subscription-based Talend Platform solution with Big data or Talend Data Fabric.
For more technologies supported by Talend, see Talend components.
After indexing lookup data in Elasticsearch using tMatchIndex, you do not need to restart the matching process from scratch. The tMatchIndexPredict component compares new data records with the lookup stored in ElasticSearch.
In this example, a list of early childhood education centers in Chicago coming from ten different source has been cleaned, deduplicated and indexed in Elasticsearch. You want to match new records which contain information about early childhood education centers in Chicago against the reference data set stored in Elasticsearch.
tMatchIndexPredict uses pairing and matching models to group together records from the input data and the matching records from the reference data set indexed in Elasticsearch and label the suspect pairs.
tMatchIndexPredict outputs potential duplicates and unique records in separate files.
You generated a pairing model.
For examples of how to generate a pairing model, see Scenario 2: Computing suspect pairs and suspect sample from source data and Scenario 1: Computing suspect pairs and writing a sample in Talend Data Stewardship.
You generated a matching model.
For examples of how to generate a matching model, see Scenario 2: Generating a matching model and Scenario 1: Generating a matching model from a Grouping campaign.
Clean and deduplicated data has been indexed in Elasticsearch to match against new data records and determine whether they are unique records or suspect duplicates.
For further information, see Scenario: Indexing a reference data set in Elasticsearch.
The Elasticsearch search cluster must be running ElasticSearch 5+.