Compares a new data set with a lookup data set stored in ElasticSearch, using tMatchIndex. tMatchIndexPredict outputs unique records and suspect duplicates in separate files.
In the potential duplicates output, each record contains the fields from the source records and the fields from the potentially matching lookup records.
For more information about tMatchIndex, see tMatchIndex.
This component can run only with Spark 2.0+ and ElasticSearch 5+.
As this component does not support the Elasticsearch authentication, it cannot run on Databricks.
For more technologies supported by Talend, see Talend components.