tJoin - 6.3

Talend Open Studio for Big Data Components Reference Guide

EnrichVersion
6.3
EnrichProdName
Talend Open Studio for Big Data
task
Data Governance
Data Quality and Preparation
Design and Development
EnrichPlatform
Talend Studio

Function

tJoin joins two tables by doing an exact match on several columns. It compares columns from the main flow with reference columns from the lookup flow and outputs the main flow data and/or the rejected data.

Purpose

This component helps you ensure the data quality of any source data against a reference data source.

tJoin properties

Component family

Processing

 

Basic settings

Schema and Edit schema

A schema is a row description, it defines the number of fields to be processed and passed on to the next component. The schema is either Built-in or stored remotely in the Repository.

Since version 5.6, both the Built-In mode and the Repository mode are available in any of the Talend solutions.

Click Edit schema to make changes to the schema. If the current schema is of the Repository type, three options are available:

  • View schema: choose this option to view the schema only.

  • Change to built-in property: choose this option to change the schema to Built-in for local changes.

  • Update repository connection: choose this option to change the schema stored in the repository and decide whether to propagate the changes to all the Jobs upon completion. If you just want to propagate the changes to the current Job, you can select No upon completion and choose this schema metadata again in the [Repository Content] window.

 

 

Built-in: You create and store the schema locally for this component only. Related topic: see Talend Studio User Guide.

 

 

Repository: The schema already exists and is stored in the Repository, hence can be reused in various projects and Job flowcharts. Related topic: see Talend Studio User Guide.

 

Include lookup columns in output

Select this check box to include the lookup columns you define in the output flow.

Key definition

Input key attribute

Select the column(s) from the main flow that needs to be checked against the reference (lookup) key column.

 

Lookup key attribute

Select the lookup key columns that you will use as a reference against which to compare the columns from the input flow.

 

Inner join (with reject output)

Select this check box to join the two tables first and gather the rejected data from the main flow.

Advanced settings

tStat Catcher Statistics

Select this check box to collect log data at the component level.

Global Variables

ERROR_MESSAGE: the error message generated by the component when an error occurs. This is an After variable and it returns a string. This variable functions only if the Die on error check box is cleared, if the component has this check box.

A Flow variable functions during the execution of a component while an After variable functions after the execution of the component.

To fill up a field or expression with a variable, press Ctrl + Space to access the variable list and choose the variable to use from it.

For further information about variables, see Talend Studio User Guide.

Usage

This component is not startable and it requires two input components and one or more output components.

Limitation/prerequisite

n/a

Log4j

If you are using a subscription-based version of the Studio, the activity of this component can be logged using the log4j feature. For more information on this feature, see Talend Studio User Guide.

For more information on the log4j logging levels, see the Apache documentation at http://logging.apache.org/log4j/1.2/apidocs/org/apache/log4j/Level.html.

Scenario 1: Doing an exact match on two columns and outputting the main and rejected data

This scenario describes a five-component Job aiming at carrying out an exact match between the firstnameClient column of an input file against the data of the reference input file, and the lastnameClient column against the data of the reference input file. The outputs of this exact match are written in two separate files: exact data are written in an Excel file, and inaccurate data are written in a delimited file.

In this scenario, we have already stored the input schemas of the input and reference files in the Repository. For more information about storing schema metadata in the Repository tree view, see Talend Studio User Guide.

Dropping and linking the components

  1. In the Repository tree view, expand Metadata and the file node where you have stored the input schemas and drop the relevant file onto the design workspace.

    The [Components] dialog box appears.

  2. Select tFileInputDelimited from the list and click OK to close the dialog box.

    The tFileInputDelimited component displays in the workspace. The input file used in this scenario is called ClientSample. It holds four columns including the two columns firstnameClient and lastnameClient we want to do the exact match on.

  3. Do the same for the second input file you want to use as a reference, ClientSample_Update in this scenario.

  4. Drop the following components from the Palette onto the design workspace: tJoin, tFileOutputExcel, and tFileOutputDelimited.

  5. Connect the main and reference input files to tJoin using Main links. The link between the reference input file and tJoin appears as a lookup link on the design workspace.

  6. Connect tJoin to tFileOutputExcel using the Main link and tJoin to tFileOutputDelimited using the Inner join reject link.

Configuring the components

  1. If needed, double-click the main and reference input files to display their Basic settings views. All their property fields are automatically filled in. If you do not define your input files in the Repository, fill in the details manually after selecting Built-in in the Property Type field.

  2. Double click tJoin to display its Basic settings view and define its properties.

  3. Click the Edit schema button to open a dialog box that displays the data structure of the input files, define the data you want to pass to the output components, three columns in this scenario, idClient, firstnameClient and lastnameClient, and then click OK to validate the schema and close the dialog box.

  4. In the Key definition area of the Basic settings view of tJoin, click the plus button to add two columns to the list and then select the input columns and the output columns you want to do the exact matching on from the Input key attribute and Lookup key attribute lists respectively, firstnameClient and lastnameClient in this example.

  5. Select the Inner join (with reject output) check box to define one of the outputs as inner join reject table.

  6. Double click tFileOutputExcel to display its Basic settings view and define its properties.

  7. Set the destination file name and the sheet name, and select the Include header check box.

  8. Double click tFileOutputDelimited to display its Basic settings view and define its properties.

  9. Set the destination file name, and select the Include header check box.

Saving and executing the Job

  1. Press Ctrl+S to save your Job.

  2. Press F6, or click Run on the Run tab to execute the Job.

    The output of the exact match on the firstnameClient and lastnameClient columns is written to the defined Excel file.

    The rejected data is written to the defined delimited file.