tReplaceList - 6.3

Talend Open Studio for Big Data Components Reference Guide

EnrichVersion
6.3
EnrichProdName
Talend Open Studio for Big Data
task
Data Governance
Data Quality and Preparation
Design and Development
EnrichPlatform
Talend Studio

Function

Carries out a Search and Replace operation in the input columns defined based on an external lookup.

Purpose

Helps to cleanse all files before further processing.

tReplaceList Properties

Component family

Data Quality

 

Basic settings

Schema and Edit schema

A schema is a row description, it defines the number of fields to be processed and passed on to the next component. The schema is either Built-in or stored remotely in the Repository.

Since version 5.6, both the Built-In mode and the Repository mode are available in any of the Talend solutions.

Two read-only columns, Value and Match are added to the output schema automatically.

Warning

The data Type defined in the schemas must be consistent, ie., an integer can only be replaced by another integer using an integer as a look up field. Values of one type cannot be replaced by values of another type.

 

 

Built-in: The schema will be created and stored locally for this component only. Related topic: see Talend Studio User Guide.

 

 

Repository: The schema already exists and is stored in the Repository, hence can be reused in various projects and job designs. Related topic: see Talend Studio User Guide.

 

Lookup search column

Select the column to be searched in the lookup schema.

 

Lookup replacement column

Select the column where the replacement values are stored.

 

Column options

Select the columns of the main flow where the replacement is to be carried out.

Advanced settings

tStat Catcher Statistics

Select this check box to collect log data at the component level.

Global Variables

ERROR_MESSAGE: the error message generated by the component when an error occurs. This is an After variable and it returns a string. This variable functions only if the Die on error check box is cleared, if the component has this check box.

A Flow variable functions during the execution of a component while an After variable functions after the execution of the component.

To fill up a field or expression with a variable, press Ctrl + Space to access the variable list and choose the variable to use from it.

For further information about variables, see Talend Studio User Guide.

Usage

tReplaceList is an intermediary component. It requires an input flow and an output component.

Scenario: Replacement from a reference file

The following Job searches and replaces a list of states with their corresponding two-letter codes. The relevant codes are taken from a reference file placed as lookup flow in the Job.

Setting up the Job

  1. Drop the following components from the Palette onto the design workspace: two tFileInputDelimited components, a tReplaceList and a tLogRow.

  2. Connect the two tFileInputDelimited components to the tReplaceList component using Row > Main connections. Note that the link between the reference input component (the second tFileInputDelimited) and the tReplaceList component appears as a lookup row.

  3. Connect the tReplaceList component to the tLogRow component using a Row > Main connection.

Configuring the components

  1. Double-click the first tFileInputDelimited component to open its Basic settings view and set the parameters of the main input flow, including the path and name of the file to read and the number of header rows to skip.

    In this example, the main input file provides a list of people names and US state names. The following shows an extract of the file content:

    name;state
    Andrew Kennedy;Mississippi
    Benjamin Carter;Louisiana
    Benjamin Monroe;West Virginia
    Bill Harrison;Tennessee
    Calvin Grant;Virginia
    Chester Harrison;Rhode Island
    Chester Hoover;Kansas
    Chester Kennedy;Maryland
    Chester Polk;Indiana
    Dwight Nixon;Nevada
    Dwight Roosevelt;Mississippi
    Franklin Grant;Nebraska
  2. Click the [...] button next to Edit schema to open the [Schema] dialog box and set the input schema.

    According to the structure of the main input file, the input schema should contain two columns: name and state.

    When done, click OK to close the dialog box and propagate the changes to the next component.

  3. Define the properties of the second tFileInputDelimited component similarly.

    In this example, the reference input file provides a list of states and their two-letter codes. Accordingly, the reference input schema should have two columns: state and code.

  4. Double-click the tReplaceList component to open its Basic settings view to set the operation to carry out.

  5. From the Lookup search column list, select the column to be searched. In this use case, we want to carry out a search on the state column.

  6. From the Lookup replacement column list, select the column containing the replacement values, code for the two-letter state codes in this example.

  7. In the Column options table, select Replace check box for the states column, to replace the state names with their corresponding codes.

  8. In the tLogRow component, select the Table check box for a better readability of the output.

Executing the Job

  • Save the Job and press F6 to execute it.

    The state names have been replaced with their respective two-letter codes.