Configuring the process of standardizing the unstructured data - 6.4

Standardization

author
Talend Documentation Team
EnrichVersion
6.4
EnrichProdName
Talend Big Data
Talend Big Data Platform
Talend Data Fabric
Talend Data Integration
Talend Data Management Platform
Talend Data Services Platform
Talend ESB
Talend MDM Platform
Talend Open Studio for Big Data
Talend Open Studio for Data Integration
Talend Open Studio for ESB
Talend Open Studio for MDM
Talend Real-Time Big Data Platform
task
Data Governance > Third-party systems > Data Quality components > Standardization components
Data Quality and Preparation > Third-party systems > Data Quality components > Standardization components
Design and Development > Third-party systems > Data Quality components > Standardization components
EnrichPlatform
Talend Studio

About this task

To do this, proceed as follows:

Procedure

  1. Connect the tFixedFlowInput component to the tStandardizeRow component with a Row > Main link.
  2. Double-click the tStandardizeRow component to display its Basic settings view.
    The advanced rule types are always executed after the ANTLR specific rules regardless of rule order.
  3. In the Column to parse list, select input.
  4. Select the Standardize this field check box.
  5. In the Conversion rules table, click six times on the [+] button to add six columns. Name them and configure them as shown in the capture. For "StreetName" rule:
    1. Select the Index type in the Type field.
    2. Enter the path to your index in the Value field. Before the path, enter the protocol: file:// or hdfs:// if the index is on a cluster.
    3. Select Match exact as the search mode from the Search mode list.
      By using the Match exact mode, you will extract from the input flow only the strings that exactly match the street name indexes. For further information about available search modes, see Search modes for Index rules
    Then the other rules are:

    Name

    Type

    Value

    "Zip"

    Format

    " DIGIT DIGIT DIGIT DIGIT DIGIT "

    "City" Enumeration

    " 'Paris' | 'Paris' | 'PARIS' | 'Bagnolet' "

    "SpecialStreetNumber" Format

    " (INT ('bis'|'ter')) | ( INT '-' INT) "

    "INT" Shape

    "<SpecialStreetNumber>"

    "Address" Shape

    " <INT> <StreetName> <Zip> <City> "

    As advanced rule types are always executed after the ANTLR specific rules, the basic rules "Zip", "City" and "SpecialStreetNumber" are executed first followed by "INT" and "Address", the advanced rules.

  6. Click the Generate parser code in Routines button in order to generate the code in the Routines.
    If you do not click on this button, the Job will not be executed.
  7. In the Advanced settings view, leave the options selected by default in the Output format area as they are.
    The Max edits for fuzzy match is set to 1 by default.
  8. In the design workspace, right-click the tStandardizeRow component to connect it to the tLogRow below with a Row > Reject link.
  9. Double-click the tLogRow component linked to the tStandardizeRow component to display its Basic settings view.
  10. In the Mode area, select the Table (print values in cells of a table) option.

Results

Then continue to configure the process of filtering and extracting the data of interest.