tWriteDynamicFields - 6.3

Talend Components Reference Guide

EnrichVersion
6.3
EnrichProdName
Talend Big Data
Talend Big Data Platform
Talend Data Fabric
Talend Data Integration
Talend Data Management Platform
Talend Data Services Platform
Talend ESB
Talend MDM Platform
Talend Open Studio for Big Data
Talend Open Studio for Data Integration
Talend Open Studio for Data Quality
Talend Open Studio for ESB
Talend Open Studio for MDM
Talend Real-Time Big Data Platform
task
Data Governance
Data Quality and Preparation
Design and Development
EnrichPlatform
Talend Studio

Function

tWriteDynamicFields creates an output dynamic column from input columns.

Purpose

tWriteDynamicFields allows you to create a dynamic schema from input columns in the component.

tWriteDynamicFields properties

Component family

Processing/Fields

 

Basic settings

Schema and Edit Schema

A schema is a row description, it defines the number of fields to be processed and passed on to the next component. The schema is either Built-in or stored remotely in the Repository.

Since version 5.6, both the Built-In mode and the Repository mode are available in any of the Talend solutions.

Click Edit schema to make changes to the schema. If the current schema is of the Repository type, three options are available:

  • View schema: choose this option to view the schema only.

  • Change to built-in property: choose this option to change the schema to Built-in for local changes.

  • Update repository connection: choose this option to change the schema stored in the repository and decide whether to propagate the changes to all the Jobs upon completion. If you just want to propagate the changes to the current Job, you can select No upon completion and choose this schema metadata again in the [Repository Content] window.

Click Sync columns to retrieve the schema from the previous component in the Job.

 

 

Built-in: You create the schema and store it locally for the relevant component. Related topic: see Talend Studio User Guide.

 

 

Repository: You have already created the schema and stored it in the Repository. You can reuse it in various projects and Job flowcharts. Related topic: see Talend Studio User Guide.

Advanced settings

tStatCatcher Statistics

Select this check box to gather the Job processing metadata at a Job level as well as at each component level.

Global Variables

ERROR_MESSAGE: the error message generated by the component when an error occurs. This is an After variable and it returns a string. This variable functions only if the Die on error check box is cleared, if the component has this check box.

NB_LINE: the number of rows read by an input component or transferred to an output component. This is an After variable and it returns an integer.

A Flow variable functions during the execution of a component while an After variable functions after the execution of the component.

To fill up a field or expression with a variable, press Ctrl + Space to access the variable list and choose the variable to use from it.

For further information about variables, see Talend Studio User Guide.

Usage

This component handles flow of data therefore it requires input and output components.

Log4j

If you are using a subscription-based version of the Studio, the activity of this component can be logged using the log4j feature. For more information on this feature, see Talend Studio User Guide.

For more information on the log4j logging levels, see the Apache documentation at http://logging.apache.org/log4j/1.2/apidocs/org/apache/log4j/Level.html.

Limitation

n/a

Scenario 1: Create a dynamic column and extract its content

This Java scenario contains five components. It aims at creating a Dynamic type column from a schema and extracting the columns contained in this dynamic column. These transformation results are displayed into the console.

Linking the components

  1. From the Palette to the design workspace, drag and drop the following components:

    - a tRowGenerator to generate a flow containing customer information,

    - a tWriteDynamicFields to replace columns with a Dynamic column,

    - two tLogRow to display the results into the console,

    - a tExtractDynamicFields to extract content from the dynamic column.

  2. Connect tRowGenerator to tWriteDynamicFields using a Row > Main link.

  3. Connect tWriteDynamicFields to the first tLogRow using a Row > Main link.

  4. Connect tLogRow to tExtractDynamicFields using a Row > Main link.

  5. Connect tExtractDynamicFields to the second tLogRow using a Row > Main link.

  6. Rename both tLogRow by double-clicking them.

    Type in their new name, dynamicColumn and standard column respectively.

Configuring the components

  1. In the design workspace, double-click the tRowGenerator to open its editor.

  2. Click seven times on the [+] button to add seven columns to the schema. Name them id, firstName, lastName, street, city, States and otherData, respectively.

  3. In the Type column, select Integer for id and String for all other fields.

    In the Functions columns, select the function you want to apply to each column in the schema.

    Click OK to validate the configuration and close the editor.

    A pop-up window opens and asks you to propagate the changes. Click Yes.

  4. In the design workspace, double-click tWriteDynamicFields component to display its Basic settings view.

    Click the [...] button next to Edit schema to set the schema of the component.

  5. In the schema editor, click on otherData column, then click to add the column on the output side.

    This way, the column will be displayed in the output, but it will not be included into the dynamic column.

  6. Click the [+] button in the output area to add a column.

    Name it columnDynamic and set its Type to Dynamic.

    Warning

    The dynamic column must be defined in the last row of the schema.

    Warning

    The column names must be exactly the same on both sides of the editor, otherwise the output columns will not be recognised.

  7. Click OK to validate your changes and close the editor.

    Warning

    The dynamic schema feature is only supported in Built-In mode.

    All columns that are not in the output area in the editor will be included into the dynamic column.

  8. In the design workspace, double-click the first tLogRow to display its Basic settings view.

    In the Mode area, select the Table (print value in cells of a table) option.

  9. In the design workspace, double-click tWriteDynamicFields component to display ots Basic settings view.

    Click the [...] button next to Edit schema.

  10. Select in the left area the columns you want to add to the tExtractDynamicFields schema and click the button to copy them.

    Click OK to close the editor.

  11. In the design workspace, double-click the tExtractDynamicFields component to display its Basic settings view.

    Click the [...] button next to Edit schema.

    In the schema editor, click the button in the output area to paste the columns copied from tWriteDynamicFields component.

    In the input area, click the Dynamic column and click the button to add the dynamic column to the output schema.

    Click OK to validate your changes and close the editor.

  12. In the design workspace, double-click the second tLogRow to display its Basic settings view.

  13. In the Mode area, select Table (print value in cells of a table) option.

Executing the Job

  1. Press Ctrl+S to save the Job.

  2. Press F6 to execute it.

    The first tLogRow component displays in the console the dynamic column in addition to the otherData column you have already defined in the output schema of the tWriteDynamicFields component.

    The second tLogRow displays in the console all the extracted columns (id, firstName and lastName) from the dynamic one, the otherData column that has not been included into the dynamic column and the dynamic column which still contains all columns voluntarily not extracted.

Scenario 2: Extracting the contents of a dynamic column via tJavaRow

In this scenario, a dynamic column is created via tWriteDynamicFields and its contents are then extracted by the customized code in tJavaRow.

Linking the components

  1. Drop tFixedFlowInput, tWriteDynamicFields, tLogRow and tJavaRow onto the workspace.

  2. Link tFixedFlowInput to tWriteDynamicFields using the Row > Main connection.

  3. Link tWriteDynamicFields to tLogRow using the Row > Main connection.

  4. Link tLogRow to tJavaRow using the Row > Main connection.