Mapping the Output setting

Talend Data Management Platform Studio User Guide

EnrichVersion
6.2
EnrichProdName
Talend Data Management Platform
task
Data Quality and Preparation
Design and Development
EnrichPlatform
Talend Studio

On the design workspace, the creation of a Row connection from the tMap component to the output components adds Output schema tables in the Map Editor.

You can also add an Output schema in your Map Editor, using the plus sign from the tool bar of the Output area.

You have as well the possibility to create a join between your output tables. The join on the tables enables you to process several flows separately and unite them in a single output. For more information about the output join tables feature, see Talend Components Reference Guide.

Note

The join table retrieves the schema of the source table.

When you click the [+] button to add an output schema or to make a join between your output tables, a dialog box opens. You have then two options.

Select...To...

New output

Add an independent table.

Create join table from

Create a join between output tables. In order to do so, select in the drop down list the table from which you want to create the join. In the Named field, type in the name of the table to be created.

Unlike the Input area, the order of output schema tables does not make such a difference, as there is no subordination relationship between outputs (of Join type).

Once all connections, hence output schema tables, are created, you can select and organize the output data via drag & drops.

You can drop one or several entries from the Input area straight to the relevant output table.

Press Ctrl or Shift, and click entries to carry out multiple selection.

Or you can drag expressions from the Var area and drop them to fill in the output schemas with the appropriate reusable data.

Note that if you make any change to the Input column in the Schema Editor, a dialog prompts you to decide to propagate the changes throughout all Input/Variable/Output table entries, where concerned.

Action

Result

Drag & Drop onto existing expressions.

Concatenates the selected expression with the existing expressions.

Drag & Drop to insertion line.

Inserts one or several new entries at start or end of table or between two existing lines.

Drag & Drop + Ctrl.

Replaces highlighted expression with selected expression.

Drag & Drop + Shift.

Adds the selected fields to all highlighted expressions. Inserts new lines if needed.

Drag & Drop + Ctrl + Shift.

Replaces all highlighted expressions with selected fields. Inserts new lines if needed.

You can add filters and rejections to customize your outputs.

Creating complex expressions

If you have complex expressions to create, or advanced changes to be carried out on the output flow, then the Expression Builder interface can help in this task.

Click the Expression field of your input or output table to display the [...] button. Then click this three-dot button to open the Expression Builder.

For more information regarding the Expression Builder, see How to write code using the Expression Builder.

Filters

Filters allow you to make a selection among the input fields, and send only the selected fields to various outputs.

Click the [+] button at the top of the table to add a filter line.

You can enter freely your filter statements using Java operators and functions.

Drop expressions from the Input area or from the Var area to the Filter row entry of the relevant Output table.

An orange link is then created. Add the required Java operator to finalize your filter formula.

You can create various filters on different lines. The AND operator is the logical conjunction of all stated filters.

Output rejection

Reject options define the nature of an output table.

It groups data which do not satisfy one or more filters defined in the standard output tables. Note that as standard output tables, are meant all non-reject tables.

This way, data rejected from other output tables, are gathered in one or more dedicated tables, allowing you to spot any error or unpredicted case.

The Reject principle concatenates all non Reject tables filters and defines them as an ELSE statement.

To define an output table as the Else part of the regular tables:

  1. Click the tMap settings button at the top of the output table to display the table properties.

  2. Click in the Value field corresponding to Catch output reject and then click the [...] button that appears to display the [Options] dialog box.

  3. In the [Options] dialog box, double-click true, or select it and click OK to validate the setting and close the dialog box.

You can define several Reject tables, to offer multiple refined outputs. To differentiate various Reject outputs, add filter lines, by clicking on the plus arrow button.

Once a table is defined as Reject, the verification process will be first enforced on regular tables before taking in consideration possible constraints of the Reject tables.

Note that data are not exclusively processed to one output. Although a data satisfied one constraint, hence is routed to the corresponding output, this data still gets checked against the other constraints and can be routed to other outputs.

Lookup Inner Join rejection

The Inner Join is a Lookup Join. The Inner Join Reject table is a particular type of Rejection output. It gathers rejected data from the main row table after an Inner Join could not be established.

To define an Output flow as container for rejected Inner Join data, create a new output component on your Job that you connect to the Map Editor. Then in the Map Editor, follow the steps below:

  1. Click the tMap settings button at the top of the output table to display the table properties.

  2. Click in the Value field corresponding to Catch lookup inner join reject and then click the [...] button that appears to display the [Options] dialog box.

  3. In the [Options] dialog box, double-click true, or select it and click OK to validate the setting and close the dialog box.

Removing Output entries

To remove Output entries, click the cross sign on the Schema Editor of the selected table.

Handling errors

The Die on error option prevents error to be processed. To do so, it stops the Job execution as soon as an error is encountered. The tMap component provides this option to prevent processing erroneous data. The Die on error option is activated by default in tMap.

Deactivating the Die on error option will allow you to skip the rows on error and complete the process for error-free rows on one hand, and to retrieve the rows on error and manage them if needed.

To deactivate the Die on error option:

  1. Double-click the tMap component on the design workspace to open the Map Editor.

  2. Click the Property Settings button at the top of the input area to display the [Property Settings] dialog box.

  3. In [Property Settings] dialog box, clear the Die on error check box and click OK.

A new table called ErrorReject appears in the output area of the Map Editor. This output table automatically comprises two columns: errorMessage and errorStackTrace, retrieving the message and stack trace of the error encountered during the Job execution. Errors can be unparseable dates, null pointer exceptions, conversion issues, etc.

You can also drag and drop columns from the input tables to this error reject output table. Those erroneous data can be retrieved with the corresponding error messages and thus be corrected afterward.

Once the error reject table is set, its corresponding flow can be sent to an output component.

To do so, on the design workspace, right-click the tMap component, select Row > ErrorReject in the menu, and click the corresponding output component, here tLogRow.

When you execute the Job, errors are retrieved by the ErrorReject flow.

The result contains the error message, its stack trace, and the two columns, id and date, dragged and dropped to the ErrorReject table, separated by a pipe "|".