tHMap Standard properties - 7.2

Data mapping

EnrichVersion
7.2
EnrichProdName
Talend Big Data Platform
Talend Data Fabric
Talend Data Management Platform
Talend Data Services Platform
Talend MDM Platform
Talend Real-Time Big Data Platform
EnrichPlatform
Talend Studio
task
Data Governance > Third-party systems > Processing components (Integration) > Data mapping
Data Quality and Preparation > Third-party systems > Processing components (Integration) > Data mapping
Design and Development > Third-party systems > Processing components (Integration) > Data mapping

These properties are used to configure tHMap running in the Standard Job framework.

The Standard tHMap component belongs to the Processing family.

This component is available in Talend Data Management Platform, Talend Big Data Platform, Talend Real Time Big Data Platform, Talend Data Services Platform, Talend MDM Platform and Talend Data Fabric.

Basic settings

Open Map Editor

Click the [...] button to open the tHMap Structure Generate/Select wizard where you can either have the hierarchical mapper structure generated automatically based on the schema, or select an existing hierarchical mapper structure. You must do this for both the input and output sides of your Map. The following lists the options for the output structure:

  • Generate hierarchial mapper structure based on the schema option: When you connect multiple output connections to the tHMap, the page displays a confirmation message that informs you that the mapper structures are generated based on the output connections.
  • Select an existing hierarchical mapper structure option: You can connect multiple outputs that are payload-based connections to the tHMap. If there is a single payload-type connection, you can select the Allow support for multiple output connections checkbox. The generated output map inherits from the existing payload structure.

If Talend Studio detects multiple output connections available, the window displays both output structure options without the support for multiple output connections checkboxes.

If neither input nor output connection exists, the Structure Selection page is displayed.

Map Path

Specifies the map to be executed.

If the map was automatically created using the wizard described above, this is path is set automatically.

If you want to use an existing map, click the [...] button next to the Map Path field to open a dialog box in which you can select the map you want to use, then click the [...] button next to Open Map Editor to work with the map selected. Note that this map must have previously been created in the Mapping perspective.

Read Input As

Select the radio button which corresponds to how you want the input to be read. Depending on your map, only some of the options may be available.

  • Data Integration columns (default): Use this option if you are working with Talend Data Integration metadata.

  • Single column: Use this option if you are working with Talend Data Mapper metadata.

Write Output As

Select the radio button which corresponds to how you want the output to be written. Depending on your map, only some of the options may be available.

  • Data Integration columns (default): Use this option if you are working with Talend Data Integration metadata.

  • String (single column): Use this option if the data in the output column is to be a String.

  • Byte array (single column): Use this option if the data in the output column is to be a Byte array.

  • InputStream (single column): Use this option if you are working with Talend Data Mapper metadata and the input data is a stream.

  • Document (single column); Use this option if the output column is to be a Document.

    Note: If you have selected Document (single column) in the Job you created in versions 6.5 or below of Talend Studio and would like to import the same Job to the latest versions 7.0 or onwards, ensure that you have updated the output representation of the map to XML prior to executing the Job. Versions 7.0 or onwards only accept the Document output type with XML representation.

Advanced settings

Map Variable

In this field, enter a context variable that contains a path to a map. For instance, if you enter context.mymapfile, then mymapfile can point to different map files at runtime. This can be useful in cases where you want to use multiple maps without creating a new Job each time.

In the Contexts tab, the value must be an relative path. For instance, assuming you have a map called mapA in the folder Maps/FolderA, your context variable should contain the value "FolderA/mapA.xml". You can write the path to the file with or without the .xml extension.

Note that all maps that might be referenced by the context variable must be present in the same Project. This way, when the job is built, it will contain all candidate maps and it will be possible to switch from one to another at runtime.

For further information on working with context variables, see Talend Studio User Guide.

Alternate contextual Map(s)

From the drop-down list, select a map or several maps that can be referenced by the context variable added in the Map Variable at runtime. When the Job is built, it contains the selected alternate contextual map or maps. For an example scenario on using Alternate contextual Map(s), see Specifying a map variable and alternate map in tHMap.

Map each row (disable virtual component)

Select this check box to have tHMap process the input as a single output row. This prevents tHMap from buffering the input rows before delivering them downstream.

This can be useful, for example, when you use the tHMap component with tSAPIDocReceiver as the input component and any schema-aware component as the output component, because telling the tSAPIDocReceiver component to keep listening forever would otherwise lead to rows never being delivered.

If the tHMap is used with an input component that can remain open to continuously receive inputs, such as the tRestRequest, this option should be selected. It allows the tHMap to directly process the input received.

Log Level

From the drop-down list, select how often you want events to be logged.

  • Infrequent: Logs only events related to startup, shutdown and exceptions.

  • Frequent (default): Logs events related to startup, shutdown and exceptions, and once per map execution.

  • Info: Logs all events at an informational level or higher.

  • All: Logs all events.

  • None: Logs nothing.

Exception Threshold

Talend Data Mapper returns an execution status with an severity value which can be OK, Info, Warning, Error or Fatal. By setting the exception threshold, you can specify the severity level at which an exception is thrown, thus enabling downstream components to detect the error in cases other than the default value of Fatal.

From the drop-down list, select the severity level at which an exception may be thrown during the execution of a map.

  • Fatal (default): An exception is thrown when a fatal error occurs.

  • Error: An exception is thrown when an error (or higher) occurs.

  • Warning: An exception is thrown when a warning (or higher) occurs.

Note that, in order to help you diagnose problems with your map, when you test the map in the Studio, any errors that occur which are at warning level or above will be printed in the console window, regardless of the setting of the Exception Threshold.

tStatCatcher Statistics

Select this check box to gather the Job processing metadata at the Job level as well as at each component level.

Global Variables

Global Variables

ERROR_MESSAGE: the error message generated by the component when an error occurs. This is an After variable and it returns a string. This variable functions only if the Die on error check box is cleared, if the component has this check box.

MapExecutionStatus: returns an object of the org.talend.transform.runtime.common.MapExecutionStatus class

EXECUTION_SEVERITY: the Overall Severity numeric value. This is an After variable and it returns an integer.

A Flow variable functions during the execution of a component while an After variable functions after the execution of the component.

To fill up a field or expression with a variable, press Ctrl + Space to access the variable list and choose the variable to use from it.

For further information about variables, see Talend Studio User Guide.

Usage

Usage rule

tHMap is used for Jobs that require complex data mapping from a variety of different sources.

The input and output connections can use Talend Data Mapper metadata, Talend Data Integration metadata, or a combination of the two. Each connection is independent.

When you open the Map Editor for the first time for each connection, it either generates a Talend Data Mapper structure definition based on the schema of the Talend Data Integration component, or allows you to select an existing Talend Data Mapper structure if you are using Talend Data Mapper metadata. It then creates a map with the structure selected or generated.

This component can be used in several ways:

Advanced use case
  • For the output Structure of the thMap, you can also use the WriteURL function in Talend Data Mapper . In case the output Structure of the tHMap includes both a connection to a Talend Data Integration metadata or component and a WriteURL function, the WriteURL value takes precedence. Talend Data Integration metadata or component does not receive any data from tHMap.
  • Using tHMap, you can test the generated Talend Data Mapper Map in Test Run mode.

    To use the feature, ensure that you have a compatible input document of CSV format with these properties:
    • the default delimiter is the (,) comma;
    • the system default line separator is used as the newline;
    • there exists no header row in the input file;
    • there will be no header row in the output file.

    Specify your input document in Show Document > Select Sample Input Document > Select Resource/Import Document.

    To execute the transformation, click Test Run. After execution, your generated output document is in CSV format, which you can save by clicking Save to File. To configure this feature, see Test preferences in the Talend Data Mapper User Guide.

    Tip: If you have multiple inputs and outputs in your Map, specify an input document for each row or each ReadURL map element. To execute, use the Test Run to External Output option and specify a file URL for each WriteURL in the output structure of the Map.
Limitations
  • If you created a tHMap with multiple input or output connections, you can only add new schema-based input or output connections.
  • When you add a new connection with a single column, you cannot associate it with a payload structure. The Map Regeneration mechanism only generates a single-element record for the new connection.
  • If you rename a payload-type connection, regenerate the corresponding map and its structure by double-clicking on the tHMap before closing the Job editor. The renamed record structure then contains the payload structure.
  • If in the event that you closed the Job prior to regenerating a renamed payload-type connection, the system treats the connection as a schema-type connection. The payload structure does not get associated with the new name.
Note:
  • For further information about performing transformations using Talend Data Mapper , see Talend Data Mapper User Guide.

  • For tHMap using representations other than Hashmap, CSV, Avro, XML, and Cobol: Running tHMap may create temporary folders, which are not automatically removed after completing the Eclipse runtime. If you frequently run Talend Data Mapper, these files may pile up and eventually lead to a disk full error. When the error occurs, the runtime fails. To avoid such scenario, ensure that you perform periodical cleaning.