tMapRStreamsOutput Standard properties

Schema and Edit schema

A schema is a row description. It defines the number of fields (columns) to be processed and passed on to the next component. When you create a Spark Job, avoid the reserved word line when naming the fields.

Note that the schema of this component is read-only. It stores the messages to be published.

Use an existing connection

Select this check box and from the list displayed select the relevant connection component to reuse the connection details you have already defined.

Distribution and Version

Select the MapR distribution to be used. Only MapR V5.2 onwards is supported by the MapRDB components.

If the distribution you need to use with your MapRDB database is not officially supported by this MapRBD component, that is to say, this distribution is MapR but is not listed in the Version drop-down list of this components or this distribution is not MapR at all, select Custom.

1. Select Import from existing version to import an officially supported distribution as base and then add other required jar files which the base distribution does not provide.

2. Select Import from zip to import the configuration zip for the custom distribution to be used. This zip file should contain the libraries of the different Hadoop elements and the index file of these libraries.

   Note that custom versions are not officially supported by Talend . Talend and its community provide you with the opportunity to connect to custom versions from the Studio but cannot guarantee that the configuration of whichever version you choose will be easy, due to the wide range of different Hadoop distributions and versions that are available. As such, you should only attempt to set up such a connection if you have sufficient Hadoop experience to handle any issues on your own.

   Information noteNote:

   In this dialog box, the active check box must be kept selected so as to import the jar files pertinent to the connection to be created between the custom distribution and this component.

   For a step-by-step example about how to connect to a custom distribution and share this connection, see Hortonworks.

Enter the name of the topic you want to publish messages to. This topic must already exist. You must enter the name of the stream to which this topic belongs. The syntax is path_to_the_stream:topic_name.

Compress the data

Select the Compress the data check box to compress the output data.

Producer properties

Add the MapR Streams producer properties you need to customize to this table.

For further information about the producer configuration you can define in this table, see the section describing the important producer configuration properties for MapR Streams in MapR documentation at MapR Streams Overview.

tStatCatcher Statistics

Select this check box to gather the processing metadata at the Job level as well as at each component level.

Global Variables

ERROR_MESSAGE: the error message generated by the component when an error occurs. This is an After variable and it returns a string. This variable functions only if the Die on error check box is cleared, if the component has this check box.

A Flow variable functions during the execution of a component while an After variable functions after the execution of the component.

To fill up a field or expression with a variable, press Ctrl + Space to access the variable list and choose the variable to use from it.

For further information about variables, see Talend Studio User Guide.

Usage rule

This component is an end component. It requires a tJavaRow or tJava component to transform the incoming data into serialized byte arrays.

The following sample shows how to construct a statement to perform this transformation:

output_row.serializedValue = input_row.users.getBytes();

In this code, the output_row variable represents the schema of the data to be output to tMapRStreamsOutput and output_row.serializedValue the single read-only column of that schema; the input_row variable represents the schema of the incoming data and input_row.users the input column called users to be transformed to byte arrays by the getBytes() method.

Prerequisites

The Hadoop distribution must be properly installed, so as to guarantee the interaction with Talend Studio . The following list presents MapR related information for example.

• Ensure that you have installed the MapR client in the machine where the Studio is, and added the MapR client library to the PATH variable of that machine. According to MapR's documentation, the library or libraries of a MapR client corresponding to each OS version can be found under MAPR_INSTALL\ hadoop\hadoop-VERSION\lib\native. For example, the library for Windows is \lib\native\MapRClient.dll in the MapR client jar file. For further information, see the following link from MapR: http://www.mapr.com/blog/basic-notes-on-configuring-eclipse-as-a-hadoop-development-environment-for-mapr.

  Without adding the specified library or libraries, you may encounter the following error: no MapRClient in java.library.path.

• Set the -Djava.library.path argument, for example, in the Job Run VM arguments area of the Run/Debug view in the Preferences dialog box in the Window menu. This argument provides to the Studio the path to the native library of that MapR client. This allows the subscription-based users to make full use of the Data viewer to view locally in the Studio the data stored in MapR.

For further information about how to install a Hadoop distribution, see the manuals corresponding to the Hadoop distribution you are using.