tMomInput - 6.1

Talend Components Reference Guide

EnrichVersion
6.1
EnrichProdName
Talend Big Data
Talend Big Data Platform
Talend Data Fabric
Talend Data Integration
Talend Data Management Platform
Talend Data Services Platform
Talend ESB
Talend MDM Platform
Talend Open Studio for Big Data
Talend Open Studio for Data Integration
Talend Open Studio for Data Quality
Talend Open Studio for ESB
Talend Open Studio for MDM
Talend Real-Time Big Data Platform
task
Data Governance
Data Quality and Preparation
Design and Development
EnrichPlatform
Talend Studio

tMomInput Properties

Component family

Internet

Function

This component fetches a message from a queue on a Message-Oriented Middleware (MOM) system and passes it on to the next component.

Purpose

tMomInput makes it possible to set up asynchronous communications via a MOM server.

Basic settings

Use existing connection

Select this check box and in the Component List click the relevant connection component to reuse the connection details you already defined.

Note

When a Job contains the parent Job and the child Job, Component List presents only the connection components in the same Job level.

 

Keep Listening

Select this check box to keep the MOM server listening for and fetching new messages.

This check box is not visible when ActiveMQ is selected from the MQ server list and the receive number of messages check box is selected.

 

Sleeping time (in sec)

Specify the timeout interval (in seconds) within which the next message must be received.

This field is available in any one of the following situations:

  • JBoss Messaging is selected from the MQ server list and the Keep Listening check box is selected.

  • ActiveMQ is selected from the MQ server list and the Keep Listening check box is cleared.

  • ActiveMQ is selected from the MQ server list and the receive number of messages and Timeout for receiver check boxes are selected.

 

receive number of messages

Select this check box to set the maximum number of messages that can be received.

This check box is available only when ActiveMQ is selected from the MQ server list and the Keep Listening check box is cleared.

 

maximum messages

Enter the maximum number of messages you can receive.

This field is available only when the receive number of messages check box is selected.

 

Timeout for receiver

Select this check box and specify the timeout interval for the receiver in the Sleeping time (in sec) field.

This check box is available only when the receive number of messages check box is selected.

 

Failover

Select this check box to use the ActiveMQ failover transport. For more information about the failover transport, see Failover Transport Reference.

This check box is available only when ActiveMQ is selected from the MQ server list and the Static Discovery check box is cleared.

 

Static Discovery

Select this check box to use the ActiveMQ static transport discovery mechanism. For more information about the static transport, see Static Transport Reference.

This check box is available only when ActiveMQ is selected from the MQ server list and the Failover check box is cleared.

 

URI parameters

Enter the URI parameters.

This field is available only when the Failover or Static Discovery check box is selected.

 

MQ server

Select the type of the MQ server(s) to be connected from the list: ActiveMQ, JBoss Messaging, or WebSphere MQ. According to the server type selected, the parameters required differ slightly.

 

Use SSL Transport

Select this check box to use the ActiveMQ SSL transport. For more information about the SSL transport, see SSL Transport Reference.

This check box is available only when ActiveMQ is selected from the MQ server list.

 

Host

Enter the host name or IP address of the MQ server.

 

Port

Enter the listening port of the MQ server.

 

Username and Password

Enter the user authentication credentials.

To enter the password, click the [...] button next to the password field, and then in the pop-up dialog box enter the password between double quotes and click OK to save the settings.

 

Channel

Specify the name of the channel through which the connection is established. The default value is DC.SVRCONN.

This field is available only when WebSphere MQ is selected from the MQ server list.

 

Message From

Type in the message source, exactly as expected by the server; this must include the type and name of the source. e.g.: queue/A or topic/testtopic

Note that the field is case-sensitive.

This field is available only when ActiveMQ or JBoss Messaging is selected from the MQ Server list.

 

Message Type

Select the message type, either Topic or Queue.

This field is available only when ActiveMQ or JBoss Messaging is selected from the MQ Server list.

 

Message Body Type

Select the message body type: Text Message, Bytes Message or Map Message.

 

Schema and Edit Schema

A schema is a row description, it defines the number of fields that will be processed and passed on to the next component.

In the context of tMomInput usage, the schema is comprised of two columns: From and Message, and the column names are read only.

 

QueueManager

Specify the name of the queue manager to which the connection is established.

This field is available only when WebSphere MQ is selected from the MQ server list.

 

MessageQueue

Enter the name of the message queue into which message queueing applications can put messages.

This field is available only when WebSphere MQ is selected from the MQ server list.

 

Is using message id to fetch

Select this check box to fetch messages according to their IDs.

This check box is available only when Text Message or Bytes Message is selected from the Message Body Type list.

 

Commit (delete message after read from the queue)

Select this check box to force a commit after reading each message from the queue.

This check box is available only when WebSphere MQ is selected from the MQ server list and the Browse message check box in the Advanced settings view is cleared.

 

Backout messages to input queue

Select this check box to put the message back into the queue after reading it.

This check box and the Browse message check box in the Advanced settings view enable you to read messages non-destructively from the queue. For further information, see https://publib.boulder.ibm.com/iseries/v5r2/ic2924/books/csqzaw07.pdf.

This check box is available only when WebSphere MQ is selected from the MQ server list and the Keep Listening, Commit (delete message after read from the queue) and Browse message check boxes are cleared.

 

Backout messages to backout queue (when backoutCount of message reach the threshold value)

Select this check box to backout the messages to the backout queue defined in the Websphere MQ manager when the backout count reaches the configured threshold which is a property of the queue in the Websphere MQ server.

This check box is available only when WebSphere MQ is selected from the MQ server list and the Keep Listening, Commit (delete message after read from the queue) and Browse message check boxes are cleared.

 

Set Transacted

Select this check box to transact the session. For further information about this parameter, see https://publib.boulder.ibm.com/iseries/v5r2/ic2924/books/csqzaw07.pdf.

Note

Selecting this check box will ignore the settings in the Acknowledgement Mode list in the Advanced settings view of tMomInput.

This check box is available only when ActiveMQ is selected from the MQ server list.

 

start server

Select this check box to force a start of the ActiveMQ server.

This check box is available only when ActiveMQ is selected from the MQ server list.

Advanced settings

Additional options

Define the additional transport options in the table. This table is available only if you select the Failover or Static Discovery check box in the Basic settings view.

  • Option name: the name of the option.

  • Option value: the value of the option.

For example, enter maxReconnectAttempts in the Option name field and then 0 in the Option value field so that no attempt is made to retry the connection when no server is available for the failover mode.

For more information about the failover transport options, see Failover Transport Reference.

For more information about the static discovery transport options, see Static Transport Reference.

 

Acknowledgement Mode

Select an acknowledgement mode from the list to indicate that the client will acknowledge any messages it receives:

Auto Acknowledge: With this acknowledgement mode, the client automatically acknowledges a message when it has either successfully returned from a call to receive,or the message listener it has called to process the message successfully returns.

Client Acknowledge: With this acknowledgement mode, the client acknowledges a message by calling a message's acknowledge method.

Dups OK Acknowledge: This acknowledgement mode instructs the session to lazily acknowledge the delivery of messages.

For further information about the usage of Jms headers, see https://publib.boulder.ibm.com/iseries/v5r2/ic2924/books/csqzaw07.pdf.

Note

If the Set Transacted check box of tMomOutput is selected, Acknowledgement Mode will be ignored. This check box is enabled when the MQ server is ActiveMQ or JBoss Messaging.

 

Get Jms Header

Select this check box to receive the Jms headers through the mapping from Jms fields onto MQ Series fields. When this check box is checked, you can specify the Jms header and the corresponding reference column name in the line(s) you added by clicking the plus button in the Parameters table. For further information about the usage of Jms headers, see https://publib.boulder.ibm.com/iseries/v5r2/ic2924/books/csqzaw07.pdf.

Note

This check box is enabled when the MQ server is ActiveMQ or JBoss Messaging.

 

Get Jms Properties

Select this check box to receive the Jms properties mapped to MQMD fields. When this check box is checked, you can specify the property name, the property type and the reference column name in the line(s) you added by clicking the plus button in the Parameters table. For further information about the usage of Jms properties, see https://publib.boulder.ibm.com/iseries/v5r2/ic2924/books/csqzaw07.pdf.

(Available when ActiveMQ or JBoss Messaging is selected in the MQ Server list.)

 

Browse message

Select this check box to disable Commit (delete message after read from the queue) check box and Backout messages to input queue check box in the Basic settings view and open the queue to browse messages.

Note

This check box and the Backout messages to input queue check box in the Basic settings view enable you to read messages non-destructively from the queue. Browse message check box is visible only when the MQ server is WebSphere MQ with the Backout messages to input queue check box cleared. For further information, see http://publib.boulder.ibm.com/infocenter/wmqv7/v7r0m0/index.jsp?topic=%2Fcom.ibm.mq.java.doc%2Fcom%2Fibm%2Fmq%2FMQC.html/.

 

Get MQMD Fields

Select this check box to set one or more message descriptors by adding new fields for MQMD(message queuing message descriptor) in the Parameters table:

Field Name: Select one or more message descriptors from the list to retrieve header information of the message.

Reference Column Name: The header and properties information of the message.

For further information, see http://publib.boulder.ibm.com/infocenter/wmqv7/v7r0m0/index.jsp?topic=%2Fcom.ibm.mq.csqzak.doc%2Ffr13040_.htm/.

(Available when WebSphere MQ is selected in the MQ Server list.)

 

Include Header

Select this check box to specify the header of the message:

MQRFH2 fixed Portion: Select this check box and click the plus button to add one or more lines to specify the Version, Encoding, Format, Flags, NameValueCCSID and CodedCharSetId as needed for the fixed portion of MQRFH2 header.

MCD folder: Select this check box and click the plus button to add one or more lines to specify the fields and the reference column names for the properties that describe the format of the message.

JMS folder: Select this check box and click the plus button to add one or more lines to specify the fields and the reference column names for the transportation of JMS header fields and JMSX properties.

USR folder: Select this check box and click the plus button to add one or more lines to specify the fields and the reference column names for the transportation of application-defined properties associated with the message.

For further information about MQRFH2 header, see http://publib.boulder.ibm.com/infocenter/wmbhelp/v6r1m0/index.jsp?topic=%2Fcom.ibm.etools.mft.doc%2Faq06920_.htm.

(Available when WebSphere MQ is selected in the MQ Server list.)

 

Set CipherSpec

Select this check box to enable the CipherSpec list from which you can specify the CipherSpec to be used with WebSphere MQ SSL.

For further information about CipherSpec, see http://publib.boulder.ibm.com/infocenter/wmqv6/v6r0/index.jsp?topic=%2Fcom.ibm.mq.csqzas.doc%2Fsy12870_.htm.

(Available when WebSphere MQ is selected in the MQ Server list.)

 

tStatCatcher Statistics

Select this check box to gather the Job processing metadata at a Job level as well as at each component level.

Dynamic settings

Click the [+] button to add a row in the table and fill the Code field with a context variable to choose your MQ Server connection dynamically from multiple connections planned in your Job.

The Dynamic settings table is available only when the Use an existing connection check box is selected in the Basic settings view. Once a dynamic parameter is defined, the Component List box in the Basic settings view becomes unusable.

For examples on using dynamic parameters, see Scenario 3: Reading data from MySQL databases through context-based dynamic connections and Scenario: Reading data from different MySQL databases using dynamically loaded connection parameters. For more information on Dynamic settings and context variables, see Talend Studio User Guide.

Global Variables

ERROR_MESSAGE: the error message generated by the component when an error occurs. This is an After variable and it returns a string. This variable functions only if the Die on error check box is cleared, if the component has this check box.

NB_LINE: the number of rows read by an input component or transferred to an output component. This is an After variable and it returns an integer.

A Flow variable functions during the execution of a component while an After variable functions after the execution of the component.

To fill up a field or expression with a variable, press Ctrl + Space to access the variable list and choose the variable to use from it.

For further information about variables, see Talend Studio User Guide.

Usage

This component is generally used as a start component. It must be linked to an output component.

Limitation

Make sure the relevant ActiveMQ, JBoss Messaging or Websphere MQ server is launched.

This component requires installation of its related jar files. For more information about the installation of these missing jar files, see the section describing how to configure the Studio of the Talend Installation Guide.

Scenario 1: Asynchronous communication via a MOM server

This scenario is made of two Jobs. The first Job posts messages on a JBoss server queue and the second Job fetches the message from the server.

Configuring and executing the first Job

In the first Job, a string message is created using a tRowGenerator and put on a JBoss server using a tMomOutput. An intermediary tLogRow component displays the flow being passed.

  1. Drop the three components required for the first Job from the Palette onto the design workspace and right-click to connect them using a Main row link.

  2. Double-click on tRowGenerator to set the schema to be randomly generated.

  3. Set just one column called message. This is the message to be put on the MOM queue.

  4. This column is of String type and is nullable. To produce the data, use a preset function which concatenates randomly chosen ascii characters to form a 6-char string. This function is getAsciiRandomString. (Java version). Click the Preview button to view a random sample of data generated.

  5. Set the Number of rows to be generated to 10.

    Click OK to validate.

  6. The tLogRow is only used to display a intermediary state of the data to be handled. In this example, it doesn't require any specific configuration.

  7. Then select the tMomOutput component.

  8. In this case, the MQ server to be used is JBoss.

    In the Host and Port fields, fill in the relevant connection information.

  9. Select the Message type from the list. The message can be of Queue or Topic type. In this example, select the Queue type from the list.

  10. In the To field, type in the message source information strictly respecting the syntax expected by the server. This should match the Message Type you selected, such as: queue/A.

    Note

    The message name is case-sensitive, therefore queue/A and Queue/A are different.

  11. Then click Sync Columns to pass on the schema from the preceding component. The schema being read-only, it cannot be changed. The data posted onto the MQ comes from the first schema column encountered.

  12. Press F6 to execute the Job and view the data flow being passed on in the console, thanks to the tLogRow component.

Configuring and executing the second Job

Then set the second Job in order to fetch the queuing messages from the MOM server.

  1. Drop the tMomInput component and a tLogRow from the Palette to the design workspace.

  2. Select the tMomInput to set the parameters.

  3. Select the MQ server from the list. In this example, a JBoss messaging server is used.

  4. Set the server Host and Port information.

  5. Set the Message From and the Message Type to match the source and type expected by the messaging server.

  6. The Schema is read-only and is made of two columns: From and Message.

  7. Select the Keep listening check box and set the verification frequency to 5 seconds.

    Note

    When using the Keep Listening option, you'll need to kill the Job to end it.

  8. No need to change any default setting from the tLogRow.

  9. Save the Job and run it (when launching for the first time or if you killed it on a previous run).

    The messages fetched on the server are displayed on the console.

Scenario 2: Transmitting XML files via a MOM server

This scenario describes a five-component Job composed of two subjobs that sends XML files from a local folder to a MOM queue, and then fetches the files from the MOM queue and displays the contents of the files on the console.

Dropping and links the components

  1. From the Palette, drop the following components one after another onto the design workspace: tFileList, tFileInputXML, tMomOutput, tMomInput, and tLogRow.

  2. Connect tFileList to tFileInputXML using a Row > Iterate link, and connect tFileInputXML to tMomOutput using a Row > Main link to form the first subjob. This subjob will read each XML file in a given folder and send it to a MOM queue.

  3. Connect tMomInput to tLogRow using a Row > Main link to form the second subjob. This subjob will fetch the XML files from MOM queue and display the file contents on the console.

  4. Connect tFileInputXML to tMomInput using a Trigger > On Component Ok connection to link the two subjobs.

Configuring the first subjob

Configuring the input components
  1. Double-click the tFileList component to open its Basic settings view.

  2. In the Directory field, enter the path to the directory to read XML files from, or browse to the path by clicking the [...] button next to the field.

  3. Select Use Glob Expressions as Filemask check box, add a new line in the Files field by clicking the [+] button, and enter "*.xml" as the file mask so that all XML files in the directory will be used. Keep all the other settings as they are.

  4. Double-click the tFileInputXML component to open its Basic settings view.

  5. Click the [...] button next to Edit schema to open the [Schema] dialog box.

  6. Click the [+] button to add a column, give it a name, message in this example, and select Document from the Type list to handle XML format files. Then, click OK to close the dialog box.

  7. In the File name/Stream field, press Ctrl+Space to access the global variable list, and select tFileList_1.CURRENT_FILEPATH to loop on the context files' directory.

  8. In in the Loop XPath query fields, enter "/" to define the root as the loop node of the input files' structure; in the Mapping table, fill the XPath query column with "." to extract all data from context node of the source files, and select the Get Nodes check box to build a Document type data flow.

Configuring the tMomOutput component
  1. Double-click the tMomOutput component to open its Basic settings view.

  2. Select WebSphere MQ from the MQ server list, and enter the host name or IP address of the MQ server and the port number.

  3. Enter the login authentication information in the Username and Password fields, and enter the channel name of the transmission queue in the Channel field.

  4. As we are handling file messages, select Text Message from the Message Body Type list.

  5. Click Sync columns to retrieve the schema structure from the preceding component.

  6. Fill in the queue manager and message queue details in the corresponding fields, and leave the other settings as they are.

Configuring the second subjob

  1. Double-click the tMomInput component to open its Basic settings view.

  2. Set the basic parameters of the component using the same settings you have done in the tMomOutput component, including the MQ server details, login authentication details, channel, message body type, queue manager and message queue.

  3. Click the [...] button next to Edit schema to open the [Schema] dialog box.

  4. From the Type list for the message column, select Document to handle XML format files, and then click OK to close the dialog box.

Saving and executing the Job

  1. Press Ctrl+S to save your Job.

  2. Press F6 or click Run on the Run tab to execute the Job.

    The XML files in the specified folder are written to the message queue and then retrieved from the queue. The contents of the files are displayed on the console.