tESBConsumer - 6.1

Talend Components Reference Guide

EnrichVersion
6.1
EnrichProdName
Talend Big Data
Talend Big Data Platform
Talend Data Fabric
Talend Data Integration
Talend Data Management Platform
Talend Data Services Platform
Talend ESB
Talend MDM Platform
Talend Open Studio for Big Data
Talend Open Studio for Data Integration
Talend Open Studio for Data Quality
Talend Open Studio for ESB
Talend Open Studio for MDM
Talend Real-Time Big Data Platform
task
Data Governance
Data Quality and Preparation
Design and Development
EnrichPlatform
Talend Studio

tESBConsumer properties

Component family

ESB/Web Services

 

Function

Calls the defined method from the invoked Web service and returns the class as defined, based on the given parameters.

Purpose

Invokes a Method through a Web service.

 Basic settings

Service configuration

Description of Web service bindings and configuration. The Endpoint field gets filled in automatically upon completion of the service configuration.

 

Input Schema and Edit schema

A schema is a row description. It defines the number of fields (columns) to be processed and passed on to the next component. The schema is either Built-In or stored remotely in the Repository.

Since version 5.6, both the Built-In mode and the Repository mode are available in any of the Talend solutions.

Click Edit schema to make changes to the schema. If the current schema is of the Repository type, three options are available:

  • View schema: choose this option to view the schema only.

  • Change to built-in property: choose this option to change the schema to Built-in for local changes.

  • Update repository connection: choose this option to change the schema stored in the repository and decide whether to propagate the changes to all the Jobs upon completion. If you just want to propagate the changes to the current Job, you can select No upon completion and choose this schema metadata again in the [Repository Content] window.

 

 

Built-in: The schema is created and stored locally for this component only. Related topic: see Talend Studio User Guide.

 

 

Repository: The schema already exists and is stored in the Repository, hence can be reused. Related topic: see Talend Studio User Guide.

 

Response Schema and Edit schema

A schema is a row description. It defines the number of fields (columns) to be processed and passed on to the next component. The schema is either Built-In or stored remotely in the Repository.

Since version 5.6, both the Built-In mode and the Repository mode are available in any of the Talend solutions.

Click Edit schema to make changes to the schema. If the current schema is of the Repository type, three options are available:

  • View schema: choose this option to view the schema only.

  • Change to built-in property: choose this option to change the schema to Built-in for local changes.

  • Update repository connection: choose this option to change the schema stored in the repository and decide whether to propagate the changes to all the Jobs upon completion. If you just want to propagate the changes to the current Job, you can select No upon completion and choose this schema metadata again in the [Repository Content] window.

 

 

Built-in: The schema is created and stored locally for this component only. Related topic: see Talend Studio User Guide.

 

 

Repository: The schema already exists and is stored in the Repository, hence can be reused. Related topic: see Talend Studio User Guide.

 

Fault Schema and Edit schema

A schema is a row description. It defines the number of fields (columns) to be processed and passed on to the next component. The schema is either Built-In or stored remotely in the Repository.

Since version 5.6, both the Built-In mode and the Repository mode are available in any of the Talend solutions.

Click Edit schema to make changes to the schema. If the current schema is of the Repository type, three options are available:

  • View schema: choose this option to view the schema only.

  • Change to built-in property: choose this option to change the schema to Built-in for local changes.

  • Update repository connection: choose this option to change the schema stored in the repository and decide whether to propagate the changes to all the Jobs upon completion. If you just want to propagate the changes to the current Job, you can select No upon completion and choose this schema metadata again in the [Repository Content] window.

 

 

Built-in: The schema is created and stored locally for this component only. Related topic: see Talend Studio User Guide.

 

 

Repository: The schema already exists and is stored in the Repository, hence can be reused. Related topic: see Talend Studio User Guide.

 

Use Service Registry

This option is only available if you subscribed to Talend Enterprise ESB solutions.

Select this check box to enable the Service Registry. It provides dynamic endpoint lookup and allows services to be redirected based upon information retrieved from the registry. It works in runtime only.

Enter the authentication credentials in the Username and Password field.

If SAML token is registered in the service registry, you need to specify the client's role in the Role field. You can also select the Propagate Credentials check box to make the call on behalf of an already authenticated user by propagating the existing credentials. You can enter the username and the password to authenticate via STS to propagate using username and password, or provide the alias, username and the password to propagate using certificate. For more information, see the Use Authentication option. Select the Encryption/Signature body check box to enable XML Encryption/XML Signature. For more information, see the chapter about XKMS Service in the Talend ESB Infrastructure Services Configuration Guide.

In the Correlation Value field, specify a correlation ID or leave this field empty. For more information, see the Use Business Correlation option.

For more information about how to set up and use the Service Registry, see the Talend Administration Center User Guide and Talend ESB Infrastructure Services Configuration Guide.

 

Use Service Locator

Maintains the availability of the service to help meet demands and service level agreements (SLAs).

This option will not show if the Use Service Registry check box is selected.

 

 Use Service Activity Monitor

Captures events and stores this information to facilitate in-depth analysis of service activity and track-and-trace of messages throughout a business transaction. This can be used to analyze service response times, identify traffic patterns, perform root cause analysis and more.

This option is disabled when the Use Service Registry check box is selected if you subscribed to Talend Enterprise ESB solutions.

 

 Use Authentication

Select this check box to enable the authentication option. Select from Basic HTTP, HTTP Digest, Username Token, and SAML Token (ESB runtime only). Enter a username and a password in the corresponding fields as required. Authentication with Basic HTTP, HTTP Digest, and Username Token work in both the studio and runtime. Authentication with the SAML Token works in runtime only.

When SAML Token (ESB runtime only) is selected, you can either provide the user credentials to send the request or make the call on behalf of an already authenticated user by propagating the existing credentials. Select from:

-: Enter the username and the password in the corresponding fields to access the service.

Propagate using U/P: Enter the user name and the password used to authenticate against STS.

Propagate using Certificate: Enter the alias and the password used to authenticate against STS.

To enter the password, click the [...] button next to the password field, and then in the pop-up dialog box enter the password between double quotes and click OK to save the settings.

This option will not show if the Use Service Registry check box is selected.

 

Use Authorization

This option is only available if you subscribed to Talend Enterprise ESB solutions.

Select this check box to enable authorized call. Specify the client's role in the Role field. This option appears when SAML Token (ESB runtime only) is selected in the Use Authentication list.

For more information about the management of user roles and rights, see the Talend Administration Center User Guide and Talend ESB Infrastructure Services Configuration Guide.

 

Encryption/Signature body

Select this check box to enable XML Encryption/XML Signature. For more information, see the chapter about XKMS Service in the Talend ESB Infrastructure Services Configuration Guide.

This option appears when SAML Token (ESB runtime only) is selected in the Use Authentication list.

 

Use Business Correlation

Select this check box to create a correlation ID in this component.

You can specify a correlation ID in the Correlation Value field. In this case the correlation ID will be passed on to the service it calls so that chained service calls will be grouped under this correlation ID. If you leave this field empty, this value will be generated automatically at runtime.

When this option is enabled, tESBConsumer will also extract the correlation ID from the response header and store it in the component variable for further use in the flow.

This option will be enabled automatically when the Use Service Registry check box is selected.

 

Die on error

Select this check box to kill the Job when an error occurs.

Advanced settings

Log messages Select this check box to log the message exchange between the service provider and the consumer.
 

Service Locator Custom Properties

This table appears when Use Service Locator is selected. You can add as many lines as needed in the table to customize the relevant properties. Enter the name and the value of each property between double quotation marks in the Property Name field and the Property Value field respectively.

 

Service Activity Custom Properties

This table appears when Use Service Activity Monitor is selected. You can add as many lines as needed in the table to customize the relevant properties. Enter the name and the value of each property between double quotation marks in the Property Name field and the Property Value field respectively.

 

Connection time out(second)

Set a value in seconds for Web service connection time out.

This option only works in the studio. To use it after the component is deployed in runtime:

  1. Create a configuration file with the name org.apache.cxf.http.conduits-<endpoint_name>.cfg in the <TalendRuntimePath>/container/etc/ folder.

  2. Specify the url of the Web service and the client.ConnectionTimeout parameter in milliseconds in the configuration file. If you need to use the Receive time out option, specify the client.ReceiveTimeout in milliseconds too. The url can be a full endpoint address or a regular expression containing wild cards, for example:

    url = http://localhost:8040/*
    client.ConnectionTimeout=10000000
    client.ReceiveTimeout=20000000

    , in which http://localhost:8040/* matches all urls starting with http://localhost:8040/.

 

Receive time out(second)

Set a value in seconds for server answer.

This option only works in the studio. For how to use it after the component is deployed in runtime, see the Connection time out option.

 

Trust server with SSL/TrustStore file and TrustStore password

Select this check box to validate the server certificate to the client via an SSL protocol and fill in the corresponding fields:

TrustStore file: Enter the path (including filename) to the certificate TrustStore file that contains the list of certificates that the client trusts.

TrustStore password: Enter the password used to check the integrity of the TrustStore data.

 

Use http proxy/Proxy host, Proxy port, Proxy user, and Proxy password

Select this check box if you are using a proxy server and fill in the necessary information.

To enter the password, click the [...] button next to the password field, and then in the pop-up dialog box enter the password between double quotes and click OK to save the settings.

 

tStatCatcher Statistics

Select this check box to gather the Job processing metadata at a Job level as well as at each component level.

Dynamic settings

Click the [+] button to add a row in the table and fill the Code field with a context variable to turn on or off the Use Authentication or Use HTTP proxy option dynamically at runtime. You can add two rows in the table to set both options.

Once a dynamic parameter is defined, the corresponding option becomes highlighted and unusable in the Basic settings view or Advanced settings view.

For examples on using dynamic parameters, see Scenario 3: Reading data from MySQL databases through context-based dynamic connections and Scenario: Reading data from different MySQL databases using dynamically loaded connection parameters. For more information on Dynamic settings and context variables, see Talend Studio User Guide.

Global Variables 

NB_LINE: the number of rows processed. This is an After variable and it returns an integer.

CORRELATION_ID: the correlation ID by which chained service calls will be grouped. This is a Flow variable and it returns a string.

ERROR_MESSAGE: the error message generated by the component when an error occurs. This is an After variable and it returns a string. This variable functions only if the Die on error check box is cleared, if the component has this check box.

A Flow variable functions during the execution of a component while an After variable functions after the execution of the component.

To fill up a field or expression with a variable, press Ctrl + Space to access the variable list and choose the variable to use from it.

For further information about variables, see Talend Studio User Guide.

Usage

This component can be used as an intermediate component. It requires to be linked to an output component.

Limitation

A JDK is required for this component to operate.

Scenario 1: Using tESBConsumer

This scenario describes a Job that uses a tESBConsumer component to retrieve the valid email.

Dropping and linking the components

  1. Drop the following components from the Palette onto the design workspace: a tFixedFlowInput, a tESBConsumer, two tXMLMap, and two tLogRow components.

  2. Right-click the tFixedFlowInput component, select Row > Main from the contextual menu and click the first tXMLMap component.

  3. Right-click the tXMLMap component, select Row > *New Output* (Main) from the contextual menu and click the tESBConsumer component. Enter payload in the popup dialog box to name this row and accept the propagation that prompts you to get the schema from the tESBConsumer component.

  4. Right-click the tESBConsumer component, select Row > Response from the contextual menu and click the second tXMLMap component.

  5. Right-click the second tXMLMap component, select Row > *New Output* (Main) from the contextual menu and click the second tLogRow component. Enter response in the popup dialog box to name this row.

  6. Right-click the tESBConsumer component again, select Row > Fault from the contextual menu and click the other tLogRow component.

Configuring the components

Configuring the tESBConsumer component

In this scenario, a public web service which is available at http://www.webservicex.net/ValidateEmail.asmx will be called by the tESBConsumer component to returns true or false for an email address. You can view the WSDL definition of the service at http://www.webservicex.net/ValidateEmail.asmx?WSDL for the service description.

  1. In the design workspace, double-click the tESBConsumer component to open its Basic settings view in the Component tab.

  2. Click the three-dot button next to Service configuration.

  3. In the dialog box that appears, type in: http://www.webservicex.net/ValidateEmail.asmx?WSDL in the WSDL field and click the refresh button to retrieve port name and operation name. In the Port Name list, select the port you want to use, ValidateEmailSoap in this example.

    Select the Populate schema to repository on finish to retrieve the schema from the WSDL definition, which will be used by the tFixedFlowInput component. This option is only available to users of Talend Studio with ESB. If you don't have this option, please ignore it. The schema can be created manually in the tFixedFlowInput component, which will be shown later.

    Click Finish to validate your settings and close the dialog box.

  4. Click the Advanced settings view in the Component tab.

  5. Select the Log messages check box to show the exchange log in the execution console.

Configuring the tFixedFlowInput component

  1. Double-click the tFixedFlowInput component to open its Basic settings view in the Component tab.

  2. For users of Talend Studio with ESB who have retrieved the schema from the service WSDL definition in the configuration of the tESBConsumer component, select Repository from the Schema list. Then click the [...] of the next field to show the Repository Content dialog box. Select the metadata under the IsValidEmail node to use it as the schema of the input message. Click OK to close the dialog box.

    For users of Talend Studio without ESB, please go to the next step.

  3. For users of Talend Studio without ESB, the schema need to be created manually. Select Built-In from the Schema list.

    Click the three-dot button next to Edit Schema. In the schema dialog box, click the plus button to add a new line of String type and name it Email. Click OK to close the dialog box.

  4. In the Number of rows field, set the number of rows as 1.

  5. In the Mode area, select Use Single Table and input the following request in double quotation marks into the Value field:

    nomatter@gmail.com

Configuring the tXMLMap component in the input flow

Talend data integration uses schemas based on rows and columns since it has roots in relational data warehouse integration. But SOAP messages uses the XML format. XML is hierarchical and supports richer structure than rows or columns. So we need the tXMLMap to convert from the relational row/column structure to the schema expected by the SOAP service.

  1. In the design workspace, double-click the tXMLMap component to open the Map Editor.

  2. In the output table, right-click the root node and select Rename from the contextual menu. Enter IsValidEmail in the dialog box that appears.

  3. Right-click the IsValidEmail node and select Set A Namespace from the contextual menu. Enter http://www.webservicex.net in the dialog box that appears.

  4. Right-click the IsValidEmail node again and select Create Sub-Element from the contextual menu. Enter Email in the dialog box that appears.

  5. Right-click the Email node and select As loop element from the contextual menu.

  6. Click the Email node in the input table and drop it to the Expression column in the row of the Email node in the output table.

  7. Click OK to validate the mapping and close the Map Editor.

Configuring the tXMLMap component in the output flow

The tXMLMap in the output flow will convert the response message from the XML format to the row/column structure.

  1. In the design workspace, double-click the tXMLMap component in the output flow to open the Map Editor.

  2. In the input table, right-click the root node and select Rename from the contextual menu. Enter IsValidEmailResponse in the dialog box that appears.

  3. Right-click the IsValidEmailResponse node and select Set A Namespace from the contextual menu. Enter http://www.webservicex.net in the dialog box that appears.

  4. Right-click the IsValidEmailResponse node again and select Create Sub-Element from the contextual menu. Enter IsValidEmailResult in the dialog box that appears.

  5. Right-click the IsValidEmailResult node and select As loop element from the contextual menu.

  6. On the lower right part of the map editor , click [+] to add a row of String type to the output table and name it response.

  7. Click the IsValidEmailResult node in the input table and drop it to the Expression column in the row of the response node in the output table.

  8. Click OK to validate the mapping and close the Map Editor.

The tLogRow components will monitor the exchanges from the response and fault messages and does not need any configuration. Press Ctrl+S to save your Job.

Executing the Job

Click the Run view to display it and click the Run button to launch the execution of your Job. You can also press F6 to execute it. In the execution log you will see:

The email address nomatter@gmail.com is returned as false. The input and output SOAP messages in XML are also shown in the console.

Scenario 2: Using tESBConsumer with custom SOAP Headers

This scenario is similar to the previous one. It describes a Job that uses a tESBConsumer component to retrieve a valid email address with custom SOAP headers in the request message.

Dropping and linking the components

  1. Drop the following components from the Palette onto the design workspace: a tESBConsumer, a tMap, two tFixedFlowInput, three tXMLMap, and two tLogRow.

  2. Connect each of the tFixedFlowInput with a tXMLMap using the Row > Main connection.

  3. Right-click the first tXMLMap, select Row > *New Output* (Main) from the contextual menu and click tMap. Enter payload in the popup dialog box to name this row.

    Repeat this operation to connect another tXMLMap to tMap and name the output row header.

  4. Right-click the tMap component, select Row > *New Output* (Main) from the contextual menu and click the tESBConsumer component. Enter request in the popup dialog box to name this row and accept the propagation that prompts you to get the schema from the tESBConsumer component.

  5. Right-click the tESBConsumer component, select Row > Response from the contextual menu and click the third tXMLMap component.

  6. Right-click the third tXMLMap component, select Row > *New Output* (Main) from the contextual menu and click one of the tLogRow components. Enter response in the popup dialog box to name this row.

  7. Right-click the tESBConsumer component again, select Row > Fault from the contextual menu and click the other tLogRow component.

Configuring the components

Configuring the tESBConsumer component

In this scenario, a public web service which is available at http://www.webservicex.net/ValidateEmail.asmx will be called by the tESBConsumer component to returns true or false for an email address. You can view the WSDL definition of the service at http://www.webservicex.net/ValidateEmail.asmx?WSDL for the service description.

  1. In the design workspace, double-click the tESBConsumer component to open its Basic settings view in the Component tab.

  2. Click the [...] button next to Service configuration.

  3. In the dialog box that appears, type in: http://www.webservicex.net/ValidateEmail.asmx?WSDL in the WSDL field and click the refresh button to retrieve port name and operation name. In the Port Name list, select the port you want to use, ValidateEmailSoap in this example. Click OK to validate your settings and close the dialog box.

    Select the Populate schema to repository on finish to retrieve the schema from the WSDL definition, which will be used by the tFixedFlowInput component. This option is only available to users of Talend Studio with ESB. If you don't have this option, please ignore it. The schema can be created manually in the tFixedFlowInput component, which will be shown later.

    Click Finish to validate your settings and close the dialog box.

  4. In the Advanced settings view, select the Log messages check box to log the content of the messages.

Configuring the tFixedFlowInput components

  1. Double-click the first tFixedFlowInput component to open its Basic settings view in the Component tab.

  2. For users of Talend Studio with ESB who have retrieved the schema from the service WSDL definition in the configuration of the tESBConsumer component, select Repository from the Schema list. Then click the [...] of the next field to show the Repository Content dialog box. Select the metadata under the IsValidEmail node to use it as the schema of the input message. Click OK to close the dialog box.

    For users of Talend Studio without ESB, please go to the next step.

  3. For users of Talend Studio without ESB, the schema need to be created manually. Select Built-In from the Schema list.

    Click the [...] button next to Edit Schema. In the schema dialog box, click the [+] button to add a new line of String type and name it Email. Click OK to close the dialog box.

  4. In the Number of rows field, set the number of rows as 1.

  5. In the Mode area, select Use Single Table and enter "nomatter@gmail.com" into the Value field, which is the payload of the request message.

  6. Configure the second tFixedFlowInput as the first one, except for its schema.

    Add two rows of String type to the schema and name them id and company respectively.

    Give the value Hello world! to id and Talend to company, which are the headers of the request message.

Configuring the tXMLMap components in the input flow

Talend data integration uses schemas based on rows and columns since it has roots in relational data warehouse integration. But SOAP messages uses the XML format. XML is hierarchical and supports richer structure than rows or columns. So we need the tXMLMap to convert from the relational row/column structure to the schema expected by the SOAP service.

  1. In the design workspace, double-click the first tXMLMap component to open the Map Editor.

  2. In the output table, right-click the root node and select Rename from the contextual menu. Enter IsValidEmail in the dialog box that appears.

  3. Right-click the IsValidEmail node and select Set A Namespace from the contextual menu. Enter http://www.webservicex.net in the dialog box that appears.

  4. Right-click the IsValidEmail node again and select Create Sub-Element from the contextual menu. Enter Email in the dialog box that appears.

  5. Right-click the Email node and select As loop element from the contextual menu.

  6. Click the Email node in the input table and drop it to the Expression column in the row of the Email node in the output table.

  7. Click OK to validate the mapping and close the Map Editor.

  8. Configure the other tXMLMap in the same way. Add a row of Document type to the output table and name it header. Create two sub-elements to it, id and company. Map the id and the company nodes in the input table to the corresponding nodes in the output table.

Configuring the tMap component

  1. In the design workspace, double-click tMap to open the Map Editor.