tRESTClient - 6.1

Talend Components Reference Guide

EnrichVersion
6.1
EnrichProdName
Talend Big Data
Talend Big Data Platform
Talend Data Fabric
Talend Data Integration
Talend Data Management Platform
Talend Data Services Platform
Talend ESB
Talend MDM Platform
Talend Open Studio for Big Data
Talend Open Studio for Data Integration
Talend Open Studio for Data Quality
Talend Open Studio for ESB
Talend Open Studio for MDM
Talend Real-Time Big Data Platform
task
Data Governance
Data Quality and Preparation
Design and Development
EnrichPlatform
Talend Studio

tRESTClient properties

Component family

ESB/REST

 

Function

The tRESTClient component sends HTTP and HTTPS requests to a REpresentational State Transfer (REST) Web service provider and gets the corresponding responses. This component integrates well with Talend Runtime to get HTTPS support, with more QoS features to be supported in time.

Purpose

The tRESTClient component is used to interact with RESTful Web service providers by sending HTTP and HTTPS requests using CXF (JAX-RS).

Basic settings

URL

Type in the URL address of the REST server to be invoked. When the Use Service Locator check box is selected, this field will not show and the URL of the REST server will be obtained from the Service Locator server automatically.

  Relative Path

Enter the relative path of the REST server to be invoked.

For example, if you want to access http://localhost:8888/services/Customers/list:

If Use Service Locator is disabled: You can enter any of the first part of the address in the URL field, and the second part in the Relative Path field. For example, you can enter http://localhost:8888 in URL and /services/Customers/list in Relative Path. You can also enter the full path of the REST server in URL and leave Relative Path blank.

If Use Service Locator is enabled: The URL part will be given by the Service Locator. In this case, you need to know the URL part, and specify the rest part in Relative Path. This depends on the service you request. For example, on tRESTRequest, you specify REST Endpoint as http://localhost:8888/services and enable Use Service Locator. Then, if you want to use this service, on tRESTClient side, you should specify /customers/list in Relative Path.

 

HTTP Method

From this list, select an HTTP method that describes the desired action. The specific meanings of the HTTP methods are subject to definitions of your Web service provider. Listed below are the generally accepted HTTP method definitions:

- GET: retrieves data from the server end based on the given parameters.

- POST: uploads data to the server end based on the given parameters.

- PUT: updates data based on the given parameters, or if the data does not exist, creates it.

- PATCH: modifies data partially based on the given parameters.

- DELETE: removes data based on the given parameters.

  Content Type

Select XML, JSON, or FORM according to the media type of the content to be uploaded to the server end.

This list appears only when you select the POST, PUT, or PATCH HTTP method.

  Accept Type

Select the media type the client end is prepared to accept for the response from the server end.

Available options are XML, JSON, and ANY. When ANY is selected, the response message can be of any type and will be transformed into a string.

  Query parameters

Specify the URI query parameters in the form of name-value pairs.

This option is mostly used with the GET method.

Schema and Edit Schema

A schema is a row description, it defines the number of fields that will be processed and passed on to the next component.

This component uses three built-in, read-only schemas.

Click Edit Schema to view the schema structure.

Warning

Changing the schema type may result in loss of the schema structure and therefore failure of the component.

  Input Schema

Schema for the input data. This schema contains two columns:

- body: stores the content of structured input data

- string: stores the input content when it is, or is handled as, a string.

  Response Schema

Schema for server response. This schema is passed onto the next component via a Row > Response link, and it contains three columns:

- statusCode: stores the HTTP status code from the server end.

- body: stores the content of a structured response from the server end.

- string: stores the response content from the server end when it is, or is handled as, a string.

  Error Schema

Schema for error information. This schema is passed onto the next component via a Row > Error link, and it contains two columns:

- errorCode: stores the HTTP status code from the server end when an error occurs during the invocation process. The specific meanings of the errors codes are subject to definitions of your Web service provider. For reference information, visit en.wikipedia.org/wiki/List_of_HTTP_status_codes.

- errorMessage: stores the error message corresponding the error code.

  Use Service Locator Select this check box to enable the Service Locator. It maintains the availability of the service to help meet demands and service level agreements (SLAs). Specify the Service namespace and the Service name in the corresponding fields.
  Use Service Activity Monitor Select this check box to enable the Service Activity Monitor. It captures events and stores this information to facilitate in-depth analysis of service activity and track-and-trace of messages throughout a business transaction. This can be used to analyze service response times, identify traffic patterns, perform root cause analysis and more.
  Use Authentication

Select this check box if authentication is required on the REST server end. Select from Basic HTTP, HTTP Digest, SAML Token (ESB runtime only), and OAuth2 Bearer in the list. Authentication with the SAML Token works in runtime only.

If you use Basic HTTP, HTTP Digest, or SAML Token (ESB runtime only), you need to provide your username and password.

To enter the password, click the [...] button next to the password field, and then in the pop-up dialog box enter the password between double quotes and click OK to save the settings.

If you use OAuth2 Bearer, you need to fill the Bearer Token field with a base64-encoded credential string.

 

Use Authorization

Select this check box to enable authorized call. Specify the client's role in the Role field. This option appears when SAML Token (ESB runtime only) is selected in the Use Authentication list.

For more information about the management of user roles and rights, see the Talend Administration Center User Guide and Talend ESB Infrastructure Services Configuration Guide.

 

Use Business Correlation

Select this check box to create a correlation ID in this component.

You can specify a correlation ID in the Correlation Value field. In this case the correlation ID will be passed on to the service it calls so that chained service calls will be grouped under this correlation ID. If you leave this field empty, this value will be generated automatically at runtime.

When this option is enabled, tRESTClient will also extract the correlation ID from the response header and store it in the component variable for further use in the flow.

  Die on error This check box is selected to kill the Job when an error occurs. Clear the check box to skip the row on error and complete the process for error-free rows.

Advanced settings

Log messages

Select this check box to log the message exchange between the service provider and the consumer.

 

Convert Response To DOM Document

Select this check box to convert the response from the server to document type.

Clear this check box if you want the response to be handled as a string.

  Drop JSON Request Root

This option appears when HTTP Method is POST, PUT or PATCH and Content Type is JSON. Select this check box to drop root JSON elements.

 

Wrap JSON Response

This option appears and is enabled by default when JSON is selected from the Accept Type list in the Basic settings view.

With this check box selected, the response is wrapped with a root element. Clear this check box to remove the root element from the response.

  HTTP Headers

Type in the name-value pair(s) for HTTP headers to define the parameters of the requested HTTP operation.

For the specific definitions of HTTP headers, consult your REST Web service provider. For reference information, visit en.wikipedia.org/wiki/List_of_HTTP_headers.

 

Disable chunked encoding

This option appears when HTTP Method is POST, PUT or PATCH. Select this check box to disable encoding the payload as chunks.

  Service Locator Customer Properties This option appears when Use Service Locator is enabled in the Basic settings tab. Click [+] to add as many properties as needed to the table. Enter the name and the value of each property in the Property Name field and the Property Value field respectively to identify the service.
  Service Activity Customer Properties This option appears when Use Service Activity Monitor is enabled in the Basic settings tab. Click [+] to add as many properties as needed to the table. Enter the name and the value of each property in the Property Name field and the Property Value field respectively to identify the service.
 

Connection timeout

Set the amount of time, in seconds, that the client will attempt to establish a connection before it times out. If set to 0, the client will continue to attempt to open a connection indefinitely. (default: 30)

  Receive timeout Set the amount of time, in seconds, that the client will wait for a response before it times out. If set to 0, the client will wait indefinitely. (default: 60)
  Use HTTP proxy Select this check box if you are using a proxy server. Once selected, you need to provide the connection details: host, port, username and password.
  tStatCatcher Statistics Select this check box to gather the Job processing metadata at the Job level as well as at each component level.

Dynamic settings

Click the [+] button to add a row in the table and fill the Code field with a context variable to turn on or off the Use Authentication or Use HTTP proxy option dynamically at runtime. You can add two rows in the table to set both options.

Once a dynamic parameter is defined, the corresponding option becomes highlighted and unusable in the Basic settings view or Advanced settings view.

For examples on using dynamic parameters, see Scenario 3: Reading data from MySQL databases through context-based dynamic connections and Scenario: Reading data from different MySQL databases using dynamically loaded connection parameters. For more information on Dynamic settings and context variables, see Talend Studio User Guide.

Usage

This component is used as a RESTful Web service client to communicate with a RESTful service provider, with the ability to input a request to a service into a Job and return the Job result as a service response. Depending on the actions to perform, it usually works as a start or middle component in a Job or subjob.

Connections

Outgoing links:

Row: Response; Error.

Trigger: On Subjob Ok; On Subjob Error; Run if; On Component Ok; On Component Error.

Incoming links:

Row: Main; Reject.

Trigger: Run if; On Subjob Ok; On Subjob Error; On component Ok; On Component Error.

For further information regarding connections, see Talend Studio User Guide.

Global Variables

NB_LINE: the number of rows processed. This is an After variable and it returns an integer.

HEADERS: the HTTP response headers. This is a Flow variable and it returns a list of HTTP response header values.

CORRELATION_ID: the correlation ID by which chained service calls will be grouped. This is a Flow variable and it returns a string.

ERROR_MESSAGE: the error message generated by the component when an error occurs. This is an After variable and it returns a string. This variable functions only if the Die on error check box is cleared, if the component has this check box.

A Flow variable functions during the execution of a component while an After variable functions after the execution of the component.

To fill up a field or expression with a variable, press Ctrl + Space to access the variable list and choose the variable to use from it.

For further information about variables, see Talend Studio User Guide.

Limitation

n/a

Scenario 1: Getting user information by interacting with a RESTful service

This scenario describes a three-component Job that retrieves user information based on the user ID from a REST service via HTTP GET and displays the retreived user information, as well as the message exchange between the client and the server, on the Run console.

Prerequisites:

If you are a Talend ESB user, create a Job as described in Scenario 2: Using URI Query parameters to explore the data of a database, run the Job to expose a REST service, and enter the REST service URL in your Web browser, http://localhost:8088/users in this example. You should see information like the following:

If you are not a Talend ESB user, then you need to get from your REST service provider the URL, the data structure, and the required parameters of the REST service you are going to call and make necessary modifications in the scenario configurations accordingly.

Setting up the Job

  1. Drop the following components from the Palette onto the design workspace:

    • tRESTClient, used to call the REST service and retrieve user information from the server end,

    • tXMLMap, used to adapt the tree structure of the REST service, and

    • tLogRow, to display the retrieved user information on the Run console.

  2. Connect the tRESTClient to the tXMLMap using a Row > Response connection.

  3. Connect the tXMLMap to the tLogRow using a Row > Main connection, and give it a name, out in this example.

  4. Label the components to best describe the actions to perform.

Configuring the components

Configuring the service call

  1. Double-click the tRESTClient component to open its Basic settings view.

  2. Fill the URL field with the URL of the REST service you are going to invoke, "http://localhost:8088/users" in this example. Note that the URL provided in this use case is for demonstration purposes only and not a live address.

  3. From the HTTP Method list, select GET to send an HTTP request for retrieving the existing records.

    From the Accept Type list, select the type the client end is prepared to accept for the response from the server end, XML. Leave the rest of the settings as they are.

  4. Click the [+] button beneath the Query parameters table to add two parameters, from and to, and set both parameters to 2, to get the information of the user with the ID of 2.

    Alternatively, you can query the information of the user with the ID of 2 by adding ?from=2&to=2 to the service URL.

  5. In the Advanced settings view of the tRESTClient component, select the Log messages and the Convert Response To DOM Document check boxes to log the message exchange to the server and convert the response from the server to document type.

Mapping the service structure and displaying the retrieved user information

  1. Double-click the tXMLMap component to open the Map Editor.

  2. If you selected XML in the Accept Type list of the tRESTClient component, define the input XML tree structure according to the service structure.

    1. In the input table, right-click the default root node of the body column, select Rename from the contextual menu, and rename it to users.

    2. Right-click the users node, select Create Sub-Element from the contextual menu, and create sub-element named user. Set user as the loop element because the XML structure of the Web service to be invoked is looped on this element.

    3. Right-click the user node, select Create Attribute from the contextual menu, and enter id in the [Create New Attribute] dialog box to create an attribute named id for the user node.

    4. Right-click the user node again, select Create Sub-Element from the contextual menu, and enter first_name in the [Create New Element] dialog box to create an sub-element named first_name for the user node.

      Repeat this operation to create another sub-element under the user node, last_name.

    5. Drop the id, first_name and last_name columns from the input table to the output table, and then click OK to validate the mapping and close the Map Editor.

    If you selected JSON in the Accept Type list of the tRESTClient component, the response from the server end will be sent back in JSON format and converted to document type. In this example, the converted response structure looks like the following:

    <root>
        <users>
            <user>
                <id>2</id>
                <first_name>Theodore</first_name>
                <last_name>Harding</last_name>
            </user>
        </users>
    </root>

    Note that the <root> element is removed if the Wrap JSON Response check box is cleared in the Advanced settings of the tRESTClient component.

    Define the input XML tree structure accordingly and map it with the output data flow in a similar manner as described above.

  3. Double-click the tLogRow component to open its Basic settings view.

  4. Click the Sync columns button to make sure the component schema is synchronized with the output schema of the tXMLMap component.

  5. In the Mode field, select the Table option to display the GET result in table cells.

Saving and executing the Job

  1. Press Ctrl+S to save your Job.

  2. Press F6 or click Run on the Run console to launch the Job.

    The console shows that the tRESTClient component successfully reads the user information from the server end corresponding to the specified user ID.

    If you selected XML in the Accept Type list of the tRESTClient component, the execution result will be:

    If you selected JSON in the Accept Type list of the tRESTClient component, the execution result will be:

Scenario 2: Updating user information by interacting with a RESTful service

This scenario describes a three-component Job that updates the information of a list of users to a remote database through a REST service using the HTTP POST method. When executed, the Job displays the client-server message exchange information on the Run console.

The user information to be updated to the server is stored in a CSV file, which looks like the following:

id;first_name;last_name
1;John;Smith
2;Martin;Reagan
3;James;White
4;Jenny;Jackson
5;Robert;Thomson

Prerequisites:

If you are a Talend ESB user, create a Job as described in Scenario 3: REST service accepting HTTP POST requests and run the Job as a REST server to expose a REST service that accepts HTTP POST requests. Upon execution of the Job, the console displays the service implementation information, including the service endpoint URL, which is http://localhost:8045/users in this example. If you enter http://localhost:8045/users?_wadl in your Web browser, you should see the service definition information like the following:

If you are not a Talend ESB user, then you need to get the service-related information from your REST service provider including the URL, the resource path, and the data structure, and make necessary modifications in the scenario configurations accordingly.

Setting up the Job

  1. Create a Job and add the following components by typing their names in the design workspace or drop them from the Palette:

    • tFileInputDelimited, to read user information of a local file,

    • tXMLMap, to adapt the input structure to the REST service structure, and

    • tRESTClient, used to call the REST service to send data to the remote database.

  2. Connect the tFileInputDelimited to the tXMLMap using a Row > Main connection.

  3. Connect the tXMLMap to the tRESTClient using a Row > Main connection, and give the output flow a name, request in this example.

  4. Label the components to best describe the actions to perform.

Configuring the components

Configuring the input data and the structure mappings

  1. Double-click the tFileInputDelimited component to open its Basic settings view.

  2. Specify the input file in the File name field, fill the Header field with 1 to skip the header row, and keep the rest parameters as they are.

  3. Click the [...] button next to Edit schema to open the [Schema] dialog box, and edit the input schema as follows:

    • id, type Integer, 2 characters long, set as the key column

    • first_name, type String

    • last_name, type String

  4. Double-click the tXMLMap component to open the Map Editor.

  5. Rename the root node in the output table: right-click the node, select Rename from the contextual menu, and specify a new name in the pop-up dialog box, user in this example.

  6. Select all the three columns in the input table and drop them onto the user node, and select the Create as sub-element of target node option from the pop-up dialog box to set these columns as sub-elements of the user node. When done, click OK to validate the mappings and close the Map Editor.

Configuring the service call

  1. Double-click the tRESTClient component to open its Basic settings view.

  2. Fill the URL field with the URI location where the REST service is accessible, "http://localhost:8088/users" in this example.

  3. In the Relative Path field, enter the resource path, "/post/" + row1.id+ "/" + row1.first_name + "/" + row1.last_name in this example. This will send the data from the input row to the server end via the resource path.

  4. From the HTTP Method list, select GET to send an HTTP request for retrieving the existing records.

    From the Accept Type list, select the type the client end is prepared to accept for the response from the server end, XML.

  5. In the Advanced settings view of the tRESTClient component, select the Log messages check box to log the message exchange information with the server.

    Leave the rest of the settings as they are.

Running the Job and checking the result

Press Ctrl+S to save your Job, and press F6 to execute it.

The console displays the client-server exchange information:

The console of the server Job displays the exchange information and the database update result.