tRESTRequest - 6.1

Talend Components Reference Guide

EnrichVersion
6.1
EnrichProdName
Talend Big Data
Talend Big Data Platform
Talend Data Fabric
Talend Data Integration
Talend Data Management Platform
Talend Data Services Platform
Talend ESB
Talend MDM Platform
Talend Open Studio for Big Data
Talend Open Studio for Data Integration
Talend Open Studio for Data Quality
Talend Open Studio for ESB
Talend Open Studio for MDM
Talend Real-Time Big Data Platform
task
Data Governance
Data Quality and Preparation
Design and Development
EnrichPlatform
Talend Studio

tRESTRequest properties

Warning

This component is available only to users of Talend Studio with ESB.

Component family

ESB/REST

 

Function

tRESTRequest is a server-side component which accepts the HTTP and/or HTTPS requests from the clients and support GET, POST, PUT, PATCH and DELETE HTTP methods.

Note

To enable the HTTPS support, you have to generate a keystore and add some HTTPS security configuration properties in the org.ops4j.pax.web.cfg file of your Runtime container before deploying the service on it. For more information, see the Talend ESB Container Administration Guide.

Purpose

This component allows you to receive GET/POST/PUT/PATCH/DELETE requests from the clients on the server end.

Basic settings

REST Endpoint

Fill this field with the URI location where REST-ful web service will be accessible for requests.

Note

If you want your service to be available on both HTTP and HTTPS, fill the field with a relative path.

 

REST API Mapping

Click the [+] button beneath the mapping table to add lines to specify HTTP request:

Output Flow: Click the [...] button to specify the name of an output flow and set the schema for that output flow in the dialog box afterwards.

The schema is not mandatory, so if you do not need to pass additional parameters to the tRESTRequest component, you can leave the schema empty. However, you will have to populate the schema if you have URI Path parameters set in the URI Pattern field or if you need to add optional request parameters such as URI Query, HTTP Header or Form parameters, to the URI specified in the REST Endpoint field.

Add a schema with the name body to get the request body of POST and PUT methods. It supports Document, String, and Byte[] types.

If you specify URI parameters in the output flow schema, you might need to define what type of parameter it is in the Comment field of the schema. By default, if you leave the Comment field empty, the parameter is considered as a Path parameter. Below is a list of supported Comment values:

  • empty or path corresponds to the default @PathParam,

  • query corresponds to @QueryParam,

  • form corresponds to @FormParam,

  • header corresponds to @HeaderParam.

  • matrix corresponds to @MatrixParam.

  • multipart corresponds to the CXF specific @Multipart, representing the request body. It can be used only with POST and PUT HTTP methods.

Note

We recommend you to set the default values of your optional parameters (Header, Query, Form). To do so, fill in the Default columns of the schema.

HTTP Verb: Select a HTTP method (GET/POST/PUT/PATCH/DELETE/OPTIONS/HEAD) from the list.

URI pattern: Fill this field with REST-ful URIs that describe the resource.

Consumes: Select the format type of the consume content that the component will use from XML or JSON, XML, JSON, Form, Multipart and Any in the list when the HTTP method is POST, PUT or PATCH.

Produces: When the HTTP method is GET, POST, PUT, PATCH, or DELETE, select the format type of the produce content that the component will use from XML or JSON, XML, JSON, HTML and Any, or select <oneway> in the list to accept one way requests.

Streaming: Select this check box to stream the response data by chunks so that the large volumes of data can be processed efficiently.

  Use Service Locator Select this check box to enable the Service Locator. It maintains the availability of the service to help meet demands and service level agreements (SLAs). Specify the Service namespace and the Service name in the corresponding fields.
  Use Service Activity Monitor Select this check box to enable the Service Activity Monitor. It captures events and stores this information to facilitate in-depth analysis of service activity and track-and-trace of messages throughout a business transaction. This can be used to analyze service response times, identify traffic patterns, perform root cause analysis and more.

 

Use Authentication (ESB runtime only)

Select this check box to enable the authentication option for the current service. Select from Basic HTTP and SAML Token. This option works in runtime only.

 

Use Authorization

Select this check box to enable authorized call. This option appears when SAML Token (ESB runtime only) is selected in the Use Authentication list.

For more information about the management of user roles and rights, see the Talend Administration Center User Guide and Talend ESB Infrastructure Services Configuration Guide.

 

Use Business Correlation

Select this check box to enable the correlation option so that chained service calls will be grouped under the same correlation ID. tRESTRequest will extract the correlation ID from the request header and store it in the component variable for further use in the flow.

If this option is not enabled on the client side, a correlation ID will be generated automatically in tRESTRequest.

Advanced settings Log messages Select this check box to log the message exchange between the service provider and the consumer.
  Wrap JSON Request Select this check box to wrap the JSON request with a root element.
  Convert JSON values to String in responseSelect this check box to convert the JSON values to string format in the response.
  Service Locator Customer Properties This option appears when Use Service Locator is enabled in the Basic settings tab. Click [+] to add as many properties as needed to the table. Enter the name and the value of each property in the Property Name field and the Property Value field respectively to identify the service.
  Service Activity Customer Properties This option appears when Use Service Activity Monitor is enabled in the Basic settings tab. Click [+] to add as many properties as needed to the table. Enter the name and the value of each property in the Property Name field and the Property Value field respectively to identify the service.
 

tStatCatcher Statistics

Select this check box to gather the Job processing metadata at a Job level as well as at each component level.

Global Variables

NB_LINE: the number of rows processed. This is an After variable and it returns an integer.

URI: the URI of the REST request. This is a Flow variable and it returns a string.

URI_BASE: the base URI of the REST request. This is a Flow variable and it returns a string.

URI_ABSOLUTE: the absolute URI of the REST request. This is a Flow variable and it returns a string.

HTTP_METHOD: the http method. This is a Flow variable and it returns a string.

ATTACHMENT_HEADERS: the attachment headers from the REST request. This is a Flow variable and it returns a list of attachment header values.

ATTACHMENT_FILENAMES: the attachment filenames from the REST request. This is a Flow variable and it returns all attachment filenames.

PRINCIPAL_NAME: the principal name of the REST request. This is a Flow variable and it returns a string.

CORRELATION_ID: the correlation ID by which chained service calls will be grouped. This is a Flow variable and it returns a string.

ERROR_MESSAGE: the error message generated by the component when an error occurs. This is an After variable and it returns a string. This variable functions only if the Die on error check box is cleared, if the component has this check box.

A Flow variable functions during the execution of a component while an After variable functions after the execution of the component.

To fill up a field or expression with a variable, press Ctrl + Space to access the variable list and choose the variable to use from it.

For further information about variables, see Talend Studio User Guide.

Usage

This component covers the possibility that a Talend Job can be wrapped as a service, with the ability to input a request to a service into a Job and return the Job result as a service response.

The tRESTRequest component should be used with the tRESTResponse component to provide a Job result as a response, in case of a request-response communication style.

Limitation

n/a

Scenario 1: REST service accepting HTTP GET requests and sending responses

This scenario describes the process of accepting an HTTP request from the client, processing it and sending the response back.

Configuring the tRESTRequest component

  1. Drop the following components from the Palette onto the design workspace: a tRESTRequest, a tXMLMap and two tRESTResponse.

  2. Double-click tRESTRequest in the design workspace to display its Basic settings view.

  3. Fill the REST Endpoint field with the URI location where the REST-ful web service will be accessible for requests. For example, "http://192.168.0.235:8088/user".

    Note

    If you want your service to be available on both HTTP and HTTPS, fill the field with a relative path. For example, if you type in "/test", your service will be available on both http://<DefaultHTTPEnpointAddress>/test and https://<DefaultHTTPSEnpointAddress>/test, provided that you have configured your Runtime container to support HTTPS. For more information, see the Talend ESB Container Administration Guide.

  4. Click the [+] button to add one line in the REST API Mapping table.

  5. Select the newly-added line and click the [...] button in the Output Flow column to add a schema for the output flow.

    In this scenario, the output flow will be named as GetOneUser.

    Then click the [+] button to add a new line id to the schema in the dialog box.

  6. Click OK to save the schema.

  7. Select GET from the list in the HTTP Verb column.

  8. Fill the field in the URI Pattern column with "/{id}/".

  9. In the same way, add another line in the REST API Mapping table and name this schema GetUserNumber. Add a line string of String type to the schema. Enter query in the Comment field to get the number parameter in the request.

    Select GET from the list in the HTTP Verb column. Fill the field in the URI Pattern column with "/number".

Configuring the tXMLMap component

  1. Connect tRESTRequest to tXMLMap using the Row > GetOneUser connection.

  2. Double-click tXMLMap in the design workspace to open the Map Editor.

  3. Click the [+] button on the top right to add an output and name it as ResponseUsers.

  4. Click the [+] button on the bottom right to add two columns for the output.

    Name the first column as body and set the Type to Document.

    Name the second column as string and set the Type to String.

  5. Right-click on the node root and select Create Sub-Element to create a sub-element. Name the sub-element as foo in the popup dialog box.

  6. Right-click on the foo node created in the previous step and select As loop element.

  7. Select the id column of the GetOneUser table to the left and drop it onto the Expression field of the foo node of the ResponseUsers table to the right.

  8. Click OK to save the settings.

Configuring the tRESTResponse component

  1. Connect tXMLMap to a tRESTResponse using Row > ResponseUsers connection.

    The schema defined in tXMLMap is retrieved in tRESTResponse automatically.

  2. Select OK(200) from the Return status code list.

  3. Leave the rest of the settings as they are.

  4. Connect tRESTRequest to the other tRESTResponse using the Row > GetUserNumber connection.

Saving and executing the Job

  1. Save the Job and press F6 to execute it.

  2. Go to your browser if you want to test the service.

    The HTTP request for a user id is accepted by the REST service and the HTTP response is sent back to the server.

  3. Repeat this step and type in http://localhost:8088/user/number?string=123 in the address bar. Press Enter.

    An XML parsing error is reported because the returned body content is String type. Right-click the browser an select View Page Source in the context menu.

    You can see that 123 is returned in the response.

Scenario 2: Using URI Query parameters to explore the data of a database

This scenario describes how to use URI query parameters in tRESTRequest to explore data of a database, and send the response via the tRESTResponse.

To do so, you can create two subjobs linked together by an OnSubjobOk connection; this way the two subjobs will be executed sequentially. For more information on Trigger connection, see the Talend Studio User Guide. The first subjob will create and populate the database and the second one will allow to explore the database through the REST service.

Creating the first subjob

To do this, proceed as follows:

  1. Drop the following components from the Palette onto the design workspace: tFixedFlowInput from the Misc family and tMysqlOutput from the Databases > Mysql family.

  2. Link tFixedFlowInput to tMysqlOutput using a Row > Main connection.

  3. Double-click tFixedFlowInput to display its Basic settings view:

  4. Click the [...] button next to Edit schema to open the schema editor.

  5. In the schema editor, click the [+] button three times to add three lines and set them as displayed in the above screenshot.

  6. Click Ok.

  7. Back to tFixedFlowInput Basic settings view, in the Mode area, select the Use inline table option.

  8. Under the inline table, click the [+] button three times to add three rows in the table.

  9. In the inline table, click the id field of the first row and type in 1.

  10. Click the firstname field of the first row, press Ctrl+Space to display the autocompletion list and select the TalendDataGenerator.getFirstName() variable in the list.

  11. Click the lastname field of the first row, press Ctrl+Space to display the autocompletion list and select the TalendDataGenerator.getLastName() variable in the list.

  12. Do the same for the two following rows to obtain the settings displayed in the screenshot.

  13. Double-click tMysqlOutput to display its Basic settings view:

  14. From the Property Type list, leave Built-in and fill in the Host, Port, Database, Username and Password fields manually. If you centralized your connection information to the database in the Metadata > DB Connections node of the Repository, you can select Repository from the list and the fields will be automatically filled in.

    For more information about storing metadata, see Talend Studio User Guide.

  15. In the Table field, type in the name of the table in which the data will be loaded, for example: users.

  16. From the Action on table list, select Drop table if exists and create, select Insert from the Action on data list.

  17. Click Sync columns to retrieve the schema coming from the previous component.

Creating the second subjob

To do this, proceed as follows:

  • Drop and place the following components as displayed in the first screenshot:

    • tRESTRequest and tRESTResponse from the ESB > REST family,

    • tFlowToIterate from the Orchestration family,

    • tMysqlInput from the Databases > Mysql family,

    • tXMLMap from the Processing family.

Configuring the tRESTRequest component

To do this, proceed as follows:

  1. Double-click tRESTRequest in the design workspace to display its Basic settings view:

  2. Fill the REST Endpoint field with the URI location where the REST-ful web service will be accessible for requests. For example, "http://localhost:8088/users".

  3. Click the [+] button to add one line in the REST API Mapping table.

  4. Select the newly-added line and click the [...] button in the Output Flow column to add a schema for the output flow.

  5. In the dialog box, name the output flow getUsers. A schema editor dialog box appears.

  6. In the schema editor, click the [+] button twice to add two lines and set them as displayed in the above screenshot.

  7. Click OK.

  8. Back to tRESTRequest Basic settings view, select GET from the list in the HTTP Verb column.

  9. Leave the URI Pattern column as is.

Now that you created the tRESTRequest output flow, you can use the corresponding link to connect to the following component:

  1. Connect tRESTRequest to tFlowToIterate using Row > getUsers connection.

  2. Leave the tFlowToIterate settings as is.

  3. Connect tFlowToIterate to tMysqlInput using Row > Iterate connection.

Configuring the tMysqlInput component

To do this, proceed as follows:

  1. Double-click tMysqlInput to display its Basic settings view:

  2. From the Property Type list, leave Built-in and fill in the Host, Port, Database, Username and Password fields manually. If you centralized your connection information to the database in the Metadata > DB Connections node of the Repository, you can select Repository from the list and the fields will be automatically filled in.

    For more information about storing metadata, see Talend Studio User Guide.

  3. Leave the Schema list as Built-in and click the [...] button next to the Edit schema field.

  4. In the schema editor, define the schema exactly like the one of the tFixedFlowInput.

  5. In the Table Name field, fill in the name of the table in which the data are stored: users.

  6. Leave the Query Type list as Built-in and fill in the Query field with the following SQL query allowing to explore the database data with the URI query set in the tRESTRequest component:

    "select * from users where id >= " + globalMap.get("getUsers.from") + " and  id <= " + globalMap.get("getUsers.to")
Configuring the tXMLMap component
  1. Right-click tMysqlInput, hold and drag to tXMLMap to connect the two components together.

  2. Double-click tXMLMap in the design workspace to open the Map Editor.

  3. Click the [+] button on the top right to add an output and name it as ResponseUsers.

  4. Click the [+] button on the bottom right to add two columns for the output.

    Name the first column as body and set the Type to Document.

    Name the second column as string and set the Type to String.

  5. Right-click on the root node, select Rename in the list and rename it users

  6. Right-click on the users node and select Create Sub-Element to create a sub-element. Name the sub-element user in the popup dialog box.

  7. Right-click on the user node created in the previous step and select As loop element.

  8. Select the id column of the row2 table to the left and drop it onto the user node of the ResponseUsers table to the right.

  9. In the [Selection] dialog box, select the Create as attribute of target node option and click OK.

  10. Select the firstname and lastname columns of the row2 table to the left and drop it onto the user node of the ResponseUsers table to the right.

  11. In the [Selection] dialog box, select the Create as sub-element of target node option and click OK.

  12. Click the wrench icon on the top of the ResponseUsers table to open the setting panel.

  13. Set the All in one feature as true, this way all XML data is outputted in one single flow.

  14. Click OK to save the settings.

Configuring the tRESTResponse component
  1. Connect tXMLMap to tRESTResponse using Row > ResponseUsers connection.

  2. The schema defined in tXMLMap is retrieved in tRESTResponse automatically. Leave the other settings as they are.

Connecting the two subjobs

Now that the two subjobs are created, you can connect them together:

  1. Right-click the tFixedFlowInput component of the first subjob.

  2. Select Trigger > OnSubjobOk on the list.

  3. Click the tRESTRequest component of the second subjob.

This way, when executing the job, the second subjob will be executed only if the first one's execution succeeded.

Saving and executing the Job

  1. Save the Job and press F6 to execute it.

  2. Go to your browser if you want to test the service.

    For example, use the URI query ?to=2 to retrieve the data of the two first users.

    The HTTP request for a user id is accepted by the REST service and the HTTP response is sent back to the server.

    For a use case that calls this REST service using Talend ESB components, see Scenario 1: Getting user information by interacting with a RESTful service.

Scenario 3: REST service accepting HTTP POST requests

This scenario describes a Job composed of two subjobs: the first subjob exposes a REST service that accepts HTTP POST requests from REST clients, writes data to a database upon receiving an HTTP request, and displays the server-client exchange information on the Run console; the second subjob displays the results of database updates.

Setting up the Job

  1. Create a Job and add the following components to the Job by typing their names in the design workspace or dropping them from the Palette: a tRESTRequest, a tMysqlOutput, a tXMLMap, and a tRESTResponse.

  2. Connect the tRESTRequest to the tMysqlOutput component using a Row > Main connection, and give the output flow a name, request in this example.

  3. Connect the tMysqlOutput component to the tXMLMap component using a Row > Main connection.

  4. Connect the tXMLMap component to the tRESTResponse component using a Row > Main connection, and give it a name, response in this example. Click OK in the pop-up dialog box to accept the schema propagation from the tRESTResponse component.

  5. Add a tMysqlInput component and a tLogRow component to the Job by typing their names in the design workspace or dropping them from the Palette, and connect the tMysqlInput to the tLogRow using a Row > Main connection.

  6. Connect the tMysqlOutput to the tMysqlInput using a Trigger > OnComponentOk connection to link the two subjobs.

Configuring the components

Configuring REST request parameters

  1. Double-click the tRESTRequest component to open its Basic settings view.

  2. Fill the REST Endpoint field with the URI location where the REST service will be accessible for requests, "http://localhost:8045/users" in this example.

  3. Click the output flow name, which is request in this example, in the Output Flow column of the REST API Mapping table to show the [...] button, and click the button to open the [Schema] dialog box. Then define the schema for the request flow as follows:

    • id, type Integer, 2 characters long, set as the key column

    • first_name, type String

    • last_name, type String

    When done, click OK to validate the schema setting and close the dialog box.

  4. Click the HTTP Verb column and select POST from the list.

  5. In the URI Pattern column, enter the acceptable URI pattern of POST requests, "/post/{id}/{first_name}/{last_name}" in this example.

    Leave the rest parameters as they are.

  6. Click the Advanced settings tab, and select the Log messages check box.

Configuring the database and the response parameters

  1. Double-click the tMysqlOutput component to open its Basic settings view.

  2. Leave the Property Type as Built-in and fill in the database connection details manually in the Host, Port, Database, Username and Password fields.

    If you have centralized your connection information to the database in the Metadata > DB Connections node of the Repository, you can select Repository from the Property Type list and browse to the centralized connection to have the fields automatically filled in. For more information about storing metadata, see Talend Studio User Guide.

  3. Fill in the Table field with the target database table name, users in this example.

  4. Select the actions to be carried out on the database table and data according to your needs from the corresponding lists. In this example, the target table will be created if it does not exist in the specified database, and data from client requests will be inserted, or updated if it already exists, to the database table.

  5. Click the Sync columns button to synchronize the table schema with the input schema.

  6. Double-click the tXMLMap component to open the Map Editor.

  7. Rename the root node in the output table: right-click the node, select Rename from the contextual menu, and specify a new name in the pop-up dialog box, user in this example.

  8. Select all the three columns in the input table and drop them onto the user node, and select the Create as sub-element of target node option from the pop-up dialog box to set these columns as sub-elements of the user node. When done, click OK to validate the mappings and close the Map Editor.

  9. Double-click the tRESTResponse component and set the response parameters according to your needs. In this example, we keep the default settings for all the parameters.

Configuring result display

  1. Double-click the tMysqlInput component to open its Basic settings view.

  2. Specify the database connection details, table name and table schema, which are the same as in the tMysqlOutput component.

  3. Click the Guess Query button to fill the Query field with the query statement.

  4. Double-click the tLogRow component and select the Table option to display the content of the database in table cells.