How to create a data service - 6.1

Talend Real-time Big Data Platform Studio User Guide

EnrichVersion
6.1
EnrichProdName
Talend Real-Time Big Data Platform
task
Data Quality and Preparation
Design and Development
EnrichPlatform
Talend Studio

To create a data service using the Integration perspective of Talend Studio, you have to design a data service Job that addresses all of the different sources and targets required for data integration processes and combines them with Web services.

The Integration perspective of Talend Studio provides the Services item to help create a data service Job from a given WSDL defined in the Repository tree view.

The following sections present a scenario to illustrate how to create a WSDL file and the data service Jobs to provide and consume a Web service.

Discovering the scenario

To illustrate the way Talend Studio combines data integration with Web services, find below a real-life example scenario. In this scenario, you will define an airport Web service by creating a WSDL file, send a request to this Web service to get country codes and retrieve the response from the Web service for a further use. To do this, you need to create two data service Jobs:

  • one Job that will give access to the Web service via a WSDL, to send a request and retrieve the response - the data service provider.

  • one Job that will actually send data to request the Web service - the data service consumer.

Defining the Web service

From the Services item of the Repository tree view, you are able to define the Web service of interest by creating a WSDL file or by importing an existing WSDL file.

In this scenario, we will create a new WSDL file to define the airport Web service. For further information about how to create a WSDL file from scratch, see How to create a Service and How to edit a WSDL file.

To define the airport Web service, proceed as follows:

  1. On the Repository tree view, right-click the Services node and from the contextual menu, select Create Service.

  2. In the pop-up wizard, enter the information corresponding to the fields you need to complete. In this scenario, enter airport in the Name field. Click Next to go to the next step.

  3. Select the Create new WSDL option.

  4. Click Finish to validate this creation.

    The service opens in the design workspace with a basic WSDL skeleton, which contains one service, one binding and one port type of one operation.

  5. Right-click the port airportPort in the WSDL skeleton and select Show properties.

    In its Properties view, define its name in the Name field, for example, airportSoap.

    In the Address field, specify the address where you want the service to be published, for example, http://localhost:8200/airport.service.

  6. Click the binding in the WSDL skeleton.

    In its Properties view, define the name of the binding, for example airportSoap.

  7. Click the portType airportPortType in the WSDL skeleton.

    In its Properties view, define the name of the portType, for example airportSoap.

  8. Click the operation airportOperation in the WSDL skeleton.

    In its Properties view, set its name as getAirportInformationByISOCountryCode.

  9. Save the WSDL file. We will use it to build the Web service.

    Then the newly defined Web service with exclamation icon displays under the Services node of the Repository view. The exclamation icon means that this defined Web service is not yet used.

Creating data service provider

In this scenario, the data service provider uses the tESBProviderRequest and the tESBProviderResponse components to create the access to the airport Web service and uses the tXMLMap component to join the airport data provided by a given MySQL database into the request-response main flow for publication. The database data is loaded by the tMysqlInput component.

To create this data service provider, proceed as follows:

  1. Under the Services node of the Repository tree view, right-click the operation of the newly defined airport Web service and from the contextual menu, select Assign Job. In this scenario, this operation is getAirportInformationByISOCountryCode.

  2. The [Assign Job] wizard opens. Select the Create a new Job and Assign it to this Service operation and click Next.

  3. In the [New Job] view of the wizard, the Job to be created is already named automatically, so simply click Finish.

    A draft Job is opened on the workspace.

Dropping and linking the components

In the draft Job, a tESBProviderRequest and a tESBProviderResponse are already selected and configured. tESBProviderRequest will send a request to the specified Web service and tESBProviderResponse will send back the response corresponding to the request. These two components can be found in the ESB group of the Palette.

To create the data service provider Job, proceed as follows:

  1. Right-click tESBProviderRequest, hold and drag to tESBProviderResponse to link these two components.

  2. Drop a tXMLMap in the middle of the row link from the Palette and in the pop-up window, name the output link as, for example, airport_response. This will also be used as the name of the output table in the map editor of tXMLMap. For further information, see Designing a Job.

    Then your data service provider Job should look like:

    The red cross icon prompts you to configure the tXMLMap component.

  3. From the Db Connections node of the Repository tree view, drop the connection to the airport data, the airport database table in this example, onto the workspace. Then the [Components] wizard is opened.

    For further information about how to create a database connection in the Repository, see Centralizing database metadata.

  4. Double click tMysqlInput in this wizard to create the corresponding component on the workspace and link it to tXMLMap.

In this scenario, the airport data is composed of airport names and the corresponding country codes. The following figure presents the database table in use.

Till now, you need only to configure tXMLMap as the other components are already configured automatically.

Configuring tXMLMap

To to this, perform the following operations:

  1. On the workspace, double click tXMLMap to open its editor. At this moment, the editor should look like:

  2. In the main row table of the input flow side (left), right-click the column name payload and from the contextual menu, select Import from Repository. Then the [Metadatas] wizard is opened. For further information, see Using the document type to create the XML tree.

  3. Expand the File XML node in this wizard, select the schema of the request side and click OK to validate this selection. In this example, the schema is getAirportInformationByISOCountryCode.

  4. Do the same to import the hierarchical schema for the response side (right). In this example, this schema is getAirportInformationByISOCountryCodeResponse.

  5. Then to create the join to the lookup data, click the CountryAbbrviation node in the main row of the input side (left), hold and drop it onto the Exp.key column of the lookup flow, corresponding to the id row.

  6. On the table representing the lookup flow, click the wrench icon on the up-right corner to open the setting panel.

  7. Set Lookup Model as Reload at each row, Match Model as All matches and Join Model as Inner join.

    For further information about Lookup Model, see Handling Lookups.

    For further information about Match Model, see How to use Explicit Join.

    For further information about Join Model, see How to use Inner Join.

    A step-by-step tutorial related to this Join topic is available on the Talend Technical Community Site. For further information, see http://talendforge.org/tutorials/tutorial.php?language=english&idTuto=101.

  8. Do the same to open the setting panel on the output side (right) and set the All in one option as true. This ensures that only one response is returned every time when one request is sent, as, otherwise, the airport data from the given database may provide several airports, thus several responses, to each country code that you send as one request.

    For further information about All in one, see How to output elements into one document

  9. Click the name row in the lookup flow (left), hold and drop it onto the Expression column corresponding to the tns:getAirportInformationByISOCountryCodeResult node in the XML tree view of the output flow (airport_response in this example).

    Then your tXMLMap editor should look like:

    Note

    In the real-world practice, you can as well add hierarchical data for lookup. For further information, see Talend Components Reference Guide.

  10. Click OK to close the editor and validate this configuration.

Executing the Job

Press F6 to run this Job and once launched, the Run view is opened for you to read the execution result.