tBufferOutput - 6.1

Talend Components Reference Guide

EnrichVersion
6.1
EnrichProdName
Talend Big Data
Talend Big Data Platform
Talend Data Fabric
Talend Data Integration
Talend Data Management Platform
Talend Data Services Platform
Talend ESB
Talend MDM Platform
Talend Open Studio for Big Data
Talend Open Studio for Data Integration
Talend Open Studio for Data Quality
Talend Open Studio for ESB
Talend Open Studio for MDM
Talend Real-Time Big Data Platform
task
Data Governance
Data Quality and Preparation
Design and Development
EnrichPlatform
Talend Studio

tBufferOutput properties

Component family

Misc

 

Function

This component collects data in a buffer in order to access it later via webservice for example.

Purpose

This component allows a Webservice to access data. Indeed it had been designed to be exported as Webservice in order to access data on the web application server directly. For more information, see Talend Studio User Guide.

Basic settings

Schema and Edit Schema

A schema is a row description, it defines the number of fields that will be processed and passed on to the next component. The schema is either built-in or remote in the Repository.

Since version 5.6, both the Built-In mode and the Repository mode are available in any of the Talend solutions.

Click Edit schema to make changes to the schema. If the current schema is of the Repository type, three options are available:

  • View schema: choose this option to view the schema only.

  • Change to built-in property: choose this option to change the schema to Built-in for local changes.

  • Update repository connection: choose this option to change the schema stored in the repository and decide whether to propagate the changes to all the Jobs upon completion. If you just want to propagate the changes to the current Job, you can select No upon completion and choose this schema metadata again in the [Repository Content] window.

In the case of the tBufferOutput, the column position is more important than the column label as this will be taken into account.

 

 

Built-in: The schema will be created and stored locally for this component only. Related topic: see Talend Studio User Guide.

 

 

Repository: The schema already exists and is stored in the Repository, hence can be reused in various projects and Job designs. Related topic: see Talend Studio User Guide.

Global Variables

ERROR_MESSAGE: the error message generated by the component when an error occurs. This is an After variable and it returns a string. This variable functions only if the Die on error check box is cleared, if the component has this check box.

A Flow variable functions during the execution of a component while an After variable functions after the execution of the component.

To fill up a field or expression with a variable, press Ctrl + Space to access the variable list and choose the variable to use from it.

For further information about variables, see Talend Studio User Guide.

Usage

This component is not startable (green background) and it requires an output component.

Scenario 1: Buffering data

This scenario describes an intentionally basic Job that bufferizes data in a child job while a parent Job simply displays the bufferized data onto the standard output console. For an example of how to use tBufferOutput to access output data directly on the Web application server, see Scenario 2: Buffering output data on the webapp server.

  • Create two Jobs: a first Job (BufferFatherJob) runs the second Job and displays its content onto the Run console. The second Job (BufferChildJob) stores the defined data into a buffer memory.

  • On the first Job, drop the following components: tRunJob and tLogRow from the Palette to the design workspace.

  • On the second Job, drop the following components: tFileInputDelimited and tBufferOutput the same way.

Let's set the parameters of the second Job first:

  • Select the tFileInputDelimited and on the Basic Settings tab of the Component view, set the access parameters to the input file.

  • In File Name, browse to the delimited file whose data are to be bufferized.

  • Define the Row and Field separators, as well as the Header.

  • Describe the Schema of the data to be passed on to the tBufferOutput component.

  • Select the tBufferOutput component and set the parameters on the Basic Settings tab of the Component view.

  • Generally the schema is propagated from the input component and automatically fed into the tBufferOutput schema. But you could also set part of the schema to be bufferized if you want to.

  • Now on the other Job (BufferFatherJob) Design, define the parameters of the tRunJob component.

  • Edit the Schema if relevant and select the column to be displayed. The schema can be identical to the bufferized schema or different.

  • You could also define context parameters to be used for this particular execution. To keep it simple, the default context with no particular setting is used for this use case.

Press F6 to execute the parent Job. The tRunJob looks after executing the child Job and returns the data onto the standard console:

Scenario 2: Buffering output data on the webapp server

This scenario describes a Job that is called as a Webservice and stores the output data in a buffer directly on the server of the Web application. This scenario creates first a Webservice oriented Job with context variables, and next exports the Job as a Webservice.

Creating a Job

  1. Drop the following components from the Palette onto the design workspace: tFixedFlowInput and tBufferOutput.

  2. Connect tFixedFlowInput to tBufferOutput using a Row Main link.

Creating a context variable

For this scenario, you will define two context variables: nb_lines and lastname. The first variable will set the number of lines the tFixedFlowInput component will generate, and the second one will set the last name to display in the output list. For more information about how to create and use context variables, see Talend Studio User Guide.

To define the two context variables:

  1. Select the Contexts tab view of your Job, and click the [+] button at the bottom of the view to add two variables, respectively nb_lines of type Integer and lastname of type String.

  2. In the Value field for the variables, set the last name to be displayed and the number of lines to be generated, respectively Ford and 3 in this example.

Configuring the input data

  1. In the design workspace, select tFixedFlowInput.

  2. Click the Component tab to define the basic settings for tFixedFlowInput.

  3. Click the three-dot [...] button next to Edit Schema to describe the data structure you want to create from internal variables. In this scenario, the schema is made of three columns, now of type Date, firstname of type String, and lastname of type String.

  4. Click OK to close the dialog box and accept propagating the changes when prompted by the system. The three defined columns display in the Values panel of the Basic settings view of tFixedFlowInput.

  5. Click in the Value cell of each of the first two defined columns and press Ctrl+Space to access the global variable list.

  6. From the global variable list, select TalendDate.getCurrentDate() and talendDatagenerator.getFirstName, for the now and firstname columns respectively.

  7. Click in the Value cell of lastname column and press Ctrl+Space to access the global variable list.

  8. From the global variable list, select context.lastname, the context variable you created for the last name column.

Building your Job as a Webservice

Before building your Job as a Web service, see Talend Studio User Guide for more information.

  1. In the Repository tree view, right-click on the above created Job and select Build Job. The [Build Job] dialog box appears.

  2. Click the Browse... button to select a directory to archive your Job in.

  3. In the Build type panel, select the build type you want to use in the Tomcat webapp directory (WAR in this example) and click Finish. The [Build Job] dialog box disappears.

  4. Copy the War folder and paste it in a Tomcat webapp directory.

Scenario 3: Calling a Job with context variables from a browser

This scenario describes how to call the Job you created in Scenario 2: Buffering output data on the webapp server from your browser with/without modifying the values of the context variables.

Type the following URL into your browser: http://localhost:8080//export_job/services/export_job3?method=runJob where "export_job" is the name of the webapp directory deployed in Tomcat and "export_job3" is the name of the Job.

Click Enter to execute your Job from your browser.

The Job uses the default values of the context variables: nb_lines and lastname, that is it generates three lines with the current date, first name and Ford as a last name.

You can modify the values of the context variables directly from your browser. To call the Job from your browser and modify the values of the two context variables, type the following URL:

http://localhost:8080//export_job/services/export_job3?method=runJob&arg1=--context_param%20lastname=MASSY&arg2=--context_param%20nb_lines=2.

%20 stands for a blank space in the URL language. In the first argument "arg1", you set the value of the context variable to display "MASSY" as last name. In the second argument "arg2", you set the value of the context variable to "2" to generate only two lines.

Click Enter to execute your Job from your browser.

The Job generates two lines with MASSY as last name.

Scenario 4: Calling a Job exported as Webservice in another Job

This scenario describes a Job that calls another Job exported as a Webservice using the tWebServiceInput. This scenario will call the Job created in Scenario 2: Buffering output data on the webapp server.

  • Drop the following components from the Palette onto the design workspace: tWebServiceInput and tLogRow.

  • Connect tWebserviceInput to tLogRow using a Row Main link.

  • In the design workspace, select tWebServiceInput.

  • Click the Component tab to define the basic settings for tWebServiceInput.

  • Set the Schema Type to Built-In and click the three-dot [...] button next to Edit Schema to describe the data structure you want to call from the exported Job. In this scenario, the schema is made of three columns, now, firstname, and lastname.

  • Click the plus button to add the three parameter lines and define your variables. Click OK to close the dialog box.

  • In the WSDL field of the Basic settings view of tWebServiceInput, enter the URL http://localhost:8080/export_job/services/export_job3?WSDL where "export_job" is the name od the webapp directory where the Job to call is stored and "export_job3" is the name of the Job itself.

  • In the Method name field, enter runJob.

  • In the Parameters panel, Click the plus button to add two parameter lines to define your context variables.

  • Click in the first Value cell to enter the parameter to set the number of generated lines using the following syntax: --context_param nb_line=3.

  • Click in the second Value cell to enter the parameter to set the last name to display using the following syntax: --context_param lastname=Ford.

  • Select tLogRow and click the Component tab to display the component view.

  • Set the Basic settings for the tLogRow component to display the output data in a tabular mode. For more information, see tLogRow.

  • Save your Job and press F6 to execute it.

The system generates three columns with the current date, first name, and last name and displays them onto the log console in a tabular mode.