How to build Jobs - 6.2

Talend MDM Platform Studio User Guide

EnrichVersion
6.2
EnrichProdName
Talend MDM Platform
task
Data Governance
Data Quality and Preparation
Design and Development
EnrichPlatform
Talend Studio

The Build Job feature allows you to deploy and execute a Job on any server, independent of Talend Studio.

By executing build scripts generated from the templates defined in Project Settings, the Build Job feature adds all of the files required to execute the Job to an archive, including the .bat and .sh along with any context-parameter files or other related files.

Note

Your Talend Studio provides a set of default build script templates. You can customize those templates to meet your actual needs. For more information, see Customizing Maven build script templates.

By default, when a Job is built, all the required jars are included in the .bat or .sh command. For a complex Job that involves many Jars, the number of characters in the batch command may exceed the limitation of command length on certain operating systems. To avoid failure of running the batch command due to this limitation, before building your Job, go to Window > Preferences, select Talend > Import/Export, and then select the Add classpath jar in exported jobs check box to wrap the Jars in a classpath.jar file added to the built Job.

Warning

The above-mentioned option is incompatible with JobServer. If your built Job will be deployed and executed in Talend Administration Center, make sure to clear the check box before building your Job.

Note that the SOAP data service Job including the tESBProviderRequest component that implements a service operation can not be built independently. The Job has to be built with the service it is assigned to. When you export a SOAP service, all the data service Jobs that implement the service operations are built in the service archive file as well.

To build Jobs, complete the following:

  1. In the Repository tree view, right-click the Job you want to build, and select Build Job to open the [Build Job] dialog box.

    Note

    You can show/hide a tree view of all created Jobs in Talend Studio directly from the [Build Job] dialog box by clicking the and the buttons respectively. The Jobs you earlier selected in the Studio tree view display with selected check boxes. This accessibility helps to modify the selected items to be exported directly from the dialog box without having to close it and go back to the Repository tree view in Talend Studio to do that.

  2. In the To archive file field, browse to the directory where you want to save your built Job.

  3. From the Select the Job version area, select the version number of the Job you want to build if you have created more than one version of the Job.

  4. Select the Build Type from the list between Standalone Job, Axis Webservice (WAR), Axis Webservice (Zip) and OSGI Bundle For ESB.

    Note that data service Jobs that include the tRESTRequest component can only be built as OSGI Bundle For ESB. If the data service Job includes the tRESTClient or tESBConsumer component, and none of the Service Registry, Service Locator or Service Activity Monitor is enabled in the component, the data service Job can be built as OSGI Bundle For ESB or Standalone Job. With the Service Registry, Service Locator or Service Activity Monitor enabled, the data service Job including the tRESTClient or tESBConsumer component can only be built as OSGI Bundle For ESB.

  5. Select the Extract the zip file check box if you want the archive file to be automatically extracted in the target directory.

  6. In the Options area, select a build type between Binaries and Sources (Maven) and the file type(s) you want to add to the archive file. The check boxes corresponding to the file types necessary for the execution of the Job are selected by default. You can clear these check boxes depending on what you want to build.

    Option

    Description

    Binaries / Sources (Maven)

    Select Binaries from the list box to build your Job as an executable Job.

    Select Sources (Maven) to build the sources of your Job and include in the archive file the Maven build scripts generated from the templates defined in Project Settings so that you can rebuild your Job in an Apache Maven system.

    Shell launcher

    Select this check box to export the .bat and/or .sh files necessary to launch the built Job.

    • All: exports the .bat and .sh files.

    • Unix exports the .sh file.

    • Windows exports the .bat file.

    Context scripts

    Select this check box to export ALL context parameters files and not just those you select in the corresponding list.

    Note

    To export only one context, select the context that fits your needs from the Context scripts list, including the .bat or .sh files holding the appropriate context parameters. Then you can, if you wish, edit the .bat and .sh files to manually modify the context type.

    Apply to children Select this check box if you want to apply the context selected from the list to all child Jobs.

    Custom log4j level

    Select this check box to activate the Log4j output level list and select an output level for the built Job.

    For more information on Log4j settings, see Log4j settings.

    Items

    Select this check box to export the sources used by the Job during its execution including the .item and .properties files, Java and Talend sources.

    Note

    If you select the Items or Source files check box, you can reuse the built Job in a Talend Studio installed on another machine. These source files are only used in Talend Studio.

    Execute tests

    Select this check box to execute the test case(s) of the Job, if any, when building the Job, and include the test report files in the sunfire-reports folder of the build archive.

    This check box is available only when the Binaries option is selected.

    For more information on how to create test cases, see Testing Jobs and Services using test cases.

    Add test sources

    Select this check box to include the sources of the test case(s) of the Job, if any, in the build archive.

    This check box is available only when the Sources (Maven) option is selected.

    For more information on how to create test cases, see Testing Jobs and Services using test cases.

    Java sources

    Select this check box to export the .java file holding Java classes generated by the Job when designing it.

    This check box is available only when the Binaries option is selected.

    Include libraries

    Select this check box to include dependencies of the Job in the build archive.

    This check box is available only when the Sources (Maven) option is selected.

  7. Click the Override parameters' values button, if necessary.

    In the window which opens you can update, add or remove context parameters and values of the Job context you selected in the list.

  8. Click Finish to validate your changes, complete the build operation and close the dialog box.

A zipped file for the Jobs is created in the defined place.

Note

If the Job to be built calls a user routine that contains one or more extra Java classes in parallel with the public class named the same as the user routine, the extra class or classes will not be included in the exported file. To export such classes, you need to include them within the class with the routine name as inner classes. For more information about user routines, see Managing user routines. For more information about classes and inner classes, see relevant Java manuals.

How to build a Job as a standalone Job

In the case of a Plain Old Java Object export, if you want to reuse the Job in Talend Studio installed on another machine, make sure you selected the Items check box. These source files (.item and .properties) are only needed within Talend Studio.

If you want to include an Ant or Maven script for each built Job, select the Add build script check boxes, and then select Ant or Maven option button.

Select a context from the list when offered. Then once you click the Override parameters' values button below the Context scripts check box, the opened window will list all of the parameters of the selected context. In this window, you can configure the selected context as needs.

All contexts parameter files are exported along in addition to the one selected in the list.

Note

After being exported, the context selection information is stored in the .bat or .sh file and the context settings are stored in the context .properties file.

How to build a Job as a Webservice

In the [Build Job] dialog box, you can change the build type in order to build the Job selection as Webservice archive.

Select the type of archive you want to use in your Web application.

Archive type

Description

WAR

The options are read-only. Indeed, the WAR archive generated includes all configuration files necessary for the execution or deployment from the Web application.

ZIP

All options are available. In the case the files of your Web application config are all set, you have the possibility to only set the Context parameters if relevant and export only the Classes into the archive.

Once the archive is produced, place the WAR or the relevant Class from the ZIP (or unzipped files) into the relevant location, of your Web application server.

The URL to be used to deploy the Job, typically reads as follow:

http://localhost:8080/Webappname/services/JobName?method=runJob&args=null

where the parameters stand as follow:

URL parameters

Description

http://localhost:8080/

Type in the Webapp host and port.

/Webappname/

Type in the actual name of your web application.

/services/

Type in "services" as the standard call term for web services.

/JobName

Type in the exact name of the Job you want to execute.

?method=runJob&args=null

The method is RunJob to execute the Job.

The call return from the Web application is 0 when there is no error and different from 0 in case of error. For a real-life example of creating and building a Job as a Webservice and calling the built Job from a browser, see An example of building a Job as a Web service.

The tBufferOutput component was especially designed for this type of deployment. For more information regarding this component, see Talend Components Reference Guide.

An example of building a Job as a Web service

This scenario describes first a simple Job that creates a .txt file and writes in it the current date along with first and last names. Secondly, it shows how to build this Job as a Webservice. And finally, it calls the Job built as a Webservice from a browser. The built Job as a Webservice will simply return the "return code" given by the operating system.

Creating the Job:

  1. Drop the following components from the Palette onto the design workspace: tFixedFlowInput and tFileOutputDelimited.

  2. Connect tFixedFlowInput to tFileOutputDelimited using a Row > Main link.

  3. In the design workspace, select tFixedFlowInput, and click the Component tab to define the basic settings for tFixedFlowInput.

  4. Set the Schema to Built-In and click the [...] button next to Edit Schema to describe the data structure you want to create from internal variables. In this scenario, the schema is made of three columns, now, firstname, and lastname.

  5. Click the [+] button to add the three parameter lines and define your variables, and then click OK to close the dialog box and accept propagating the changes when prompted by the system.

    The three defined columns display in the Values table of the Basic settings view of tFixedFlowInput.

  6. In the Value cell of each of the three defined columns, press Ctrl+Space to access the global variable list, and select TalendDate.getCurrentDate(), talendDatagenerator.getFirstName, and talendDataGenerator.getLastName for the now, firstname, and lastname columns respectively.

  7. In the Number of rows field, enter the number of lines to be generated.

  8. In the design workspace, select tFileOutputDelimited, click the Component tab for tFileOutputDelimited, and browse to the output file to set its path in the File name field. Define other properties as needed.

If you press F6 to execute the Job, three rows holding the current date and first and last names will be written to the set output file.

Building the Job as a Webservice:

  1. In the Repository tree view, right-click the above created Job and select Build Job. The [Build Job] dialog box appears.

  2. Click the Browse... button to select a directory to archive your Job in.

  3. In the Job Version area, select the version of the Job you want to build as a web service.

  4. In the Build type area, select the build type you want to use in your Web application (WAR in this example) and click Finish. The [Build Job] dialog box disappears.

  5. Copy the War folder and paste it in the Tomcat webapp directory.

Calling the Job from a browser:

  1. Type the following URL into your browser: http://localhost:8080//export_job/services/export_job2?method=runJob where "export_job" is the name of the webapp directory deployed in Tomcat and "export_job2" is the name of the Job.

  2. Click Enter to execute the Job from your browser.

    The return code from the Web application is 0 when there is no error and 1 if an error occurs.

    For a real-life example of creating and building a Job as a Webservices using the tBufferOutput component, see the tBufferOutput component in Talend Components Reference Guide.

How to build a Job as an OSGI Bundle For ESB

In the [Build Job] dialog box, you can change the build type in order to build the Job selection as an OSGI Bundle in order to deploy your Job in Talend ESB Container.

  1. In the Job Version area, select the version number of the Job you want to build if you have created more than one version of the Job.

  2. In the Build type area, select OSGI Bundle For ESB to build your Job as an OSGI Bundle.

    The extension of your build automatically change to .jar as it is what Talend ESB Container is expecting.

  3. If you want to rebuild the built Job into your own JAR with Maven, select the Add maven script check box in the Options area to include the required Maven script in the target archive, which is a .zip file in this case.

  4. Click the Browse... button to specify the folder in which building your Job.

  5. Click Finish to build it.