Continuous delivery: Automatic creation of tasks in Talend Administration Center - 7.3

Version
7.3
Language
English (United States)
Product
Talend Big Data
Talend Big Data Platform
Talend Data Fabric
Talend Data Integration
Talend Data Services Platform
Talend ESB
Talend MDM Platform
Talend Real-Time Big Data Platform
Module
Talend Administration Center
Content
Design and Development

Continuous delivery: Automatic creation of tasks in Talend Administration Center

This article explains how to use Talend MetaServlet API from a Jenkins Job to automatically create tasks in Talend Administration Center.

Overview

Continuous delivery occurs when developers frequently hand off new code to the quality assurance (QA) and operations teams for testing. It usually involves a production-like staging area, and there is often a time lag between a release and when it is reviewed, when changes are manually accepted and when the new code is released to production.

Within Talend, the goal of continuous delivery is to use Jenkins to automate the creation of Talend Administration Center tasks in the Job Conductor and deploy them on the Job Servers along with the option of configuring Triggers (Simple, Cron, File-based) for scheduling.

This article describe one of the ways to build this Jenkins Job to perform continuous delivery. However, based on the SDLC process defined within your project group, the design and implementation can differ.

For more information on MetaServlet, see Metaservlet API.

For more information on the options available under MetaServlet, see Metaservlet API commands.

For a recorded expert session on MetaServlet, see Talend Virtual CSM.

Prerequisites

  • The Continuous Integration (CI) implementation is already setup:
    • The Jobs which are to be promoted to Testing and Pre-Production environments should be published to the Releases folder of the Nexus Artifacts Repository.
    • The necessary software, such as Jenkins, Maven, Talend Administration Center, Git and Nexus are setup and running.
    • The Jenkins Job is configured to build from Git, generate source files using CI-Builder, build the artifacts, test using Maven and publish to Nexus.
Note: If the CI implementation is not done, you can manually publish to the release repository of Nexus from Studio, but this is not recommended. For more information on how to setup CI, see Continuous Integration with Talend CI Builder 6.3.

Implementing the configuration file and groovy script

Implement continuous delivery with a configuration file and a groovy script.

To implement continuous delivery, you need to implement two files:

  • a configuration file that describes the list of tasks that are identified to be deployed into the concerned testing or other environments
  • a groovy script that uses this task description file to create the Base64 encoded message needed to invoke the MetaServlet API to create all the tasks in the Talend Administration Center portal

The configuration file provides the required parameters for the MestaServlet APIs to manage the task (create, update, deploy) in a text format that a script file can parse and generate the necessary Metaservlet request API. The parameters are used to create Triggers for FileBased, Cron or Simple types. You can provide these parameters in different formats: txt, CSV, JSON.

Procedure

  1. Create a file to provide the required parameters for the MetaServlet APIs to manage the task in Talend Administration Center.
    Use the file attached, task_list.json, as a starting point. JSON has been selected for its native support in Groovy.
  2. Structure the file layout to match the MetaServlet request parameters.
    For more information, see the appendix for the MetaServlet API.
  3. Save the file in a dedicated location, Git repository in this example.
    The Jenkins Job will check out this file.
  4. Install the Config File Provider Plugin to be able to create a managed file for the script.
  5. Create a managed file for Groovy Script as follows.

    The overall logic of the script is described below.

  6. Go to Global Tool Configuration and perform a Groovy installation.

    A Groovy installation is required to execute the Groovy script. You can perform a Groovy installation manually or on demand by Jenkins.

Using a Freestyle Jenkins Job

Create and run a Jenkins Job for continuous delivery.

Procedure

  1. Click New Item to add a new Jenkins Job.
  2. Select a Freestyle project and update the name accordingly.
  3. Set up the Source Code Management (SCM) repository where the JSON configuration file is located.