Migrating from Talend CI version 6.x to 7.1 - 7.1

author
Talend Documentation Team
EnrichVersion
7.1
EnrichProdName
Talend Big Data
Talend Data Fabric
Talend Data Integration
Talend Data Management Platform
Talend Data Services Platform
Talend ESB
Talend MDM Platform
task
Installation and Upgrade
EnrichPlatform
Talend Administration Center
Talend Artifact Repository
Talend CommandLine
Talend JobServer
Talend Studio

Migrating from v6.x to v7.1

In version 7.1, several improvements have been made to automate and ease the process of building Talend Jobs and projects.

If you have used the Talend Continuous Integration features previously, you might want to compare the old and new processes.

Differences between building artifacts and projects in v6.x and v7.1

.
Table 1. Steps to reproduce to execute, publish and deploy your project items

Before 7.1

In version 7.1

Using Talend Administration Center:
  1. From the Job Conductor page, retrieve the task from the Publisher or directly from Nexus to schedule the Job executions through time-based triggers.
  2. From the Publisher page, create a task on the Job/Jobs/Project you want to publish on Nexus (not recommended since v7.0).
  • Recommended: using an external Continuous Integration server.
  • Alternative: using the Talend Studio Publish option or Talend CommandLine. For more information, see the Talend Studio User Guide.
  • Deprecated: using Talend Administration Center Publisher page

See Scheduling automatic executions in your build environment for more information.

Using Talend CI Builder combined with an execution server (Jenkins for example):
  1. Complete the prerequisites.

    Refer to the documentation provided for your version for more information.

  2. Install Talend CI Builder and upload it into the thirdparty Nexus repository that exists by default (before v7.0) or in the thirdparty Nexus repository you have created (in v7.0).

    Refer to the documentation provided for your version for more information.

  3. Depending on your version:
    • Before v7.0: Create three build projects on Jenkins: one build to get the sources and converts them to Java classes with the Talend CI Builder, one build to run all Jobs and Tests at once and one build to deploy project artifacts on Nexus.
    • In v7.0: Create one unique build project on Jenkins that will generate the sources, run alls Jobs and associated Tests and deploy them to the Nexus repository of your choice.

    Refer to the documentation provided for your version for more information.

  4. Configure these build projects:
    • link them to your Git/SVN project
    • decide how to generate sources, either locally (recommended) or remotely (script mode).

      Before v7.0, it is recommended to create a POM file holding Maven project information and instructions to generate the sources. In v7.0, POM files are automatically pre-generated for every project item. However you can change both the deployment identifier of the project (groupID) and the deployment version values from Talend Studio..

      Refer to the documentation provided for your version for more information.

    • specify all the parameters needed to build, including the filters to only build the project items you want using the <itemfilter> parameter (before v7.0) or the Maven compliant parameters (in v7.0).

      Example of filter applied to execute all Standard Jobs located in the subfolders with a name starting with Export under the MainProcess folder:

      -DitemFilter=(type=process)
      and(path%MainProcess/Export*)

      Refer to the documentation provided for your version for more information.

  5. Make sure your Talend CommandLine application points to the Jenkins workspace where your project sources are stored then build the three projects.

    Before v7.0: The packaged artifacts will be published on the Nexus repository specified in the POM file you have created.

    In v7.0: Build the Jenkins project with parameters to select in which Nexus repository the artifacts will be published.

    Refer to the documentation provided for your version for more information.

Using Talend CI Builder combined with an execution server (Jenkins for example):
  1. Complete the prerequisites.

    The customized Maven settings file are now stored in Jenkins.

  2. Starting from version 7.1 onwards, the Talend CI Builder is embedded in the Talend Studio local maven repository and thus does not require to be installed separately.
  3. Create one unique build pipeline on Jenkins that will generate the sources, run alls artifacts and associated Tests and deploy them to an artifact repository (Nexus, Artifactory) or a Docker container of your choice (new).

    In version 7.1, CI also applies to ESB (Routes and Data Services), however publishing ESB artifacts to Docker is not supported yet.

  4. Configure this build pipeline with a Groovy script that:
    • connects to your Git/SVN project
    • states how to generate sources (the remote server mode is deprecated).

      POM files are automatically pre-generated for every project item and you can still change both the deployment identifier of the project (groupID) and the deployment version values from Talend Studio.

      See Changing the deployment identifier of the project at once for more information.

    • specifies all the parameters needed to build and where to deploy the artifacts. You can use directly the Maven compliant parameters in the Jenkins build configuration to build the project items you want.

      Example of Maven command used to clean the project build directory (deleting old files), launch the generation of the Job named job_feature400 as well as its related Test cases and dependencies and package the Java code in an output file (.jar, .zip):

      mvn clean package jobs/process/job_feature400_0.1 -am

      See Filter the execution of your project on selected artifacts for more information.

      A new parameter, -Dcommandline.skip, is introduced in v7.1 to allow you to skip the source code generation of your artifacts provided that the sources have been previously generated via Talend CommandLine at least once.

      See Maven build options for more information.

  5. Make sure your artifact repository is started and the Talend CommandLine application points to the Jenkins workspace where your project sources are stored then run the Jenkins pipeline with the parameters defined in the pipeline script to generate and deploy your artifacts the way you want to in which Nexus repository the artifacts will be published.

    See Running the Jenkins pipeline for more information.