Building and Deploying - Cloud - 7.3

Talend Software Development Life Cycle Best Practices Guide
Version
Cloud
7.3
Language
English
Product
Talend Big Data
Talend Big Data Platform
Talend Cloud
Talend Data Fabric
Talend Data Integration
Talend Data Management Platform
Talend Data Services Platform
Talend ESB
Talend MDM Platform
Talend Real-Time Big Data Platform
Module
Talend Administration Center
Talend Artifact Repository
Talend CommandLine
Talend JobServer
Talend Management Console
Talend Remote Engine
Talend Studio
Content
Administration and Monitoring
Deployment
Design and Development
Last publication date
2024-02-08
Talend offers you several ways to publish your project artifacts to Talend Administration Center, Talend Cloud or artifact repositories (Nexus, Artifactory) and schedule their executions, and allows you to choose the one that suits best your needs.

  • Recommended: use an external Continuous Integration tool.

    Note: You can use other third-party Continuous Integration tools besides Azure DevOps and Jenkins. However, Talend only focuses on the configuration and pipeline script samples on Azure DevOps and Jenkins documented in these two sections, and the usage of any third-party Continuous Integration tools are out of Talend support scope.

  • Alternative: use the Talend Studio Publish, Publish to Cloud options or Talend CommandLine. For more information, see the Talend Studio User Guide.

  • ESB use case: For an example of Docker build with ESB artifacts (Data services and Routes Microservices), see Building Docker images for Data Services and Routes Microservices.

In a continuous integration environment, it is common practice to launch tests at every commit. By default, a new commit is made every time you save artifacts.

The following sections introduce two ways to automate the test executions using the Talend CommandLine and Talend Maven plugins to generate the sources, as well as a CI server (Azure and Jenkins) that use build pipelines to automate the execution of all project artifacts and to publish them to Talend Cloud or to an artifact repository or a Docker container.

Talend Continuous Integration is Maven-compliant, which means that you have the possibility to use the Talend Maven commands with any stable version of your Continuous Integration server (Jenkins, Bamboo, etc.).

In case you want to use your own Continuous Integration tool to schedule the execution and publishing of artifacts, Talend allows you to do so by providing you the Talend CI Maven plugins:
  • Talend CI Builder is a Maven plugin delivered by Talend that transforms the Talend Job sources to Java classes using the Talend CommandLine application, allowing you to execute your tests in your own company Java factory.
  • Talend Cloud Publisher is a Maven plugin delivered by Talend that allows you to publish the Jobs created from Talend Studio directly to Talend Cloud. It is activated by adding the -Pcloud-publisher profile parameter to your CI build.

Starting from version 7.1 onwards, Talend Maven plugins are embedded in the Talend Studio local Maven repository and installed at the first Talend Studio start-up and thus do not require to be installed separately.

Talend CI Builder may get upgraded upon the installation of a Talend Studio monthly update provided by Talend. You must use the Talend CI Builder version which is compatible to the Talend Studio version in your CI commands or pipeline scripts. To know the exact version of Talend CI Builder you must use, see the changelog corresponding to the Talend Studio monthly update you are using. For more information about the changelog for each Talend Studio monthly update, see Talend Release Notes.

If you encounter an exception on the artifact org.talend.ci:builder-maven-plugin:jar:7.3.x not being downloaded while using Talend CI Builder, and

  • if you are using version 7.3.10 or above, make sure the maven_settings.xml file points to the official plugin repository:
    <pluginRepository>
<id>central-talend</id>
<name>central-talend</name>
<url>https://talend-update.talend.com/nexus/content/repositories/libraries/</url>
<layout>default</layout>
</pluginRepository>
  • if you are using Nexus, you can deploy its new version automatically by logging in any project in Talend Studio.
  • if you are not using Nexus, you can install its new version to target maven repository manually.