Before scheduling the execution of your artifacts - Cloud - 8.0

Talend Software Development Life Cycle Best Practices Guide

Version
Cloud
8.0
Language
English
Product
Talend Big Data
Talend Big Data Platform
Talend Cloud
Talend Data Fabric
Talend Data Integration
Talend Data Management Platform
Talend Data Services Platform
Talend ESB
Talend MDM Platform
Talend Real-Time Big Data Platform
Module
Talend Administration Center
Talend Artifact Repository
Talend CommandLine
Talend JobServer
Talend Management Console
Talend Remote Engine
Talend Studio
Content
Administration and Monitoring
Deployment
Design and Development
Last publication date
2024-02-29
There are several prerequisites you should ensure before you start scheduling automatic artifact executions and deployment.
  • A Continuous Integration server/service is properly set up (here, Jenkins or Azure DevOps). For more information, see the Jenkins website and the Azure DevOps website.
  • You have previously created Test Cases in your project that you want to execute automatically. For more information, see Designing Tests.

    Note: When building and deploying your project, Test Cases will be generated as JUnit files and thus will be built before the packaging Maven phase.
  • The external libraries that are needed to execute your Jobs have been properly installed in this project and your artifact repository is started.

  • You have a Docker environment installed and set up to push Docker images to your Docker registry. For more information, see the Docker website. Note that you can install a Docker container registry inside your Azure DevOps account.
  • If you are using Jenkins, you also need to check that a version superior or equal to Apache Maven 3 is installed in your server. For more information, see the Apache Maven website.

  • In order to display Git information (author, commit ID, commit date) in Talend Management Console when publishing artifacts using Continuous Integration builds, you need to set up your Git project repository with this hierarchy and copy (not move) every project to the ci-workspace folder.
    ci-workspace
        |- .repositories
             |- <any_unique_name>
                  |- .git
                  |- MAINPROJ1
             |- <any_unique_name>
                  |- .git
                  |- REFPROJ1
             |- <any_unique_name>
                  |- .git
                  |- REFPROJ2
                  |- REFPROJ3
        |- MAINPROJ1
        |- REFPROJ1
        |- REFPROJ2 
        |- REFPROJ3
  • If you are using CI to build artifacts that require the Talend Data Mapper X12 HIPAA or HL7v2 packages, you need to:
    • extract the Talend_Full_Studio_p2_repository--YYYYYYYY_YYYY-VA.B.C.zip archive into <folder>/org.talend.transform.standard.p2.repository-A.B.C
    • install these packages in Talend CommandLine prior to building the Jobs using this command:
      Talend-Studio-win-x86_64.exe -nosplash -consoleLog -application org.eclipse.equinox.p2.director 
      -repository file:///<folder>/org.talend.transform.standard.p2.repository-A.B.C 
      -installIU org.talend.transform.standard.x12.hipaa.5010.feature.feature.group
Warning: The following prerequisites applies to users who have the P2 update site archive, which is provided by Talend (recommended).

Talend CommandLine is automatically downloaded from the Talend update site and installed during build time to generate and execute your project artifacts. For more information, read Talend CI zero install workflow.

To download the Talend CommandLine automatically at build time, you need to host the Talend_Full_Studio_p2_repository-YYYYMMDD_HHmm-VA.B.C.zip archive provided by Talend on your local HTTP server. For more information, see Setting up update repositories by hosting them.

Talend does not recommend running more than one Talend CommandLine at any given time on a machine. The easiest way to prevent this is to limit the number of executors on the servers to 1.

To summarize, you need the following applications: a Continuous Integration server or service, Git, and a server to host the P2 update site where the Talend CommandLine, as well as other plugins necessary to build your artifacts, are stored. Depending on where you want to deploy your artifacts, you may need access to an artifact repository, Docker, or Talend Cloud.