Before scheduling the execution of your artifacts - 7.3

Talend Software Development Life Cycle Best Practices Guide

English (United States)
Talend Big Data
Talend Data Fabric
Talend Data Integration
Talend Data Management Platform
Talend Data Services Platform
Talend ESB
Talend MDM Platform
Talend Administration Center
Talend Artifact Repository
Talend CommandLine
Talend JobServer
Talend Studio
Administration and Monitoring
Design and Development
There are several prerequisites you should ensure before you start scheduling automatic artifact executions and deployment.
  • A Continuous Integration server/service is properly set up (here, Jenkins or Azure DevOps). For more information, see the Jenkins website and the Azure DevOps website.
  • You have previously created Test Cases in your project that you want to execute automatically. For more information, see Designing Tests.

    Note: When building and deploying your project, Test Cases will be generated as JUnit files and thus will be built before the packaging Maven phase.
  • The external libraries stored in Nexus that are needed to execute your Jobs have been properly installed in this project and your Nexus instance is started.

    If you are using Talend Administration Center, you have configured it to retrieve the external modules from Nexus. See the Talend Administration Center User Guide for more information.

  • You have a Docker environment installed and set up to push Docker images to your Docker registry. For more information, see the Docker website. Note that you can install a Docker container registry inside your Azure
  • If you are using Jenkins, you also need to check that a version superior or equal to Apache Maven 3 is installed in your server. For more information, see the Apache Maven website.

  • If you are using CI to build artifacts that require the Talend Data Mapper X12 HIPAA or HL7v2 packages, you need to:
    • extract the archive into <folder>/org.talend.transform.standard.p2.repository-A.B.C
    • install these packages in Talend CommandLine prior to building the Jobs using this command:
      Talend-Studio-win-x86_64.exe -nosplash -consoleLog -application org.eclipse.equinox.p2.director 
      -repository file:///<folder>/org.talend.transform.standard.p2.repository-A.B.C 
  • If you are using reference projects, you need to set up your Git project repository with this hierarchy and copy (not move) every project to the ci-workspace folder. This will allow you to display Git information (author, commit ID, commit date) in Talend Cloud Management Console when publishing artifacts using Continuous Integration builds.
        |- .repositories
             |- <any_unique_name>
                  |- .git
                  |- MAINPROJ1
             |- <any_unique_name>
                  |- .git
                  |- REFPROJ1
             |- <any_unique_name>
                  |- .git
                  |- REFPROJ2
                  |- REFPROJ3
        |- MAINPROJ1
        |- REFPROJ1
        |- REFPROJ2 
        |- REFPROJ3
Warning: The following prerequisites only apply to users who have the P2 update site archive, which is provided by Talend.
To be able to automatically download the Talend CommandLine at build time, you need to host the archive provided by Talend on your local http server. For example, you can:
  • extract the archive in the webapp folder of Tomcat and rename it P2.
  • start Tomcat.
  • point to http://localhost:8080/P2/ when defining the -Dupdatesite.path parameter at build time. Replace <localhost> with the IP address or the hostname of the web server if the web browser IP is different from the machine you are on and replace <8080> with the Tomcat port you use.

To summarize, you need the following applications: a Continuous Integration server or service (here, Azure or Jenkins), Git, an artifact repository (Nexus, Artifactory), Docker and a server to host the P2 update site where the Talend CommandLine as well as other plugins necessary to build your artifacts are stored.