Before scheduling the execution of your artifacts - 8.0

Talend Software Development Life Cycle Best Practices Guide

Version
8.0
Language
English
Product
Talend Big Data
Talend Data Fabric
Talend Data Integration
Talend Data Management Platform
Talend Data Services Platform
Talend ESB
Talend MDM Platform
Module
Talend Administration Center
Talend Artifact Repository
Talend CommandLine
Talend JobServer
Talend Studio
Content
Administration and Monitoring
Deployment
Design and Development
There are several prerequisites you should ensure before you start scheduling automatic artifact executions and deployment.
  • A Continuous Integration server/service is properly set up (here, Jenkins or Azure DevOps). For more information, see the Jenkins website and the Azure DevOps website.
  • You have previously created Test Cases in your project that you want to execute automatically. For more information, see Designing Tests.

    Note: When building and deploying your project, Test Cases will be generated as JUnit files and thus will be built before the packaging Maven phase.
  • The external libraries that are needed to execute your Jobs have been properly installed in this project and your artifact repository is started.

  • You have a Docker environment installed and set up to push Docker images to your Docker registry. For more information, see the Docker website. Note that you can install a Docker container registry inside your Azure
  • If you are using Jenkins, you also need to check that a version superior or equal to Apache Maven 3 is installed in your server. For more information, see the Apache Maven website.

  • In order to display Git information (author, commit ID, commit date) in Talend Cloud Management Console when publishing artifacts using Continuous Integration builds, you need to set up your Git project repository with this hierarchy and copy (not move) every project to the ci-workspace folder.
    ci-workspace
        |- .repositories
             |- <any_unique_name>
                  |- .git
                  |- MAINPROJ1
             |- <any_unique_name>
                  |- .git
                  |- REFPROJ1
             |- <any_unique_name>
                  |- .git
                  |- REFPROJ2
                  |- REFPROJ3
        |- MAINPROJ1
        |- REFPROJ1
        |- REFPROJ2 
        |- REFPROJ3
  • If you are using CI to build artifacts that require the Talend Data Mapper X12 HIPAA or HL7v2 packages, you need to:
    • extract the Talend_Full_Studio_p2_repository--YYYYYYYY_YYYY-VA.B.C.zip archive into <folder>/org.talend.transform.standard.p2.repository-A.B.C
    • install these packages in Talend CommandLine prior to building the Jobs using this command:
      Talend-Studio-win-x86_64.exe -nosplash -consoleLog -application org.eclipse.equinox.p2.director 
      -repository file:///<folder>/org.talend.transform.standard.p2.repository-A.B.C 
      -installIU org.talend.transform.standard.x12.hipaa.5010.feature.feature.group
Warning: The following prerequisites only apply to users who have the P2 update site archive, which is provided by Talend.

To download the Talend CommandLine automatically at build time, you need to host the Talend_Full_Studio_p2_repository-YYYYMMDD_HHmm-VA.B.C.zip archive provided by Talend on your local HTTP server. For more information, see Setting up update repositories by hosting them.

To summarize, you need the following applications: a Continuous Integration server or service (here, Azure or Jenkins), Git, an artifact repository (Nexus, Artifactory), Docker and a server to host the P2 update site where the Talend CommandLine as well as other plugins necessary to build your artifacts are stored.