Before configuring the execution of your Jobs - 7.0

Talend Software Development Life Cycle Best Practices Guide

author
Talend Documentation Team
EnrichVersion
7.0
EnrichProdName
Talend Big Data
Talend Data Fabric
Talend Data Integration
Talend Data Management Platform
Talend Data Services Platform
Talend ESB
Talend MDM Platform
task
Administration and Monitoring
Deployment
Design and Development
EnrichPlatform
Talend Administration Center
Talend Artifact Repository
Talend CommandLine
Talend JobServer
Talend Studio
There are several prerequisites you should ensure before you start scheduling automatic test executions with the Talend CI Builder.
  • A version superior or equal to Apache Maven 3 is installed in your server. For more information, see the Apache Maven website.

  • The Jenkins Continuous Integration server is properly installed and set up. For more information, see the Jenkins website.

  • You have a Talend CommandLine application that will only be used for Continuous Integration purposes (cannot be the same as the Talend CommandLine used for generic actions such as publishing items from Talend Studio to Talend Administration Center, generating execution tasks in Talend Administration Center, etc.).

  • You have previously created Test Cases in your SVN or Git project that you want to execute automatically. For more information, see Designing Tests.

  • The external libraries stored in Nexus that are needed to execute your Jobs have been properly installed in this project and your Nexus instance is started.

    For more information, see the article Installing external modules.

    If you are using Talend Administration Center, you have configured it to retrieve the external modules from Nexus. See the Talend Administration Center User Guide for more information.

  • You have started Talend CommandLine at least once in order to initialize its default Maven repository, and closed it right after.

    Edit the <commandlinePath>/configuration/maven_user_settings.xml file and add the connection information to the Nexus repositories:

    <?xml version="1.0" encoding="UTF-8"?>
    <settings xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.1.0 http://maven.apache.org/xsd/settings-1.1.0.xsd" xmlns="http://maven.apache.org/SETTINGS/1.1.0"
        xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
      <localRepository>/<.m2Path>/repository</localRepository>
    
    <servers>
        <!-- credentials to access the default releases/snapshots repositories -->
        <server>
            <id>releases</id>
            <username>admin</username>
            <password>Talend123</password>
        </server>
        <server>
            <id>snapshots</id>
            <username>admin</username>
            <password>Talend123</password>
        </server>
        <!-- credentials to access the repositories holding external jars -->  
        <server>
            <id>talend-custom-libs-release</id>
            <username>admin</username>
            <password>Talend123</password>
        </server>
        <server>
            <id>talend-custom-libs-snapshot</id>
            <username>admin</username>
            <password>Talend123</password>
        </server>
        
        <!-- credentials to access the repositories holding maven plugins -->
        <server> <!-- central (as proxy) -->
            <id>central</id>
            <username>admin</username>
            <password>Talend123</password>
        </server>
        <server>
            <id>thirdparty</id>
            <username>admin</username>
            <password>Talend123</password>
        </server>
        
    </servers>
    
    <mirrors/>
    <proxies/> <!-- http proxies, not maven proxy repositories -->
    
    <profiles>
        <profile>
            <id>talend-ci</id>
            <repositories>
                <repository>
                    <id>central</id>
                    <name>central</name>
                    <url>http://localhost:8081/repository/maven-central/</url>
                    <layout>default</layout>
                </repository>
                <repository>
                    <id>talend-custom-libs-release</id>
                    <name>talend-custom-libs-release</name>
                    <url>http://localhost:8081/repository/talend-custom-libs-release</url>
                    <layout>default</layout>
                    <releases>
                        <enabled>true</enabled>
                    </releases>
                    <snapshots>
                        <enabled>false</enabled>
                    </snapshots>
                </repository>
                <repository>
                    <id>talend-custom-libs-snapshot</id>
                    <name>talend-custom-libs-snapshot</name>
                    <url>http://localhost:8081/repository/talend-custom-libs-snapshot</url>
                    <layout>default</layout>
                    <releases>
                        <enabled>false</enabled>
                    </releases>
                    <snapshots>
                        <enabled>true</enabled>
                    </snapshots>
                </repository>
            </repositories>
            
            <pluginRepositories>            
                <pluginRepository>
                    <id>central</id>
                    <name>central</name>
                    <url>http://localhost:8081/repository/maven-central/</url>
                    <layout>default</layout>
                </pluginRepository>
                <pluginRepository>
                    <id>thirdparty</id>
                    <name>thirdparty</name>
                    <url>http://localhost:8081/repository/thirdparty</url>
                    <layout>default</layout>
                </pluginRepository>
            </pluginRepositories>
        </profile>
    </profiles>
    
    <activeProfiles>
        <activeProfile>talend-ci</activeProfile>
    </activeProfiles>
    
    </settings>
  • For Big Data users with dynamic distributions only: add the following server and repository to the settings file in order to define the Nexus local repository as proxy, ensuring faster .jar file downloading:

    <server>
                            <id>dynamic-distrib</id>
                            <username>admin</username>
                            <password>admin123</password>
                            </server>
    <repository>
                            <id>dynamic-distrib</id>
                            <name>dynamic-distrib</name>
                            <url>http://localhost:8081/repository/dynamic-distrib/</url>
                            <layout>default</layout>
                            </repository>

    For this configuration to be complete, you need to have previously added the Talend proxy in the Talend Studio Project settings > General > Dynamic Distribution Settings menu.

    For more information, see the documentation about how to add the latest Big Data Platform dynamically.

To summarize, you need the following applications: Talend CommandLine, a Continuous Integration server (here, Jenkins), Maven, Nexus and Talend CI Builder.