Adapting the YAML parameters to your environment - Cloud

Talend Software Development Life Cycle for Cloud - Best Practices Guide

EnrichVersion
Cloud
EnrichProdName
Talend Cloud
EnrichPlatform
Talend Artifact Repository
Talend Management Console
Talend Studio
task
Administration and Monitoring
Deployment
Design and Development

Configure the pipeline script provided by Talend to generate the project items according to the Maven phase you want to achieve and the repository you want to deploy your artifacts to.

Some parameters and credentials come from the secure files and variables you have stored in Azure DevOps. For more information about the Talend Maven parameters you can use when configuring your pipeline, see Talend custom Maven build options.

Procedure

  1. Set your own values for the environment variables defined at the top of the script (project name, Job name/version/type, repository URL, etc.).
    Tip: Look for the XXX references in the azure-pipeline-*.yml script provided by Talend as an example and replace them with values corresponding to your environment.

    Example

    Example of a project named CICD, a Job named job_feature956 with a 0.1 version and a Docker image named cicd4talend:
    variables:
    - group: Talend Variables
    - name: project_name
      value: 'CICD'
    - name: artifact_name
      value: 'job_feature956'
    - name: artifact_version
      value: '0.1'
    - name: docker_image_name
      value: 'cicd4talend'
    Warning: Syntax requirements:
    • The value of the job_name variable must be in upper case otherwise it might cause build failure. If you used special characters in the project name or have any doubts on its final syntax you can check it in the parent pom.xml file of your project (talend.project.name variable in <project>/poms/pom.xml).
    • The value of the artifact_name variable must be in lower case even if the original name of your Job/Route/Service contained upper case as all artifacts are in lower case in the .pom files required to launch the build.
    • Do not add manually the "-SNAPSHOT" suffix to the version value. Snapshots and releases are decoupled from the version itself and must only be handled via the Studio preferences, see Changing the deployment version of each artifact at once.
  2. Edit the artifact repository URL stated in the mavenOptions line. The repository defined in the default script provided by Talend is the Nexus snapshots repository.

    Example

    -DaltDeploymentRepository=releases::default::http://nexus:8081/repository/releases/

    allows you to deploy artifacts into a Nexus releases repository (for stable release artifacts in a Production environment).

    Note: If you want to deploy Jobs that use custom components, you need to use the -Dcomponents.* options as described in Talend custom Maven build options.
  3. Edit the Talend Cloud workspace and environment stated in the mavenOptions line. The default workspace and environment defined in the default script provided by Talend are the default workspace and the Personal environment.

    Example

    -Dservice.url=$(talend_cloud_url) -Dcloud.publisher.environment=PROD -context=Prod
    -Dcloud.publisher.workspace=CI_WORKSPACE -Dcloud.publisher.screenshot=true

    allows you to deploy artifacts (with screenshots) into a Talend Cloud account inside a workspace named CI_WORKSPACE on a PROD environment while changing the context environment to Prod (using --context). This environment must have been defined previously in Talend Studio, however if you did not define any context parameters you can use --context_param to define them at build time.

    Note: If you have enabled Single Sign-On (SSO) to connect to Talend Cloud, you need to add the -Dcloud.token=<MyAccessToken> parameter to these commands.
  4. To define the Maven phase you want to achieve: make sure the phase and build profiles stated in the options and goals lines are the one you want to perform. The default goal defined in the default script provided by Talend is deploy.
    For more information on the Maven phases, see the Maven documentation.
    Tip: All build profiles (-Pdocker, -Pnexus, etc.) are listed in the parent pom.xml file of your project. Some default parameters, like the docker image name or the Open JDK name, can also be overwritten in this file.

    Example

    options: '--settings $(settings_xml.secureFilePath) -Pnexus -pl jobs/routes/$(artifact_name)_$(artifact_version) -am'
    goals: 'clean deploy'

    allows you to take the packaged route as a jar or .zip files and deploy it to the Nexus repository of your choice.

    Example

    options: '--settings $(settings_xml.secureFilePath) -X -Pdocker -pl jobs/process/$(artifact_name)_$(artifact_version) -am -Dtalend.docker.name=$(docker_image_name)'
    goals: 'deploy'

    allows you to take the compiled code, package it as a Docker image and deploy it to the registry of your choice.

    Example

    options: '--settings $(settings_xml.secureFilePath) -Pcloud-publisher -pl jobs/process/$(artifact_name)_$(artifact_version) -am'
    goals: 'deploy'

    allows you to copy the final packaged artifacts to the Talend Cloud account of your choice.

    Note: The artifact published to Talend Cloud cannot be larger than 400MB.
  5. Save your changes. It will automatically launch the execution of your build pipeline.