Filter the execution of your project on selected artifacts - Cloud

Talend Software Development Life Cycle for Cloud - Best Practices Guide

EnrichVersion
Cloud
EnrichProdName
Talend Cloud
EnrichPlatform
Talend Artifact Repository
Talend Management Console
Talend Studio
task
Administration and Monitoring
Deployment
Design and Development
To filter the execution of your project items to build only the artifact(s) you want, you need to declare the filter in the Maven parameters entered when configuring the build project which generates your project sources on the Continuous Integration server.

Before you begin

  • You have created the build pipeline on your Continuous Integration server, Jenkins in this example. See Creating the Jenkins pipeline.
  • You know the path to the pom files of the artifacts you want to execute (Standard Jobs, Big Data Jobs, Routes, etc.). These files are available in the Studio workspace, in the workspace/<my_project>/poms directory.
    Tip:
    • To filter on Standard Jobs, the path to be used will be jobs/process/jobName_version
    • To filter on Big Data Jobs, the path to be used will be jobs/process_mr/jobName_version
    • To filter on Routes, the path to be used will be jobs/routes/routeName_version
    • To filter on Data Services, the path to be used will be jobs/services/serviceName_version

Procedure

  1. Open the configuration page of the pipeline.
  2. In the Script area, edit the Maven commands and parameters that allow you to filter the Jobs according to your needs:
    • -pl to generate a specific artifact (Job and Test, Route, etc.) rather than every artifact located in the project.
      Example:
      -pl jobs/process/job_feature956_0.1
      to deploy only the job_feature956 Standard Job with a 0.1 version.
      Tip: To build several artifacts, separate their names with commas.
    • -am (if the artifact has been specified with -pl) to build simultaneously the child dependencies of a specific artifact located in a project.
      Example:
      -pl jobs/process/jobparent_feature950_0.1 -am

      to deploy the parent Job named jobparent_feature950 with a 0.1 version as well as its child Job (jobchild_feature950_0.1) and sub-child Job (jobsubchild_feature950_0.1).

    • -amd (if the project has been specified with -pl) to build simultaneously the parent dependencies of a specific artifact located in a project.

      Example:

      -pl jobs/process/jobsubchild_feature950_0.1 -amd
      to generate the sub-child Job named jobsubchild_feature950 with a 0.1 version as well as the Jobs that depend on this one, jobchild_feature950_0.1) and jobparent_feature950).
      Note: As the -amd command builds all modules that depend on the specified Job, all the dependencies needed to generate this Job, such as routines, need to be pre-generated.

    Example

    Example of Maven command used to clean the project build directory (deleting old files), launch the generation of the route_feature950 Route and its dependencies and deploy the packages on the Nexus Artifact repository you have defined:

    -fae -e -pl jobs/routes/route_feature950_0.1 -amd

    The -amd parameter is mandatory to publish a Route as the dependencies of the Route need to be generated in order to publish it.

    Example of Maven command used to clean the project build directory (deleting old files), launch the generation of the parent Job named parent along with is dependencies (child jobs) and deploy these packaged artifacts on Talend Cloud Management Console:

    -Pcloud-publisher -pl jobs/process/parent_0.1 -am

    Example of Maven command used to take the generated project and package the Java code of all artifacts inside this project as a Docker image.

    -Pdocker package -fae -e
  3. Save your changes and close the configuration page.

Results

When you execute your project on your Continuous Integration server, the filter will be applied and only the artifacts you have filtered will be generated and executed.