Filtering the execution of your project on selected artifacts - Cloud - 8.0

Talend Software Development Life Cycle Best Practices Guide
Version
Cloud
8.0
Language
English
Product
Talend Big Data
Talend Big Data Platform
Talend Cloud
Talend Data Fabric
Talend Data Integration
Talend Data Management Platform
Talend Data Services Platform
Talend ESB
Talend MDM Platform
Talend Real-Time Big Data Platform
Module
Talend Administration Center
Talend Artifact Repository
Talend CommandLine
Talend JobServer
Talend Management Console
Talend Remote Engine
Talend Studio
Content
Administration and Monitoring
Deployment
Design and Development
Last publication date
2024-02-29
To filter the execution of your project items to build only the artifact(s) you want, you need to declare the filter in the Maven parameters entered when configuring the build project which generates your project sources on the Continuous Integration server.

Before you begin

  • You have created the build pipeline on your Continuous Integration server, Jenkins in this example. See Creating the Jenkins pipeline.
  • You know the path to the pom files of the artifacts you want to execute (Standard Jobs, Big Data Jobs, Routes, etc.). These files are available in the Talend Studio workspace, in the workspace/<my_project>/poms directory.
    Tip:
    • To filter on Standard Jobs, the path to be used will be jobs/process/jobName_version
    • To filter on Big Data Jobs, the path to be used will be jobs/process_mr/jobName_version
    • To filter on Routes, the path to be used will be jobs/routes/routeName_version
    • To filter on Data Services, the path to be used will be jobs/services/serviceName_version

Procedure

  1. From the Jenkins home page, select TalendSimplePipeline.
  2. Click Build with Parameters on the left panel to open the pipeline configuration page.
  3. In the JOBS_TO_BUILD area, edit the Maven commands and parameters that allow you to filter the Jobs according to your needs:
    • the default -pl command generates a specific artifact (Job and Test, Route, etc.) rather than every artifact located in the project.
      Example: 
      -pl jobs/process/job_feature956_0.1
      to deploy only the job_feature956 Standard Job with a 0.1 version.
      Tip: To build several artifacts, separate their names with commas.
    • -am (if the artifact has been specified with -pl) builds simultaneously the child dependencies of a specific artifact located in a project.
      Example:
      -pl jobs/process/jobparent_feature950_0.1 -am

      to deploy the parent Job named jobparent_feature950 with a 0.1 version as well as its child Job (jobchild_feature950_0.1) and sub-child Job (jobsubchild_feature950_0.1).

    • -amd (if the project has been specified with -pl) builds simultaneously the parent dependencies of a specific artifact located in a project.

      Example:

      -pl jobs/process/jobsubchild_feature950_0.1 -amd
      to generate the sub-child Job named jobsubchild_feature950 with a 0.1 version as well as the Jobs that depend on this one, jobchild_feature950_0.1 and jobparent_feature950.
      Note: As the -amd command builds all modules that depend on the specified Job, all the dependencies needed to generate this Job, such as routines, need to be pre-generated.

    Example

    Example of Maven command used to clean the project build directory (deleting old files), launch the generation of the route_feature950 Route and its dependencies and deploy the packages on the Nexus Artifact repository you have defined:

    -fae -e -pl jobs/routes/route_feature950_0.1 -amd

    The -amd parameter is mandatory to publish a Route as the dependencies of the Route need to be generated in order to publish it.

    Example of Maven command used to clean the project build directory (deleting old files), launch the generation of the parent Job named parent along with is dependencies (child jobs) and deploy these packaged artifacts on Talend Management Console:

    -Pcloud-publisher -pl jobs/process/parent_0.1 -am

    Example of Maven command used to take the generated project and package the Java code of all artifacts inside this project as a Docker image.

    -Pdocker package -fae -e
  4. Click Build to take your changes into account and build your artifacts.

Results

When you execute your project on your Continuous Integration server, the filter will be applied and only the artifacts you have filtered will be generated and executed.