CI builder-related Maven parameters - Cloud - 7.3

Talend Software Development Life Cycle Best Practices Guide

Version
Cloud
7.3
Language
English
Product
Talend Big Data
Talend Big Data Platform
Talend Cloud
Talend Data Fabric
Talend Data Integration
Talend Data Management Platform
Talend Data Services Platform
Talend ESB
Talend MDM Platform
Talend Real-Time Big Data Platform
Module
Talend Administration Center
Talend Artifact Repository
Talend CommandLine
Talend JobServer
Talend Management Console
Talend Remote Engine
Talend Studio
Content
Administration and Monitoring
Deployment
Design and Development
Last publication date
2024-02-08

These parameters are detailed in the embedded help of the plugin (see the generate-mojo.html file) and can be used either in the mavenOpts (Jenkins)/mavenOptions(Azure DevOps) or the mvn(Jenkins)/options (Azure DevOps) commands of your CI build definition.

To get the list of Maven parameters as inline help during build time, execute this command:
mvn org.talend.ci:builder-maven-plugin:<your_version>:help 
-Ddetail=true 
-s F:/<your_studio>/configuration/maven_user_settings.xml
Tip: You can also set a specific goal in this command to only return the parameters associated with this specific goal:
mvn org.talend.ci:builder-maven-plugin:<your_version>:help 
-Dgoal=generateAllPoms 
-Ddetail=true 
-s F:/<your_studio>/configuration/maven_user_settings.xml
Maven parameter Value
Talend CommandLine parameters -Dproduct.path

Path to the Talend CommandLine installation directory.

Example:
-Dproduct.path=/opt/Talend/studio
-Dp2Installer.path Path to the Installer files.

If the -Dproduct.path parameter is specified, the Talend CommandLine product will be installed in the value specified for this parameter.

Example:
-Dproduct.path=/home/installation/commandline
-Dp2Installer.path=/home/installation/p2Installer
This allows you to use different paths for installing and patching different product versions.
-Dlicense.path Path to the license of your Talend product.

If the licence path value is a URL and credentials are needed to access it, use the -Dlicense.remote.user and -Dlicense.remote.password parameters.

-Dupdatesite.path

URL address of your server where the P2 repository with Talend CommandLine can be downloaded and installed at build time.

This site also contains the Maven plugins and all the external libraries necessary to build your pipelines.

For an example of how to embed the P2 repository in your Tomcat application, see Before scheduling the execution of your artifacts.

Example:
-Dupdatesite.path=http://localhost:8080/P2/ 
Warning: This parameter only applies to users who have the P2 update site archive, which is provided by Talend.

-DforceUpdate

-Dinstaller.clean

-U

Parameters needed when installing updates from the update site:
  • -DforceUpdate forces the installation of Talend CommandLine from the update site, even if a local installation is present.
    Example:
    -DforceUpdate=true
  • -Dinstaller.clean re-downloads all plugins necessary to execute your build successfully.
    Example:
    -Dinstaller.clean=true
  • -U (with no values) re-downloads all JAR files necessary to execute your build successfully.
-Dpatch.path Path to the patch you have installed or want to install (Talend Studio p2 patch, manual patch, component patch, etc.).
Example if you have installed the patch locally:
-Dpatch.path=D:/patches/Patch_TPS-2222.zip,D:/patches/Patch_TPS-3333.zip 
Example of patch to download:
-Dpatch.path=http://localhost:8085/nexus/content/repositories/releases/Patch_TPS-2222.zip,http://localhost:8085/nexus/content/repositories/releases/Patch_TPS-3333.zip

If credentials are needed to access the remote patch location, use the -Dpatch.remote.user and -Dpatch.remote.password parameters. The patch.remote.password parameter can set a password encrypted by Maven.

-Dgeneration.type Type local as the value to generate the sources of your artifacts and Tests locally without having to keep the Talend CommandLine up and running during the build execution.
Example:
-Dgeneration.type=local

The server generation type is deprecated.

-Dcommandline.skip Type true as the value to skip the source code generation of your artifacts.

This parameter can only be used if you have previously generated your artifacts using Talend CommandLine.

Example:
-Dcommandline.skip=true
-DprojectFilter Allows you to build only specific projects.
Example of filter on all projects with a name starting with CI:
-DprojectFilter="CI.*"
-DitemFilter Allows you to build only specific project items.

Example of filter on Job types:

-DitemFilter=(type=process) to filter on all Standard Jobs of the project

-DitemFilter=(type=process_mr) to filter on all Big Data Map/Reduce and Spark Batch Jobs of the project

-DitemFilter=(type=process_storm) to filter on all Big Data Storm and Spark Streaming Jobs of the project

-DitemFilter=(type=route) to filter on all Routes of the project

Example of filter on Job labels:

-DitemFilter=(type=process_mr)and(label=job_ProcessWeatherData) to filter on one specific Big Data Map/Reduce Job named job_ProcessWeatherData

-DitemFilter=(type=process)and(label%job_dev*) to filter on Jobs which names start with job_dev

Example of filter on Job paths:

-DitemFilter=(type=process)and(path=Integration) to filter on Jobs located in subfolder called Integration

-DitemFilter=(type=process)and(path%Integration*) to filter on Jobs located in subfolders with a name starting with Integration

Example of filter on the person who created the Job:

-DitemFilter=(type=process_storm)and(author=rbunch@talend.com) to filter on Big Data Storm Streaming Jobs whose author ID is rbunch@talend.com

Example of exclusion filter:

-DitemFilter=(!path=sandbox)and(type=process)and(label%job_Export*)or(label%job_Monitor*) to filter on Jobs with a name starting with job_Export or job_Monitor, but that are not in the sandbox folder.

-DitemFilter=(!path%MainProcess/Import*)and(type=process)and(label%job_Export*) to filter on Jobs with a name starting with job_Export, but that are not in the subfolders with a name starting with Import under the MainProcess folder.

Example of filter applied in order to execute Big Data Spark Batch Jobs located in the subfolders with a name starting with Export under the MainProcess folder, and with the exception of the Job named job_batch_feature22:

-DitemFilter=(type=process_mr)and(!label=job_batch_feature22)and(path%MainProcess/Export*)

-Dtalend.job.context and -Dtalend.job.applyContextToChildren When building or scheduling the execution of Jobs, specifies the context group to apply to all Jobs at build time.
Example to apply a context group named dev only to the parent Jobs and apply the default context group to the child Jobs:
-Dtalend.job.context=dev 
Example to apply a context group named dev to the parent Jobs and apply the same context group to the child Jobs:
-Dtalend.job.context=dev -Dtalend.job.applyContextToChildren=true 

If the context group you specify does not exist on the Job, the default context group will be applied.

-Dstudio.forceMavenOnline=true In case of dependency error when building or scheduling the execution of Jobs in offline mode, allows the Studio to download dependencies according to what is defined in your Maven settings file.

The error you may get is the following:

Plugin X.Y.Z or one of its dependencies could not be resolved: Cannot access central

It can also be avoided by re-generating all POM files before build time as these POM files would then contain all the dependencies needed to execute your Jobs.

Custom component parameters -Dcomponents.* When deploying Jobs that use custom components to an artifact repository, specifies the repository in which the component libraries are stored.

This is only required for the custom components created using Talend Component Kit.

For more information, read this documentation.

Example:
-Dcomponents.nexus.user=<user> 
-Dcomponents.nexus.password=<password> 
-Dcomponents.nexus.url=http://localhost:8081/
-Dcomponents.nexus.repository=talend-custom-libs-release 
-Dcomponents.nexus.repository.snapshot=talend-custom-libs-snapshot
Note: For older javajet-based components, refer to this guide.
-Xms -Xmx JVM arguments that allow you to increase the memory available to Maven.
-Dcodes.compile.failOnError To prevent the CI build to fail when encountering some routines with compilation errors that are not used in the artifacts you are building, add this parameter with a false value to skip them and continue the compilation.
Example:
-Dcodes.compile.failOnError=false