Details of the Azure script - 8.0

Talend Software Development Life Cycle Best Practices Guide

Talend Big Data
Talend Data Fabric
Talend Data Integration
Talend Data Management Platform
Talend Data Services Platform
Talend ESB
Talend MDM Platform
Talend Administration Center
Talend Artifact Repository
Talend CommandLine
Talend JobServer
Talend Studio
Administration and Monitoring
Design and Development

List of the steps the azure-pipeline-build_changes.yml script is going to follow. The lines of the script are numbered so that you can easily find the corresponding description.

  • Extracts via RegEx (see line 3-4) the name of the Azure DevOps user/account (line 5) and the name of the Azure DevOps project (line 6) from the pipeline URL.
  • Gets the current build number from the Global Variable $BUILD_ID and compute the previous build id (line 8-9). All these information are required to construct the API URL (line 11). The call requires an ACCESSTOKEN. This token is passed as an environment variable to the script (line 47).
  • Invokes the API (curl) and extracts the number of commits of the last Git push (line 15).
  • Executes the git diff command (line 21) to get the list of files in the project involved in these commits.
  • Information is given about these modified files:
    • The file variable contains the full path of the file from the root of the project: <project name>/...
    • You are interested only by the files that have changed in the folders which names start with <project name>/process. This is where the Talend Job definition is located (line 27).
    • A combination of bash commands allows you to extract the list of modules from the file path. The list in this example includes Data Integration, Big Data batch and streaming Jobs (line 30-32).
    • The Maven command requires the list of modules to be in lower case, hence the conversion command (line 33).
    • A report of list of the Jobs is displayed in the console (line 34-40).
    • This list is then saved as environment variable to be retrieved by the next pipeline step (line 44).
  • Displays a message if there are no commits in the process folders (line 42).
  • Then the variable module_list can be used by following Azure pipeline steps (line 51-54).
    Note: The pipeline condition condition: ne(variables['module_list'], '') is important to avoid to start a build if no relevant commits were detected.