Building your connector image - Cloud

Talend Remote Engine Gen2 Quick Start Guide

EnrichVersion
Cloud
EnrichProdName
Talend Cloud
EnrichPlatform
Talend Management Console
Talend Pipeline Designer
task
Deployment > Deploying > Executing Pipelines
Installation and Upgrade

To build your new connectors Docker image, you need to use the remote-engine-customizer tool.

This tool that reads the engine docker-compose.yml file, extracts the connectors image and rebuilds it adding configured connectors/processors to the image. By default, it replaces the original image with the newly built one and saves the old file in the .remote_engine_customizer/backup/ directory of the Remote Engine Gen2.

To get more information about the tool, run this command: docker run tacokit/remote-engine-customizer:${version} help

Procedure

  1. Pull the latest remote-engine-customizer image using this command:
    docker pull tacokit/remote-engine-customizer
  2. Set up the variables necessary to build the new connectors image:
    export remote_engine_folder=/opt/my-remote-engine            # The engine directory
    export component_path=/opt/dev/my-component                  # The path to your component code
    export car=custom-components-0.0.1-SNAPSHOT.car              # The name of your car package
  3. Build the new connectors image:
    docker run \
       -v ${remote_engine_folder}:/opt/re \
       -v ${component_path}:/opt/exchange \
       -v /var/run/docker.sock:/var/run/docker.sock \
       tacokit/remote-engine-customizer:${version} \
       register-component-archive \
       --remote-engine-dir=/opt/re \
       --component-archive=/opt/exchange/target/${car}

Results

Your new connectors image is built and the image ID is displayed. Copy that image ID to your clipboard, you will use it to update the existing connectors image.
Example of console output where the image ID is e761f26e4adb:20191206151130452:
docker run \
   -v /opt/my-remote-engine:/opt/re \
   -v /opt/dev/my-component:/opt/exchange \
   -v /var/run/docker.sock:/var/run/docker.sock \
   tacokit/remote-engine-customizer:1.1.15 \
   register-component-archive \
   --remote-engine-dir=/opt/re \
   --component-archive=/opt/exchange/target/custom-components-0.0.1-SNAPSHOT.car

[INFO] Building image 'e761f26e4adb:20191206151130452' from 'e761f26e4adb' adding [/opt/exchange/target/test-components-0.0.1-SNAPSHOT.car]
[INFO] Looking for component-registry.properties configuration, this can be a bit long...
[INFO] Building image 'e761f26e4adb:20191206151130452'
[INFO] Restart your remote engine to take into account the new connector image