Trying your custom connector or processor in Talend Cloud Pipeline Designer - Cloud

Talend Remote Engine Gen2 Quick Start Guide

Version
Cloud
Language
English
Product
Talend Cloud
Module
Talend Management Console
Talend Pipeline Designer
Content
Deployment > Deploying > Executing Pipelines
Installation and Upgrade
Last publication date
2024-01-24

Once your connector or processor is developed and the new Docker image is created, you can try it in the Talend Cloud Pipeline Designer app.

Before you begin

Make sure the engine containing your new Docker image has been restarted and is available.

Procedure

  1. To test your custom source or destination connector:
    1. In Talend Cloud Pipeline Designer, go to the CONNECTIONS page.
    2. Click ADD CONNECTION and fill in the connection properties parameters to access your data. Make sure the updated Remote Engine Gen2 is selected.

      Example

      After developing a new source connector based on the OpenWeather API and restarting the engine, it is displayed in the Connection type list:
      Screenshot on how to add a new connection with Remote Engine Gen2 in Talend Cloud Pipeline Designer.
    3. Validate your connection and add a dataset.

      Example

      Once the OpenWeather connection is set up, the dataset properties can be configured. Here, the weather data about Paris, France is retrieved.
      Screenshot on the configuration of the dataset properties.
    4. Validate your dataset.
      The new dataset can now be used as a source connector in a pipeline.
      Screenshot of the newly validated dataset.
  2. To test your custom processor:
    1. In Talend Cloud Pipeline Designer, open an existing pipeline.
    2. In the Select a run profile list next to the Run button, make sure the run profile associated to your updated Remote Engine Gen2 is selected.
    3. Click Add where you want to add your processor. The new processor should be displayed in the Add a processor window along with the other processors.

      Example

      A new processor allowing you to change the case of your data has been added and is displayed in the list.
      Screenshot on the display of the new processor in Add a processor window.
    4. Select your processor to add it to your pipeline.
      The new processor can now be configured and used directly in your pipeline.