Webhooks in Talend Management Console - Cloud

Talend Management Console User Guide

Version
Cloud
Language
English
Product
Talend Cloud
Module
Talend Management Console
Content
Administration and Monitoring > Managing projects
Administration and Monitoring > Managing users
Deployment > Deploying > Executing Pipelines
Deployment > Deploying > Executing Tasks
Deployment > Scheduling > Scheduling Tasks
Last publication date
2024-02-29

Webhooks can trigger the execution of a Job task or plan based on calls from external applications.

Use a webhook trigger only when the security policy of your organization allows to do so:
  • Do not submit credentials to a webhook trigger because it does not need any authentication.
  • Keep the webhook URL in a secret place because anyone can use the URL to trigger your task.
  • If authentication is required, consider to use an API instead of a webhook trigger.

You cannot add a webhook if you already had other trigger types. You can delete other triggers and add a webhook when editing the task/plan, or you can also copy the task/plan if you need both.

Note: To be able to execute a task through a webhook, the source Job must contain a webhook context parameter called resource_webhook_payload. For plans, at least one task in the first step must have a source Job with the context parameter.
Each call to the webhook endpoint is wrapped in the following JSON structure, passed to the Job execution via resource_webhook_payload parameter:
  • type: media type of the payload taken from Content-Type header
  • data: payload data
The following payload types are supported:
  • application/xml
  • application/json
  • application/x-www-form-urlencoded
  • text/plain
  • text/xml

Payloads are processed as an array. The total payload size for Big Data Jobs must not exceed 5 kb.

When generating a webhook URL in Talend Management Console, you must set the following parameters:
  • Window threshold: defines the number of calls to the service.
  • Window duration (in seconds): defines the number of seconds after the first service call.

The first event to occur triggers the execution.

Restriction: The same task or plan can only be in the execution queue (in Pending or Running status) maximum 50 times in a 60-minute time period. If you have scheduled a task or plan through the public API or a webhook to run more times than that, any new execution that would be added to the queue will fail after the limit is reached.

Example

If the window size is 2, during the time set in the Window threshold parameter two consecutive payloads are sent to the webhook endpoint:

{"param":"1"}
{"param":"2"}
However, the task/plan execution does not receive them as two separate payloads, it receives them as an array:
[
{"data":"{"param":"1"}","type":"application/x-www-form-urlencoded"},
{"data":"{"param":"2"}","type":"application/x-www-form-urlencoded"}
]
The double quotation marks in the Webhook payloads are not escaped anymore. If some of your Jobs still use backslash (\) to escape them, set the following property in the <RemoteEngineInstallation>/etc/org.talend.ipaas.rt.jobserver.client.cfg configuration file:
job.ctx.params.passthrough=true
Note: A successful webhook call returns an HTTP 204 (No Content) response.