REST Sender - 7.1

Talend ESB Infrastructure Services Configuration Guide

EnrichVersion
7.1
EnrichProdName
Talend Data Fabric
Talend Data Services Platform
Talend ESB
Talend MDM Platform
Talend Open Studio for ESB
Talend Real-Time Big Data Platform
EnrichPlatform
Talend ESB
Talend Runtime
task
Design and Development
Installation and Upgrade

The sender receives the event from the processing part via the direct-vm: component and does a technical conversion from the exchange header and body to a JSON format which is stored in the exchange body and all header fields of the event will be removed.

After the conversion from the Exchange Event Structure to the JSON structure, an aggregator will be used to optimize the network transfer with the following aggregation strategy defined in the org.talend.eventlogging.sender.rest.cfg configuration file:

sender.aggregation.eventcount=10
sender.aggregation.eventsize=1024 # in KB
sender.aggregation.sendtimeout=20000 # in milliseconds

Which will be interpreted as follows:

  • If the amount of individual aggregated events reaches 10, the collection of events will be sent as one REST - (POST) to the backend remote Event Logging Service.

  • If the count is not reached but the total size of the event collection within the aggregator reaches the maximum size of 1024 KB, the collection will also be sent, even if the count is not reached.

  • And finally, if the count is not reached and the size is still below the maximum threshold, the event will be sent if the timeout (in milliseconds) is reached, in the above case after 20000 milliseconds. In the above configuration, if the event count is set to 1, it would mean that no real aggregation is done, even though all events will be handed by the aggregator.

This way, events will be sent in a network-optimized way while still be sent in a timely manner. As the aggregator collects events in memory, this transport destination is not as reliable as the JMS option and audit events should preferably always be sent via JMS even though the default will be service for all events in order to limit the initial setup effort.

The service destination can be configured in the org.talend.eventlogging.sender.rest.cfg file as follows:

sender.destination.service.url = https://localhost:8040/eventlogging/events
sender.destination.service.authentication=NO # NO, BASIC
sender.destination.service.username=tadmin
sender.destination.service.password=tadmin