Collector - 8.0

Talend ESB Infrastructure Services Configuration Guide

Talend Data Fabric
Talend Data Services Platform
Talend ESB
Talend MDM Platform
Talend Open Studio for ESB
Talend Real-Time Big Data Platform
Talend ESB
Talend Runtime
Design and Development
Installation and Upgrade

In this part of the Server, events are received (via Direct-VM or JMS).

Each collector has its own bundle and additional collectors can be added by the user, if needed.

The product supports:

  • Direct Collector, which exposes a direct-vm endpoint and which is used by the Event Logging Service for new events (POST).

  • JMS Collector, which reads and gets events from a JMS Broker.

The Direct Collector does not need any configuration, exposes a direct-vm endpoint: eventlogcollector, and uses the direct-vm fix endpoint eventlogserver to put the event into the server processing part.

The JMS Collector needs to be configured in the org.talend.eventlogging.collector.jms.cfg configuration file:

collector.jms.url = tcp://localhost:61616
collectorjms.dlq.buffer.jms.queue = collectorjms.dlq

Here, you can configure the queue and JMS parameters to connect to the remote broker, and the queue used by the JMS Collector to store events in case it is not able to deliver them to the Event Logging server. By default, the event.logging.server queue is used but it can be changed.

In Talend ESB, only ActiveMQ is supported as broker.

The Direct Collector is able to work with collections of events (1 and more events at the same time) and it has the logic to split the events in single events for further processing. Technically, this is the counterpart to the RESTSender on the Agent side which has the feature to Aggregate events to a collection of events.

Events received via the JMS Collector are always received as one event per read. Therefore, no splitting is required in this case.

Both APIs expect that the Event is provided by the Event Logging Agent even though events can technically also be sent directly to the Collector Service if they fulfil the format and completeness required by the event structure.