Event Monitoring deployment options - 7.1

Talend ESB Infrastructure Services Configuration Guide

EnrichVersion
7.1
EnrichProdName
Talend Data Fabric
Talend Data Services Platform
Talend ESB
Talend MDM Platform
Talend Open Studio for ESB
Talend Real-Time Big Data Platform
EnrichPlatform
Talend ESB
Talend Runtime
task
Design and Development
Installation and Upgrade

The Event Monitoring can be deployed using the following options.

  • Talend Runtime (EM Agent) - Local

    This is the most easy-to-use setup as it only requires the Talend Runtime with the tesb:start-em-default command to be started.

    With the local deployment option the Event Monitoring data will be just stored in the {KARAF_DATA}/eventmonitoring/em.log file (including up to 10 rolling files names em.log.1 to em.log.9)

    The log data can be either consumed locally or captured by a third party log collection product from this file.

  • Talend Runtime (EM Agent) + Talend Log Server

    The combination of the Talend Runtime which produce the Event Monitoring logs with the Talend Log Server is the standard deployment option that is recommended to use.

    It also requires the Talend Runtime with the tesb:start-em-default command to be started. The Talend Log Server Filebeat with a preconfigured /etc/filebeat_em.yml captures the events from the local em.log file and sends them to the Talend Log Server (Elastic Search). When the events reach the ElasticSearch Server they can be searched and visualized within Talend Administration Center from the Logging page (based on Kibana).

  • Talend Runtime (EM Agent) + JMS Message Broker + Talend Runtime (EM Proxy) + Talend Log Server

    With this deployment option, a more limited setup needs to be done on each Talend Runtime which produces the Events.

    The events will be sent to a central JMS Broker (Apache ActiveMQ as included in Talend ESB) and then another central Talend Runtime which acts as a EM Proxy will read the events from the JMS Broker and store them in the em.log as described above. From here you can either use the Talend Log Server Filebeat and the related configuration, or use a third Party log management tool to get the events from this central instance. In addition to the limited installation of Filebeat on the EM Proxy only, the JMS Broker in between also provides a scalable and robust buffering of log events while they get transferred to the backend.