Event Logging Overview - 7.1

Talend ESB Infrastructure Services Configuration Guide

EnrichVersion
7.1
EnrichProdName
Talend Data Fabric
Talend Data Services Platform
Talend ESB
Talend MDM Platform
Talend Open Studio for ESB
Talend Real-Time Big Data Platform
EnrichPlatform
Talend ESB
Talend Runtime
task
Design and Development
Installation and Upgrade
The primary sub-parts of the Event Logging are:
  • Event Logging - Listener, which gets log information and passes it to the Agent for further local processing.

  • Event Logging - Agent, which receives events from listeners, buffers, processes and sends them to a final destination using one of the configured Event Logging Appenders.

  • Event Logging - Sender, which sends events to the Event Logging backend. A JMS Appender and a REST Appender for the EventLogging RESTService are provided by default to write into a local log file or to a local elastic search instance, for example. But customers can add their own appenders.

  • Event Logging - Service, which is a RESTful Service to collect events sent from the Event Logging REST Appender and to retrieve events from the backend. Events are stored in the Event Database only since version 5.4 of Talend ESB.

  • Event Logging - Collector, which consists of Routes which listen on endpoints to receive events for further server-side processing by the Event Logging Server. By default, a Standard Collector which exposes a fixed Direct-VM endpoint, and a JMS Collector which takes Event out of a JMS Queue are provided.

  • Event Logging - Server, which retrieves events from the collector and performs processing and persistence of events, with support for Event Database (RDBMS) and Service Activity Monitoring Database (Service Activity Monitoring Events only).

  • Talend Log Server , based on Elasticsearch.

  • Logging page in Talend Administration Center, based on Elasticsearch and Kibana.

The following high level architecture shows the different components and their relations:

Technically, the Event Logging Listener, Agent, Sender, Collector and Server are implemented using Apache Camel. The Event Logging Service is a CXF (JAX-RS) based service developed in Java (also using Apache Camel for the Receiver part). The Elasticsearch-based Talend Log Server and Kibana-based Logging page in Talend Administration Center is available as a separate deployment outside the Talend Runtime Container and is optional for the use of the Event Logging feature. The Event Logging Database is supported on all databases supported by the Service Activity Monitoring Database (PostgreSQL included).

The following sections describe the installation and starting of the Event Logging feature, the individual components of the overall Event Logging feature, and the Data Structures and public API of this feature. For more information about the installation and starting of the Talend Log Server and Talend Administration Center, see the Talend Installation and Upgrade Guide .