The Log Listener allows the user to get all the log information, which typically will also be available in tesb.log, into the Event Logging. This means that the Log Listener is the primary listener for the Event Logging Agent.
The pax logging, also used by Talend Runtime, is used as the entry point for the Event Logging Log Listener. This way, the user can also use the standard pax logging configuration to configure which logs should be sent to the Log Listener and which logs should not, like for any other log appender.
log4j2.rootLogger=INFO, out, osgi:VmLogAppender
The Log Listener will expose a second pax logging appender named
eventloglisteneraudit where all log events sent through this
appender will be set to
audit=true and thus, would have the highest
priority of all audit configurations. In general, if an Event has the audit flag set
to true, no other Talend
components or mappings will change it back to false. But if the audit flag is false,
it might be set to true later on, by additional mapping options. The
eventloglisteneraudit can be used in the pax logging configuration
and would automatically make all events, sent via this appender, 'Audit'
The Log Listener is implemented using Apache Camel, via the paxlogging: component, and provides a configurable Category Mapper processor with a Direct-VM communication to the Talend Event Logging Agent.
paxlogging: component will have a fixed configuration, to expose
eventloglistener as appender name:
log4jLogger.info("log4j log message");
slf4jLogger.info("slf4j log message");
jdkLogger.info("JDK log message");
jclLogger.info("JCL log message");
juliLogger.info("juli log message");
And in case the logging framework supports MDC attributes, the MDC attributes are
also taken into the event as additional metadata. This would also allow custom and
business code logs to contain a business correlation ID or other important metadata
in a structured form, for example, using
The "Log to Event" conversion step will transform the log message format into the Event format, where the Event Structure is as completely filled as possible, and the additional metadata (including the MDC attributes) is transformed in the Event Customer Information (Key, Value) list.
The Category Mapper allows the user to define an Event Logging category in a configurable way to the log message by package name from where the message comes from, and if messages from this package should be treated as audit or non-audit messages.
The Log Listener uses the org.talend.eventlogging.listener.log.cfg configuration file with the following parameters:
# Default category for events category.default=system # Define key in MDC attributes which will be used to get EventCategory category.attribute=eventCategory # Define key in MDC that specify audit audit.attribute=watchThis # Category mapping configuration, for example, if a log message comes from # package org.apache.cxf, it will be mapped to the service category category.mapping.org.apache.cxf = service category.mapping.org.apache.cxf.rt.security = security(audit=true) category.mapping.org.apache.wss4j = security(audit=true) category.mapping.org.talend.esb.sts=security(audit=true)
The configuration is primarily for the Category Mapper and will be used as follows:
The first priority to define which EventCategory is assigned to the current log event is defined by an MDC attribute. The name of this attribute can be configured under
category.attribute. By default, it is
eventCategory. If this attribute exists, the Category will be set to the value of this attribute.
If the attribute does not exist, the Category Mapper will look for the
category.mapping.configuration. It will build a tree in memory, will go from the leaves to the root of this tree to find the closest node to the given package name, as the log4j log appender package based filter configuration would do, and will apply the category and audit flag definition as defined in the configuration file, but only if the audit attribute is not already set to true and no overwrite of true is allowed for the audit flag. Once the audit flag is true, it will stay true.
audit.attributeparameter of the log event has a custom value and the audit setting is currently set to false, the audit setting will automatically be activated and it will take the custom value referred to in the
In the example above, if a log message comes from package
org.apache.cxf.binding.soap, it will be mapped to the
servicecategory, with audit implicitly set to false (
org.apache.cxfis the matching node configuration.
If a log message comes from
org.apache.cxf.rt.security.saml, it will be mapped to the
securitycategory and marked as an audit message by the additional audit definition after the category:
By default, if
auditis not explicitly defined after the category, event will be set to
If the audit setting of the event was not yet set to true, and its attribute contains a custom value, for example:
watchThis, the audit will automatically be set to true.
After this step, the log message is sent to the Event Logging Agent in a synchronous way (Direct-VM) to allow the agent to do a short pre-processing before the event is stored into a local buffer within the agent for final processing and sending to the backend.