Server Pre-processing part - 7.1

Talend ESB Infrastructure Services Configuration Guide

Version
7.1
Language
English (United States)
Product
Talend Data Fabric
Talend Data Services Platform
Talend ESB
Talend MDM Platform
Talend Open Studio for ESB
Talend Real-Time Big Data Platform
Module
Talend ESB
Talend Runtime
Content
Design and Development
Installation and Upgrade

After one of the collector Routes receives the event(s) and sends it/them via Direct-VM communication to the server Route, the pre-processing will be done as a first step.

The system Pre-processing step will do the following:

  • Check if the category of the event is ping, and if it is the case, it will discard the event without any further processing. Ping events are sent by the agent to check the Broker and Queue availability during the period where the network or the broker communication is not working as expected. But as defined in the Agent, the ping category is the third reserved category. These events are no real events and processing will stop immediately at this stage for ping events.

  • Convert the JSON body back to the Event Exchange Structure, as defined in the Event Structure Camel Exchange mapping below.

  • Check if an Event Category exists and if not, applies a default one to the event.

  • Set the server timestamp to the Event.

The following configuration in the org.talend.eventlogging.server.cfg file applies to this part:

preprocessing.eventCategory=system

This parameter set the event to the system category if no Category exists until this stage, which can only be the case if the event was not sent via the Event Logging Agent.

After this step, the user can configure a user-defined pre-processing using a customer Route, which can be configured as follows for each individual category in the org.talend.eventlogging.server.cfg file:

preprocessing.custom.routeid.default=myCustomPreProcRoute
preprocessing.custom.routeid.audit=myCustomrAuditPreProcRoute
preprocessing.custom.routeid.sam=myCustomSamPreProcRoute

As in the Event Logging Agent, the preprocessing.custom.routeid. will be followed by the category name, where default and audit are reserved. Default configuration will be applied to all categories which are not explicitly mapped. Audit will be applied to all events which have the audit flag set, regardless of the event category.

In the above example, audit, and sam events not marked as audit, will be pre-processed by the related custom Routes while all other events will be pre-processed by the default Route myCustomPreProcRoute.

By default, no custom processing is required and no processing is configured.

The custom processing will be started using the configured Route ID via a Direct-VM communication between Routes. To do this, the customer Route needs to be deployed on the same Talend Runtime or JVM as the one on which the Collector Server is running.