Introduction to Service Activity Monitoring - 8.0

Talend ESB Infrastructure Services Configuration Guide

Version
8.0
Language
English
Product
Talend Data Fabric
Talend Data Services Platform
Talend ESB
Talend MDM Platform
Talend Open Studio for ESB
Talend Real-Time Big Data Platform
Module
Talend ESB
Talend Runtime
Content
Design and Development
Installation and Upgrade
The Service Activity Monitoring component allows for logging and monitoring service calls made with the Apache CXF framework. Typical use cases are collecting usage statistics and fault monitoring.

The Service Activity Monitoring consists of two parts:

  • Agents (sam-agent) which gather and send monitoring data.
  • A server (sam-server) which processes and stores the data.

The sequence of how these are used is as follows:

  1. The Agent creates events out of requests and replies from both the service consumer and provider side.
  2. The events are first collected locally and then sent to the Service Activity Monitoring Server periodically (so as not to disturb normal message flow).
  3. When the server receives events from the Agent, it optionally uses filters and/or handlers on those events and stores them in a database.

The Service Activity Monitoring Agent and Server are made available as follows:

  • The Service Activity Monitoring Server is available in the Talend Runtime (via the command tesb:start-sam).
  • Alternatively, the Service Activity Monitoring Server can be deployed as a WAR in a servlet container with database access information configured.
  • The Agent is automatically enabled for Data Services deployed on Talend Runtime with the Use Service Activity Monitor option selected in the Talend Studio.
  • The Agent is also available as a JAR that needs to be on the classpath of the service consumer and provider.

From Talend Runtime 8.0 onwards, usernames and passwords in OSGi configurations are resolved as references to environment variables. For the username and password, first the OS environment is queried. If the parameter is not set as an OS environment variable, the Java system variables set in etc/custom.properties are queried. For the username, but not for the password, there is finally a literal default value.

When using OS environment variables for credentials, it is recommended not to have them as cleartext passwords in the startup script, but to retrieve them by calls to the credentials vault (keyrings, wallet, etc.) as it is sketched in comments in the Linux startup script in bin/setsecenv:
#export TESB_AMQ_GUI_USER_PASSWORD=`secret-tool lookup trun-user $TESB_AMQ_GUI_USER_NAME`
#export TESB_AUXSTORE_AUTHENTICATION_PASSWORD=`secret-tool lookup trun-user $TESB_AUXSTORE_AUTHENTICATION_USERNAME`
#export TESB_AUXSTORE_SAML_PASSWORD=`secret-tool lookup trun-user $TESB_AUXSTORE_SAML_USERNAME`
#export TESB_EVENTLOGGING_JMS_PASSWORD=`secret-tool lookup trun-user $TESB_EVENTLOGGING_JMS_USERNAME`
#export TESB_EVENTLOGGING_REST_PASSWORD=`secret-tool lookup trun-user $TESB_EVENTLOGGING_REST_USERNAME`
#export TESB_LOCATOR_AUTHENTICATION_PASSWORD=`secret-tool lookup trun-user $TESB_LOCATOR_AUTHENTICATION_USERNAME`
#export TESB_REGISTRY_AUTHENTICATION_PASSWORD=`secret-tool lookup trun-user $TESB_REGISTRY_AUTHENTICATION_USERNAME`
#export TESB_REGISTRY_SAML_PASSWORD=`secret-tool lookup trun-user $TESB_REGISTRY_SAML_USERNAME`
#export TESB_SAML_SECURITY_PASSWORD=`secret-tool lookup trun-user $TESB_SAML_SECURITY_USERNAME`
#export TESB_SAM_SERVICE_SECURITY_PASSWORD=`secret-tool lookup trun-user $TESB_SAM_SERVICE_SECURITY_USERNAME`
#export TESB_TIDM_PASSWORD=`secret-tool lookup tidm-user $TESB_TIDM_USERNAME`

#export TESB_TLS_KEYSTORE_PASSWORD=`secret-tool lookup keystore-pwd tls`
#export TESB_TLS_KEY_PASSWORD=`secret-tool lookup keystore-pwd tlskey`
#export TESB_STS_KEYSTORE_PASSWORD=`secret-tool lookup keystore-pwd sts`
#export TESB_CLIENT_KEYSTORE_PASSWORD=`secret-tool lookup keystore-pwd client`
#export TESB_CLIENT_KEY_PASSWORD=`secret-tool lookup keystore-pwd clientkey`
#export TESB_SERVICE_KEYSTORE_PASSWORD=`secret-tool lookup keystore-pwd service`
#export TESB_SERVICE_KEY_PASSWORD=`secret-tool lookup keystore-pwd servicekey`
#export TESB_EVENTLOGGING_AGENT_KEYSTORE_PASSWORD=`secret-tool lookup keystore-pwd el`
#export TESB_EVENTLOGGING_AGENT_KEY_PASSWORD=`secret-tool lookup keystore-pwd elkey`

The keystore property files in etc/keystores are replaced by OSGi configurations using references to environment variables. While the keystore property files are still fully valid and evaluated, values from these properties files are overridden if also set in the OSGi configuration files.

For Service Activity Monitoring, the OSGi configuration files are:

  • org.talend.esb.sam.agent.keystore.cfg - Signing keystore for messages sent by the SAM agent
  • org.talend.esb.sam.service.rest.keystore.cfg - SAM REST service response signing keystore
  • org.talend.esb.sam.service.soap.keystore.cfg - SAM SOAP service response signing keystore