Installing Service Activity Monitoring

Talend Real-time Big Data Platform Installation Guide for Linux

EnrichVersion
6.4
EnrichProdName
Talend Real-Time Big Data Platform
task
Installation and Upgrade

Service Activity Monitoring (SAM) facilitates the capture of analysis of service activity, including service response times, traffic patterns, auditing and more, by capturing events and storing information. This component consists of two parts:

  • Agents (sam-agent) which gather and send monitoring data

  • A monitoring Server (sam-server) which processes and stores the data

The sequence of how these are used is as follows:

  1. The Agent creates events out of requests and replies from both the service consumer and provider side.

  2. The events are first collected locally and then sent to the Monitoring Server periodically (so as not to disturb the normal message flow).

  3. When the Monitoring Server receives events from the Agent, it optionally uses filters and/or handlers on those events and stores them into a database.

The Agent and Monitoring Server are made available as follows:

  • The agent is by default installed as a feature in Talend ESB Container.

  • The Monitoring Server needs to be installed into a Servlet Container (Tomcat) or an OSGi Container (Talend ESB Container) and needs access to a database.

Once Talend ESB installed, you can access the Service Activity Monitoring server in the Talend-ESB-VA.B.C/add-ons/sam directory to install it or directly install it as a feature within the Talend ESB Container.

Prerequisite to the Monitoring Server

The Monitoring Server requires a database engine to store Events data. The supported databases are listed in Compatible Databases.

The following are the script files corresponding to the databases, run them to configure the database properly. You can find the SQL scripts in the Talend-ESB-VA.B.C/add-ons/sam/db directory.

SQL script filename

Database

create.sqlApache Derby
create_mysql.sqlMySQL
create_oracle.sql Oracle
create_sqlserver.sql SQL Server
create_h2.sql H2 Database Engine
create_db2.sql IBM DB2
  1. Make sure your chosen database is installed properly and is accessible.

  2. Login with a user which has CREATE permissions.

  3. Run the init SQL script for the corresponding database from the table above.

Note

If the value of db.recreate property in the logserver.properties is set to true, the init SQL script will be executed automatically when starting the Monitoring Server. But this is not recommended for any database except Apache Derby running in embedded mode.

You will then find the EVENTS and EVENTS_CUSTOMINFO table have been created in your database.

Now, you can install the Monitoring server either in standalone or as a Feature in the Talend ESB Container.

As an OSGi Feature (Recommended)

Service Activity Monitoring server can be installed as a feature in Talend ESB Container, this way, it will be automatically launched when launching Talend ESB Container.

To launch Service Activity Monitoring as a Feature in Talend ESB Container:

  1. Run the Container.

  2. Type in the following command to start the SAM server Feature: tesb:start-sam.

  3. To stop the SAM server, type in the following command: tesb:stop-sam.

As Web application (Alternative)

To install the Service Activity Monitoring (SAM) server as Web application, you need to:

  • deploy it in an Servlet Container.

  • configure the database connection information,

  • configure the Monitoring endpoint in the Talend ESB Container.

For more information, see the procedures below.

Deploy SAM into Apache Tomcat

  1. Copy the sam-server-war.war file of the Talend-ESB-VA.B.C/add-ons/sam directory.

  2. Paste it in the <TomcatPath>/webapps directory.

    The next time you will start Tomcat, the SAM Server application will automatically be deployed on the server.

    To do it in command line, you can use the following command:

    cp Talend-ESB-VA.B.C/add-ons/sam/sam-server-war.war <TomcatPath>/webapps

  3. You can check whether the SAM Server has been successfully installed and is running by going to the following URL: http://localhost:8080/sam-server-war/services/sam

    Warning

    "http://localhost:8080/sam-server-war/services/sam" is only given as example. Depending on your configuration, you may have to replace <localhost> with the IP address of the Web server and <8080> with the actual port used for the application.

Configure the database connection information

  1. Open the <TomcatPath>/conf/context.xml file and add the following lines, according to your database server:

    For H2:

    <Resource name="jdbc/datasource" auth="Container"
    type="javax.sql.DataSource" username="sa" password=""
    driverClassName="org.h2.Driver"
    url="jdbc:h2:tcp://localhost/~/test"
    maxActive="8" maxIdle="30" maxWait="10000"/>

    For Derby:

    <Resource name="jdbc/datasource" auth="Container"
    type="javax.sql.DataSource" username="test" password="test"
    driverClassName="org.apache.derby.jdbc.ClientDriver"
    url="jdbc:derby://localhost:1527/db;create=true"
    maxActive="8" maxIdle="30" maxWait="10000"/>

    For MySql:

    <Resource name="jdbc/datasource" auth="Container"
    type="javax.sql.DataSource" username="test" password="test"
    driverClassName="com.mysql.jdbc.Driver"
    url="jdbc:mysql://localhost:3306/test"
    maxActive="8" maxIdle="30" maxWait="10000"/>

    For DB2:

    <Resource name="jdbc/datasource" auth="Container"
    type="javax.sql.DataSource" username="db2admin" password="qwaszx"
    driverClassName="com.ibm.db2.jcc.DB2Driver"
    url="jdbc:db2://localhost:50000/TEST"
    maxActive="8" maxIdle="30" maxWait="10000"/>

    For SQLServer:

    <Resource name="jdbc/datasource" auth="Container"
    type="javax.sql.DataSource" username="test" password="test"
    driverClassName="com.microsoft.sqlserver.jdbc.SQLServerDriver"
    url="jdbc:sqlserver://localhost:1029;instanceName=sqlexpress;databaseName=Test"
    maxActive="8" maxIdle="30" maxWait="10000"/>

    For Oracle:

    <Resource name="jdbc/datasource" auth="Container"
    type="javax.sql.DataSource" username="xxx" password="xxx"
    driverClassName="oracle.jdbc.pool.OracleDataSource"
    url="jdbc:oracle:thin:@localhost:1521:XE"
    maxActive="8" maxIdle="30" maxWait="10000"/>
  2. Check the database connection information specified in the following file, and edit them if needed:

    <TomcatPath>/webapps/sam-server-war/WEB-INF/logserver.properties

Configure the Monitoring endpoint in the Talend ESB Container

  1. In the Service Activity Monitoring Server page available at http://localhost:8080/sam-server-war/services/sam, click the services link.

  2. In the services page, copy the Endpoint address, for example: http://localhost:8080/sam-server-war/services/MonitoringServiceSOAP

  3. Go to Talend ESB Container configuration directory: Talend-ESB-VA.B.C/container/etc to configure its SAM agent with the right Monitoring endpoint.

  4. Edit the org.talend.esb.sam.agent.cfg file.

  5. Replace the service.url field with the new Endpoint address.