Installing and configuring the Talend logging modules - 6.1

Talend Data Services Platform Installation Guide

EnrichVersion
6.1
EnrichProdName
Talend Data Services Platform
task
Installation and Upgrade
EnrichPlatform
Talend Activity Monitoring Console
Talend Administration Center
Talend Artifact Repository
Talend CommandLine
Talend DQ Portal
Talend ESB
Talend Identity Management
Talend Installer
Talend JobServer
Talend Log Server
Talend Project Audit
Talend Repository Manager
Talend Runtime
Talend SAP RFC Server
Talend Studio

This section describes the installation and configuration of the logging modules (Talend Log Server based on Elasticsearch and Kibana) that allow you to display in Talend Administration Center the Logging page which groups and display the output logs filtered by categories and event types (Data Integration, ESB or MDM events). For more information on how to display the logs in Talend Administration Center, see the Talend Administration Center User Guide.

To install the logging modules using Talend Installer, read To install and start the logging modules using Talend Installer.

To install the logging modules manually, read To install and start the logging modules manually.

To configure the name of the log cluster, read To define the name of the log cluster.

For more information on how to install Talend log server as a service, see Installing services.

To install and start the logging modules using Talend Installer

Note that, to make the best use of the Logging page of Talend Administration Center, it is recommended to install Talend Administration Center via Talend Installer as Talend Installer installs automatically the application with the items necessary to use this module (Kibana Web application with configured dashboards, pre-configured Talend Log server, etc.).

Also note that this configuration procedure only applies to standard HTTP connections to Kibana. For more information on how to configure a secure connection to Kibana, see https://help.talend.com/display/KB/How+to+configure+a+secure+connection+for+Kibana.

  1. Perform an installation via Talend Installer. If you perform a Custom type installation, make sure the Talend Administration Center and Talend Log Server check boxes are selected before completing the installation process.

    A pre-configured and ready-to-be-used Talend Log Server is installed in a directory named logserv, and the Kibana Web application is deployed in the installation directory of the Tomcat server which contains Talend Administration Center.

    Talend Installer also installs automatically on the Log Server an Ontology repository with semantic indexes. This Ontology repository is used to profile data based on semantic categories.

  2. Talend Installer allows you to install Talend Log Server as a service, but you can also start it manually via the executable (.bat or .sh) file.

    As Kibana is a Web application installed in Tomcat, start the Tomcat server to deploy and start this application.

  3. Once Tomcat and thus the Talend Administration Center and Kibana applications are started, go to the Configuration page and fill in the required fields in the Monitoring and Logging nodes to link Talend Administration Center with the logging modules. For more information, see the Talend Administration Center User Guide.

    The ESB event logs that are displayed in the Logging page of Talend Administration Center are collected thanks to the Event Logging features that can be started from the Runtime container. For more information, see the Talend ESB Container Administration Guide.

To install and start the logging modules manually

If you want to install the logging modules manually, you need to install both the Kibana web application and the Talend Log Server.

  1. Copy and extract the Talend-LogServer-VA.B.C.zip archive file in the directory of your choice.

  2. Go to the Talend Log Server directory and edit the following file: <LogServerPath>/elasticsearch-1.5.2/config/elasticsearch.yml

  3. At the very end of the file, uncomment the following lines:

    http.cors.enabled: true
    http.cors.allow-origin: "http://TAC_SERVER_HOST:TAC_SERVER_PORT"

    And replace TAC_SERVER_HOST and TAC_SERVER_PORT with the IP address and port of the server.

  4. Launch the executable file corresponding to your operating system to start Talend Log Server:

    start_logserver.bat if you are on Windows

    start_logserver.sh if you are on Linux/Unix

  5. Copy and extract the Kibana-A.B.C.zip file located in the Talend Log Server archive file.

  6. Rename the Kibana-A.B.C folder to just kibana and put it in the following folder:

    <TomcatPath>/Webapps

  7. As Kibana is now installed in Tomcat, start the Tomcat server to deploy and start this web application.

  8. Once Tomcat and thus the Talend Administration Center and Kibana applications are started, go to the Configuration page and fill in the required fields in the Monitoring and Logging nodes to link Talend Administration Center with the logging modules. For more information, see the Talend Administration Center User Guide.

    The ESB event logs that are displayed in the Logging page of Talend Administration Center are collected thanks to the Event Logging features that can be started from the Runtime container. For more information, see the Talend ESB Container Administration Guide.

To define the name of the log cluster

By default, all logs are centralized in a cluster group called talend-log-central. However, if you want to use a different cluster name, for example if you do not want to mix logs from different environments such as Quality Assurance logs and production logs, do the following.

  1. Open the logstash-talend.conf file located in the Talend Log server installation directory.

  2. Edit the name of the cluster in the output part:

    output {
      elasticsearch {
        embedded => true
        cluster => "talend-log-central"
      }