Installing the Talend logging modules - 7.3

Talend Installation Guide for Linux

Version
7.3
Language
English
EnrichDitaval
Linux
EnrichSubscriptionType
Subscription
Product
Talend Big Data
Talend Big Data Platform
Talend Data Fabric
Talend Data Integration
Talend Data Management Platform
Talend Data Services Platform
Talend ESB
Talend MDM Platform
Talend Real-Time Big Data Platform
Module
Talend Activity Monitoring Console
Talend Administration Center
Talend Artifact Repository
Talend CommandLine
Talend Data Preparation
Talend Data Stewardship
Talend DQ Portal
Talend ESB
Talend Identity and Access Management
Talend Installer
Talend JobServer
Talend LogServer
Talend MDM Server
Talend MDM Web UI
Talend Repository Manager
Talend Runtime
Talend SAP RFC Server
Talend Studio
Content
Installation and Upgrade
You need to manually install Talend LogServer which includes Kibana and Filebeat to collect logs.

Before you begin

The Elasticsearch container requires that the vm.max_map_count parameter to be set to at least 262144. Check this value on your machine and increase it if needed.
To check this value, run the following command:
sysctl vm.max_map_count
If you need to increase the value, run the following command:
sysctl -w vm.max_map_count=262144
To permanently write the value to the sysctl.conf file, run the following command:
vm.max_map_count = 262144

Procedure

  1. Copy and extract the Talend-LogServer-VA.B.C.zip archive file in the directory of your choice.
    Note: The directory name must not contain spaces or non-ASCII characters.
  2. To start Talend LogServer launch the start_logserver.sh executable file.
    You cannot run Elasticsearch as the root user. Elasticsearch is part of the Talend LogServer, therefore you cannot run the executable file as root user.
  3. Configure the values for LOG_PATH and APP_NAME for Filebeat:
    • Open the filebeat.yml file located in the Filebeat directory and set the LOG_PATH and APP_NAME values as follows:
        paths:
          - ${LOG_PATH:/home/Talend/7.2.1/tac/apache-tomcat/logs/*}
        fields:
          app_id: ${APP_NAME:TAC}
    • Or, set the LOG_PATH and APP_NAME environment variables:
      export  LOG_PATH="/home/Talend/7.2.1/tac/apache-tomcat/logs/*"
      export  APP_NAME="TAC"
  4. Start Filebeat:
    filebeat -e -c filebeat.yml

Results

You can now access Talend LogServer with the following URL: http://localhost:5601/app/kibana#/dashboard/Default-Dashboard.

Available in:

Data Fabric

Data Services Platform

ESB

MDM Platform

Real-Time Big Data Platform

The ESB event logs that are displayed in the Logging page of Talend Administration Center are collected thanks to the Event Logging features that can be started from the Runtime container. For more information, see the Talend ESB Container Administration Guide.

Troubleshooting: When you start the Talend LogServer, if you do not see logstash-*, talendesb-*, and talendaudit-* indices, complete the following steps:
  1. Delete the .kibana index.
    curl -XDELETE 'http://localhost:9200/.kibana'
  2. Stop the Talend LogServer.
  3. Start the Talend LogServer.