Configure Logstash to use Transport Layer Security (TLS) encryption - 7.0

Talend ESB Installation Guide for Windows

EnrichVersion
7.0
EnrichProdName
Talend ESB
task
Installation and Upgrade
EnrichPlatform
Talend Administration Center
Talend Artifact Repository
Talend CommandLine
Talend Data Preparation
Talend Data Stewardship
Talend ESB
Talend Identity and Access Management
Talend Installer
Talend Log Server
Talend Runtime
Talend Studio

Procedure

  1. Enable TLS on Logstash:
    1. Create a certs subdirectory under the Logstash config folder.
    2. Copy the ca/ca.crt, the public certificate and the private key of the node to the config/certs directory.
    3. Run the following command to convert the private key to the pkcs8 format:
    openssl pkcs8 -in config/certs/logstash.key -topk8 -nocrypt -out config/certs/logstash.pkcs8.key
    The password of the logstash_system user must correspond to the password generated in Configuring Transport Layer Security (TLS/SSL) in Elasticsearch.
  2. Edit the config/logstash.yml file as follows:
    node.name: logstash.local
    xpack.monitoring.elasticsearch.username: logstash_system
    xpack.monitoring.elasticsearch.password: 'TalendELK'
    xpack.monitoring.elasticsearch.url: https://node1.local:9200
    xpack.monitoring.elasticsearch.ssl.ca: config/certs/ca.crt
    The password of the logstash_system user must correspond to the password generated in Configuring Transport Layer Security (TLS/SSL) in Elasticsearch.
  3. Edit the logstash-talend.conf file as follows:
    input {
      beats {
        port => 5044
        ssl => true
        ssl_key => '/config/certs/logstash.pkcs8.key'
        ssl_certificate => '/config/certs/logstash.crt'
      }
      http {
        response_headers => {
            "Access-Control-Allow-Origin" => "*"
            "Access-Control-Allow-Headers" => "Content-Type, Access-Control-Allow-Headers, Authorization, X-Requested-With"
            "Access-Control-Allow-Methods" => "*"
            "Access-Control-Allow-Credentials" => "*"
        }
        codec => "json"
        port => 8057
        type => "Audit"
      }
    }
    filter {
        if [type] == "Audit" {
            json { source => "message" }
            mutate {
                rename => {
                    "severity" => "priority"
                    "logMessage" => "message"
                }
            }
            mutate { add_field => { "esIndex" => "talendaudit-%{+YYYY.MM.dd}" } }
        } else {
            grok {
                match => { "message" => "%{URIHOST:agentTimestamp} %{HAPROXYTIME:time} %{DATA:priority} %{SYSLOG5424SD:method} %{JAVACLASS:logger_name} %{GREEDYDATA:log_message}" }
            }
            if [log_message] {
                mutate { update => { "message" => "%{log_message}" } }
            }
            mutate { add_field => { "esIndex" => "logstash-%{+YYYY.MM.dd}" } }
            if [app_id] {
                mutate { rename => { "app_id" => "application" } }
            }
        }
        mutate {
            remove_field   => [ "beats_input_codec_plain_applied", "offset", "beat[name]", "app_id", "beat[hostname]", "host", "tags"  ]
            remove_tag   => [ "beats_input_codec_plain_applied" ]
        }
    }
    output {
      elasticsearch {
        hosts => ["https://node1.local:9200"]
        cacert => 'config/certs/ca/ca.crt'
        user => 'logstash_writer'
        password => 'TalendELK'
        index => "%{esIndex}"
      }
    }
  4. Run the following command to start Logstash with the right configuration file:
    bin/logstash -f  ../logstash-talend.conf