配置 Logstash 使用传输层安全性 (TLS) 加密 - 7.1

Talend Data Fabric 安装指南 (Linux)

EnrichVersion
7.1
EnrichProdName
Talend Data Fabric
task
数据治理
EnrichPlatform
Talend Activity Monitoring Console
Talend Administration Center
Talend Artifact Repository
Talend CommandLine
Talend Data Preparation
Talend Data Stewardship
Talend DQ Portal
Talend ESB
Talend Identity and Access Management
Talend Installer
Talend JobServer
Talend Log Server
Talend MDM Server
Talend MDM Web UI
Talend Repository Manager
Talend Runtime
Talend SAP RFC Server
Talend Studio

过程

  1. 在 Logstash 上启用 TLS:
    1. 在 Logstash config 文件夹下创建一个 certs 子目录。
    2. 将节点的 ca/ca.crt、公用证书和私钥复制到 config/certs 目录。
    3. 运行以下命令以将私钥转换为 pkcs8 格式:
    openssl pkcs8 -in config/certs/logstash.key -topk8 -nocrypt -out config/certs/logstash.pkcs8.key
    logstash_system 用户的密码必须对应于 在 Elasticsearch 中配置传输层安全性 (TLS/SSL) 中生成的密码。
  2. 按如下编辑 config/logstash.yml 文件:
    node.name: logstash.local
    xpack.monitoring.elasticsearch.username: logstash_system
    xpack.monitoring.elasticsearch.password: 'TalendELK'
    xpack.monitoring.elasticsearch.url: https://node1.local:9200
    xpack.monitoring.elasticsearch.ssl.ca: config/certs/ca.crt
    logstash_system 用户的密码必须对应于 在 Elasticsearch 中配置传输层安全性 (TLS/SSL) 中生成的密码。
  3. 按如下编辑 logstash-talend.conf 文件:
    input {
      beats {
        port => 5044
        ssl => true
        ssl_key => '/config/certs/logstash.pkcs8.key'
        ssl_certificate => '/config/certs/logstash.crt'
      }
      http {
        response_headers => {
            "Access-Control-Allow-Origin" => "*"
            "Access-Control-Allow-Headers" => "Content-Type, Access-Control-Allow-Headers, Authorization, X-Requested-With"
            "Access-Control-Allow-Methods" => "*"
            "Access-Control-Allow-Credentials" => "*"
        }
        codec => "json"
        port => 8057
        type => "Audit"
      }
    }
    filter {
        if [type] == "Audit" {
            json { source => "message" }
            mutate {
                rename => {
                    "severity" => "priority"
                    "logMessage" => "message"
                }
            }
            mutate { add_field => { "esIndex" => "talendaudit-%{+YYYY.MM.dd}" } }
        } else {
            grok {
                match => { "message" => "%{URIHOST:agentTimestamp} %{HAPROXYTIME:time} %{DATA:priority} %{SYSLOG5424SD:method} %{JAVACLASS:logger_name} %{GREEDYDATA:log_message}" }
            }
            if [log_message] {
                mutate { update => { "message" => "%{log_message}" } }
            }
            mutate { add_field => { "esIndex" => "logstash-%{+YYYY.MM.dd}" } }
            if [app_id] {
                mutate { rename => { "app_id" => "application" } }
            }
        }
        mutate {
            remove_field   => [ "beats_input_codec_plain_applied", "offset", "beat[name]", "app_id", "beat[hostname]", "host", "tags"  ]
            remove_tag   => [ "beats_input_codec_plain_applied" ]
        }
    }
    output {
      elasticsearch {
        hosts => ["https://node1.local:9200"]
        cacert => 'config/certs/ca/ca.crt'
        user => 'logstash_writer'
        password => 'TalendELK'
        index => "%{esIndex}"
      }
    }
  4. 运行以下命令以使用正确的配置文件启动 Logstash:
    bin/logstash -f  ../logstash-talend.conf