Reindex from a remote Elasticsearch cluster - 7.2

Talend Data Integration Migration and Upgrade Guide

Version
7.2
Language
English (United States)
Product
Talend Data Integration
Module
Talend Activity Monitoring Console
Talend Administration Center
Talend Artifact Repository
Talend CommandLine
Talend Data Preparation
Talend Data Stewardship
Talend Identity and Access Management
Talend JobServer
Talend Log Server
Talend Runtime
Talend Studio
Content
Installation and Upgrade
You need to reindex your Logstash and Talend ESB indices.

Procedure

  1. Run the following command to see all the existing indices:
    curl -XGET 'oldhost9200/_cat/indices'
    Replace oldhost with the IP address or the hostname of the Elasticsearch cluster used in 7.1.x.
  2. Reindex your Logstash indices from the remote cluster to pull documents from the index used in 7.1.x into the index used in 7.2.x:
    curl -XPOST 'localhost:9200/_reindex?pretty' -H 'Content-Type: application/json' -d'
    {
      "source": {
        "remote": {
          "host": "http://oldhost:9200"
        },
        "index": "logstash-YYYY.MM.dd"
      },
      "dest": {
        "index": "logstash-YYYY.MM.dd"
      }
    }
    '
  3. Reindex your Talend ESB indices from the remote cluster to pull documents from the index used in 6.5.x into the index used in 7.2.x:
    curl -XPOST 'localhost:9200/_reindex?pretty' -H 'Content-Type: application/json' -d'
    {
      "source": {
        "remote": {
          "host": "http://oldhost:9200"
        },
        "index": "talendesb-YYYY.MM.dd"
      },
      "dest": {
        "index": "talendesb-YYYY.MM.dd"
      }
    }
    '
    Note:

    You must migrate all the logstash-YYYY.MM.dd and talendesb-YYYY.MM.dd indices one by one or choose only the indices you want to migrate.

    "failures" : [
    {
    "index" : "logstash-*",
    "type" : "xxxxxxxx",
    "id" : "xxxxxxxxxxxxxx",
    "cause" : {
    "type" : "invalid_index_name_exception",
    "reason" : "Invalid index name [logstash-*], must not contain the following characters [ , \", *, \\, <, |, ,, >, /, ?]",
    "index_uuid" : "_na_",
    "index" : "logstash-*"
    },
    "status" : 400
    },
    {
    "index" : "logstash-*",
    "type" : "xxxxxx",
    "id" : "xxxxxxxxxxxxxxx",
    "cause" : {
    "type" : "invalid_index_name_exception",
    "reason" : "Invalid index name [logstash-*], must not contain the following characters [ , \", *, \\, <, |, ,, >, /, ?]",
    "index_uuid" : "_na_",
    "index" : "logstash-*"
    },
    "status" : 400
    }
    ]