Reindex from a remote Elasticsearch cluster - 7.2

Talend Big Data Migration and Upgrade Guide

EnrichVersion
7.2
EnrichProdName
Talend Big Data
EnrichPlatform
Talend Activity Monitoring Console
Talend Administration Center
Talend Artifact Repository
Talend CommandLine
Talend Data Preparation
Talend Data Stewardship
Talend Identity and Access Management
Talend JobServer
Talend Log Server
Talend Runtime
Talend Studio
task
Installation and Upgrade
You need to reindex your Logstash and Talend ESB indices.

Procedure

  1. Run the following command to see all the existing indices:
    curl -XGET 'oldhost9200/_cat/indices'
    Replace oldhost with the IP address or the hostname of the Elasticsearch cluster used in 7.1.x.
  2. Reindex your Logstash indices from the remote cluster to pull documents from the index used in 7.1.x into the index used in 7.2.x:
    curl -XPOST 'localhost:9200/_reindex?pretty' -H 'Content-Type: application/json' -d'
    {
      "source": {
        "remote": {
          "host": "http://oldhost:9200"
        },
        "index": "logstash-YYYY.MM.dd"
      },
      "dest": {
        "index": "logstash-YYYY.MM.dd"
      }
    }
    '
  3. Reindex your Talend ESB indices from the remote cluster to pull documents from the index used in 6.5.x into the index used in 7.2.x:
    curl -XPOST 'localhost:9200/_reindex?pretty' -H 'Content-Type: application/json' -d'
    {
      "source": {
        "remote": {
          "host": "http://oldhost:9200"
        },
        "index": "talendesb-YYYY.MM.dd"
      },
      "dest": {
        "index": "talendesb-YYYY.MM.dd"
      }
    }
    '
    Note:

    You must migrate all the logstash-YYYY.MM.dd and talendesb-YYYY.MM.dd indices one by one or choose only the indices you want to migrate.

    "failures" : [
    {
    "index" : "logstash-*",
    "type" : "xxxxxxxx",
    "id" : "xxxxxxxxxxxxxx",
    "cause" : {
    "type" : "invalid_index_name_exception",
    "reason" : "Invalid index name [logstash-*], must not contain the following characters [ , \", *, \\, <, |, ,, >, /, ?]",
    "index_uuid" : "_na_",
    "index" : "logstash-*"
    },
    "status" : 400
    },
    {
    "index" : "logstash-*",
    "type" : "xxxxxx",
    "id" : "xxxxxxxxxxxxxxx",
    "cause" : {
    "type" : "invalid_index_name_exception",
    "reason" : "Invalid index name [logstash-*], must not contain the following characters [ , \", *, \\, <, |, ,, >, /, ?]",
    "index_uuid" : "_na_",
    "index" : "logstash-*"
    },
    "status" : 400
    }
    ]