Reindex from a remote Elasticsearch cluster - 7.3

Talend Migration and Upgrade Guide
Version
7.3
Language
English
Product
Talend Big Data
Talend Big Data Platform
Talend Data Fabric
Talend Data Integration
Talend Data Management Platform
Talend Data Services Platform
Talend ESB
Talend MDM Platform
Talend Real-Time Big Data Platform
Module
Talend Activity Monitoring Console
Talend Administration Center
Talend Artifact Repository
Talend CommandLine
Talend Data Preparation
Talend Data Stewardship
Talend ESB
Talend Identity and Access Management
Talend JobServer
Talend LogServer
Talend MDM Server
Talend MDM Web UI
Talend Runtime
Talend SAP RFC Server
Talend Studio
Content
Installation and Upgrade
Last publication date
2023-05-19
You need to reindex your Logstash and Talend ESB indices.

Procedure

  1. Run the following command to see all the existing indices: 
    curl -XGET 'oldhost9200/_cat/indices'
    Replace oldhost with the IP address or the hostname of the Elasticsearch cluster used in 7.2.x.
  2. Reindex your Logstash indices from the remote cluster to pull documents from the index used in 7.2.x into the index used in 7.3.x: 
    curl -XPOST 'localhost:9200/_reindex?pretty' -H 'Content-Type: application/json' -d'
{
  "source": {
    "remote": {
      "host": "http://oldhost:9200"
    },
    "index": "logstash-YYYY.MM.dd"
  },
  "dest": {
    "index": "logstash-YYYY.MM.dd"
  }
}
'
  3. Reindex your Talend ESB indices from the remote cluster to pull documents from the index used in 6.5.x into the index used in 7.3.x: 
    curl -XPOST 'localhost:9200/_reindex?pretty' -H 'Content-Type: application/json' -d'
{
  "source": {
    "remote": {
      "host": "http://oldhost:9200"
    },
    "index": "talendesb-YYYY.MM.dd"
  },
  "dest": {
    "index": "talendesb-YYYY.MM.dd"
  }
}
'
    Note:

    You must migrate all the logstash-YYYY.MM.dd and talendesb-YYYY.MM.dd indices one by one or choose only the indices you want to migrate.

    ⓘ

    Available in:

    Big Data Platform

    If you specify logstash-* as the index name, you will get an error like the following one:

    ⓘ

    Available in:

    Data Fabric

    Data Management Platform

    Data Services Platform

    MDM Platform

    Real-Time Big Data Platform

    If you specify logstash-* or talendesb-* as the index name, you will get an error like the following one:

    "failures" : [
{
"index" : "logstash-*",
"type" : "xxxxxxxx",
"id" : "xxxxxxxxxxxxxx",
"cause" : {
"type" : "invalid_index_name_exception",
"reason" : "Invalid index name [logstash-*], must not contain the following characters [ , \", *, \\, <, |, ,, >, /, ?]",
"index_uuid" : "_na_",
"index" : "logstash-*"
},
"status" : 400
},
{
"index" : "logstash-*",
"type" : "xxxxxx",
"id" : "xxxxxxxxxxxxxxx",
"cause" : {
"type" : "invalid_index_name_exception",
"reason" : "Invalid index name [logstash-*], must not contain the following characters [ , \", *, \\, <, |, ,, >, /, ?]",
"index_uuid" : "_na_",
"index" : "logstash-*"
},
"status" : 400
}
]
    For more information, see https://www.elastic.co/guide/en/elasticsearch/reference/current/reindex-upgrade-remote.html.