Migration from Elastic 7.17.x to 9.1.x

The following describes how to update a Windows installation with a single node from 7.17.x to 9.1.x. Since this is an upgrade across two major versions, the installation must first be upgraded to version 8.19.x as an intermediate step. This results in the following migration path. Further information is available at Migration Path.

If you are not yet running Elastic in at least version 7.17.x, you must first update to this version.

Decisions Before the Upgrade

Before starting the upgrade, you should prepare some decisions:

  • Rolling Upgrade or Full Upgrade - This document describes a Full Upgrade, which brings downtime to the cluster. A Rolling Upgrade requires more expertise in operating Elastic.

  • The existing version 7 indices can only be used with limitations in version 9. These must either be re-indexed into a new index or the existing index must be set to read-only. For large amounts of data, reindexing can take several hours. In this case, setting the read-only status is the better choice. In this case, a new rollover index must be created as part of the migration, which will then be used for further work.

  • Additional security level: Creation of index snapshots before the upgrade to be able to return to the previous version in case of an error.

Migration Path

  1. Migration to 8.19.x

  2. Migration to 9.1.x

Upgrade Elasticsearch and Kibana from 7.17.x to 8.19.x

For detailed instructions see: Upgrade Guide

Prepare 7.17.x for the upgrade according to the Elastic guide.

Preparing Elastic specifically means using the Kibana Upgrade Assistant, which can be found in Kibana under Management → Stack Management → Upgrade Assistant. This assistant shows which actions are required for a successful upgrade. These actions must be performed before upgrading to 8.19.x.

Update Elasticsearch (8.19.x)

  • Stop Windows services for Elasticsearch and Kibana

  • Remove Windows service for Elasticsearch

    <old ES directory>\bin\elasticsearch-service.bat remove

  • Download v. 8.19.x

  • Extract the downloaded file to <elasticsearch-installation-dir>/elasticsearch-8.19.x

  • Copy the config directory from Elasticsearch 7.17.x to <elasticsearch-installation-dir>/elasticsearch-8.19.x

    • in <elasticsearch-installation-dir>/config/elasticsearch.yml:

      • check the file for absolute paths and adjust them (for example references to SSL certificates)

      • check if the Elastic data is located inside or outside the Elastic installation directory

        • If the configuration value path.data is set → no action required

        • If the configuration value path.data is not set → move the data directory from Elasticsearch 7.17.x to a location outside the installation directory, for example to <elasticsearch-installation-dir>/data and adjust the configuration value path.data in <elasticsearch-installation-dir>/config/elasticsearch.yml accordingly

  • (optional) If installed, update plugins (not documented here)

  • Start Elasticsearch via Windows services

  • Verify successful startup of Elasticsearch by sending an HTTP GET request to https://elasticsearch-host.example.com:9200. The response should contain information about the node, including the version.

Update Kibana (8.19.x)

  • Download Kibana v. 8.19.x

  • Extract the downloaded file to <kibana-installation-dir>/kibana-8.19.x

  • Copy the data and config directories from Kibana 7.17.x to <kibana-installation-dir>/kibana-8.19.x

  • Edit the Windows service for Kibana

    • Call service configuration

      <NSSM-directory>\nssm.exe edit elastic-kibana

    • Tab Application → Set Path to <kibana-installation-dir>\kibana-8.19.x\bin\kibana.bat

    • Tab Application → Set Startup directory to <kibana-installation-dir>\kibana-8.19.x\bin

    • Tab Details → Update the display name, for example to Elastic Kibana 8.19.x (elastic-kibana)

    • Tab Details → Update the description, for example to Elastic Kibana 8.19.x (windows service)

    • Confirm with Edit service

  • Start Kibana via Windows services

  • Verify successful startup of Kibana by accessing Kibana

Upgrade Elasticsearch and Kibana from 8.19.x to 9.1.x

For detailed instructions see: Upgrade Guide

Prepare 8.19.x for the upgrade according to the Elastic guide.

Preparing Elastic specifically means using the Kibana Upgrade Assistant. The upgrade assistant will display several errors for various elements. The suggested actions must be performed for each element so that the upgrade to 9.1.x can continue. Note down those indices that were set to read-only.

Update Elasticsearch (9.1.x)

  • Stop Windows services for Elasticsearch and Kibana

  • Remove Windows service for Elasticsearch

    <old ES directory>\bin\elasticsearch-service.bat remove

  • Download v. 9.1.x

  • Extract the downloaded file to <elasticsearch-installation-dir>/elasticsearch-9.1.x

  • Copy the config directory from Elasticsearch 8.19.x to <elasticsearch-installation-dir>/elasticsearch-9.1.x

    • in <elasticsearch-installation-dir>/config/elasticsearch.yml:

      • check the file for absolute paths and adjust them (for example references to SSL certificates)

    • in <elasticsearch-installation-dir>/elasticsearch-9.1.x/config/jvm.options:

      • Due to a bug in Elastic, the existing values for GC logging must be replaced: -Xlog:gc*,gc+age=trace,safepoint:file=gc.log:utctime,level,pid,tags:filecount=32,filesize=64m (~ line 96)

  • (optional) If installed, update plugins (not documented here)

  • Set startup type of the service to Automatic (Delayed)

  • Start Elasticsearch via Windows services

  • Verify successful startup of Elasticsearch by sending an HTTP GET request to https://elasticsearch-host.example.com:9200. The response should contain information about the node, including the version.

Update Kibana (9.1.x)

  • Download Kibana v. 9.1.x

  • Extract the downloaded file to <kibana-installation-dir>/kibana-9.1.x

  • Copy the data and config directories from Kibana 8.19.x to <kibana-installation-dir>/kibana-9.1.x

  • Edit the Windows service for Kibana

    • Call service configuration

      <NSSM-directory>\nssm.exe edit elastic-kibana

    • Tab Application → Set Path to <kibana-installation-dir>\kibana-9.1.x\bin\kibana.bat

    • Tab Application → Set Startup directory to <kibana-installation-dir>\kibana-9.1.x\bin

    • Tab Details → Update the display name, for example to Elastic Kibana 9.1.x (elastic-kibana)

    • Tab Details → Update the description, for example to Elastic Kibana 9.1.x (windows service)

    • Confirm with Edit service

  • Start Kibana via Windows services

  • Verify successful startup of Kibana by accessing Kibana

Create New Indices

If individual indices were set to read-only when processing with the upgrade assistant, a new write index must be created (for each data source). New data will then be written to these indices. The old indices can then no longer be edited in Kibana, but the data remains available.

The following example describes how to create a new index ct-arcgis-logfile-000002 that follows the read-only index ct-arcgis-logfile-000001. The new index is set as the write index for the alias ct-arcgis-logfile, so that all new data is written to this index.

PUT ct-arcgis-logfile-000002
{"aliases": {"ct-arcgis-logfile": {"is_write_index" : false}}}

GET /_alias/ct-arcgis-logfile

POST /_aliases
{
  "actions": [
    {
      "add": {
        "index": "ct-arcgis-logfile-000001",
        "alias": "ct-arcgis-logfile",
        "is_write_index" : false
      }
    },
    {
      "add": {
        "index": "ct-arcgis-logfile-000002",
        "alias": "ct-arcgis-logfile",
        "is_write_index" : true
      }
    }
  ]
}
Please adjust the index name to your circumstances.