Upgrading elasticsearch 2.x to 5.x

Last year elasticsearch 5.0 released to public and 5.2 just released last month, so it should be the right time to upgrade your infrastructure to the latest version of our beloved search engine.

This article give us many insight about why we should upgrade to the latest version : https://amsterdam.luminis.eu/2016/10/18/elasticsearch-5-is-coming-what-is-new-and-improved/

Upgrading elasticsearch is not easy task although it is not hard to make the cluster run again, especially if you use deprecated thing so make sure to simulate all things in your staging environment. Luckily the documentation is complete, so it can save your time. 

First step, check current stack

The most important part is to understand what will work or what will break in the new version. It is mostly divided in 2 parts, the cluster part and the application part.

At the cluster part there are mapping and configuration compatibility. There are many changes in configuration, many of them are not valid anymore. Like we couldn’t set shard configuration in the cluster level (inside elasticsearch.yml), another thing introduced are centralized jvm.options file for jvm environtment settings.

In general Elasticsearch 5.x is compatible with 2.x index, however if you have no problem with reindexing try to embrace the change and get rid all deprecated things. You can use elastic-migration plugin to help you find all this cluster part compatibility issue.

At the application part you need to check your query and make sure it doesn’t use any removed feature like scan and top level filter. Also check your importer / indexer application if you have one and make sure it is compatible. I needed to do some change to our importer to make it work with 5.x.

Site plugin is not working on 5.x , if you use plugin like kopf use cerebro instead.

Prepare your cluster

After you have complete laundry list, it is now time to prepare your cluster for upgrade maintenance. Read upgrade document for reference. Prepare your new configuration (including used java env variable), disable shard auto allocation and backup your index. You will need Java 8 so make sure your operating system have the right package for you.

Upgrading from 2.x to 5.x need full cluster restart

Upgrade

Now when you’re ready for maintenance then lets get started. The order is up to you, but this is what I prefer :

  1. Download the latest version package
  2. Install Java 8
  3. Stop es instance
  4. Install the new package
  5. Put the new configuration
  6. Remove logging.yml (if you have)
  7. Remove unused site plugin
  8. Monitor the log
  9. Release the beast

Finally

Upgrading software stack never been easy especially if you have many application depending on it. However with good planning and simulating in staging environment it will go smoothly.

Since this upgrade requires doing full cluster restart it is better to do all of that things in new instance if you can afford it. Using new instance instead of upgrading the running system will allow us to achieve zero downtime.

 

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s