How to use elastic search EC2 discovery plugin?

Problem
To increase the performance in Elasticsearch, many times we add new nodes to Elasticsearch cluster and manually change the configuration file with new IP address, etc. so that it gets added into the cluster. This takes up our time too. To make it efficient, we deceided to use AWS EC2 discovery plugin which discovers the Elasticsearch EC2 instances automatically through the host provider ‘ec2’. This plugin made our task so simple. The below changes are first tested in staging and then carefully executed in production without any issues. It takes less than an hour to install AWS EC2 plugin with the upgrade.

The discovery-ec2 plugin allows Elasticsearch to find the master-eligible nodes in a cluster running on AWS EC2 by querying the AWS API for the addresses of the EC2 instances running these nodes.

Install the ElasticSearch EC2 Discovery Plugin
$ cd /home/ec2-user/elasticsearch-7.2.0/bin
$ sudo ./elasticsearch-plugin install discovery-ec2

Enabling EC2 discovery
To enable EC2 discovery, configure Elasticsearch to use the ec2 seed hosts provider in config/elasticsearch.yml:
discovery.seed_providers: ec2

Configuring EC2 discovery
EC2 discovery supports a number of settings. Some settings are sensitive and must be stored in the Elasticsearch keystore. For example, to authenticate using a particular access key and secret key, add these keys to the keystore by running the following commands:

$ bin/elasticsearch-keystore add discovery.ec2.access_key
$ bin/elasticsearch-keystore add discovery.ec2.secret_key

Configuring EC2 discovery
EC2 discovery supports a number of settings. Some settings are sensitive and must be stored in the Elasticsearch keystore. For example, to authenticate using a particular access key and secret key, add these keys to the keystore by running the following commands:

$ bin/elasticsearch-keystore add discovery.ec2.access_key
$ bin/elasticsearch-keystore add discovery.ec2.secret_key

In detailed discription is found in this reference
– https://blog.francium.tech/install-aws-ec2-discovery-plugin-in-elasticsearch-5a973348cad9
– https://www.elastic.co/guide/en/elasticsearch/plugins/current/discovery-ec2-usage.html#discovery-ec2-usage

Rajesh Kumar

I’m Rajesh Kumar, a DevOps, SRE, DevSecOps, Cloud, and Platform Engineering expert passionate about sharing practical knowledge, real-world experiences, and industry best practices. I have worked at Cotocus and regularly write about technology, travel, investing, health, product reviews, and digital marketing through my various platforms.

I publish technical articles at DevOps School, travel stories at Holiday Landmark, stock market insights at Stocks Mantra, health and fitness guidance at My Medic Plus, product reviews at TrueReviewNow, and SEO and digital marketing strategies at Wizbrand.