How to use elastic search EC2 discovery plugin?

Problem
To increase the performance in Elasticsearch, many times we add new nodes to Elasticsearch cluster and manually change the configuration file with new IP address, etc. so that it gets added into the cluster. This takes up our time too. To make it efficient, we deceided to use AWS EC2 discovery plugin which discovers the Elasticsearch EC2 instances automatically through the host provider ‘ec2’. This plugin made our task so simple. The below changes are first tested in staging and then carefully executed in production without any issues. It takes less than an hour to install AWS EC2 plugin with the upgrade.

The discovery-ec2 plugin allows Elasticsearch to find the master-eligible nodes in a cluster running on AWS EC2 by querying the AWS API for the addresses of the EC2 instances running these nodes.

Install the ElasticSearch EC2 Discovery Plugin
$ cd /home/ec2-user/elasticsearch-7.2.0/bin
$ sudo ./elasticsearch-plugin install discovery-ec2

Enabling EC2 discovery
To enable EC2 discovery, configure Elasticsearch to use the ec2 seed hosts provider in config/elasticsearch.yml:
discovery.seed_providers: ec2

Configuring EC2 discovery
EC2 discovery supports a number of settings. Some settings are sensitive and must be stored in the Elasticsearch keystore. For example, to authenticate using a particular access key and secret key, add these keys to the keystore by running the following commands:

$ bin/elasticsearch-keystore add discovery.ec2.access_key
$ bin/elasticsearch-keystore add discovery.ec2.secret_key

Configuring EC2 discovery
EC2 discovery supports a number of settings. Some settings are sensitive and must be stored in the Elasticsearch keystore. For example, to authenticate using a particular access key and secret key, add these keys to the keystore by running the following commands:

$ bin/elasticsearch-keystore add discovery.ec2.access_key
$ bin/elasticsearch-keystore add discovery.ec2.secret_key

In detailed discription is found in this reference
– https://blog.francium.tech/install-aws-ec2-discovery-plugin-in-elasticsearch-5a973348cad9
– https://www.elastic.co/guide/en/elasticsearch/plugins/current/discovery-ec2-usage.html#discovery-ec2-usage

Rajesh Kumar

I’m a DevOps/SRE/DevSecOps/Cloud Expert passionate about sharing knowledge and experiences. I have worked at Cotocus. I share tech blog at DevOps School, travel stories at Holiday Landmark, stock market tips at Stocks Mantra, health and fitness guidance at My Medic Plus, product reviews at TrueReviewNow , and SEO strategies at Wizbrand.

Do you want to learn Quantum Computing?

Please find my social handles as below;

Rajesh Kumar Personal Website
Rajesh Kumar at YOUTUBE
Rajesh Kumar at INSTAGRAM
Rajesh Kumar at X
Rajesh Kumar at FACEBOOK
Rajesh Kumar at LINKEDIN
Rajesh Kumar at WIZBRAND

Rajesh Kumar DailyLogs