Problem
To increase the performance in Elasticsearch, many times we add new nodes to Elasticsearch cluster and manually change the configuration file with new IP address, etc. so that it gets added into the cluster. This takes up our time too. To make it efficient, we deceided to use AWS EC2 discovery plugin which discovers the Elasticsearch EC2 instances automatically through the host provider ‘ec2’. This plugin made our task so simple. The below changes are first tested in staging and then carefully executed in production without any issues. It takes less than an hour to install AWS EC2 plugin with the upgrade.
The discovery-ec2 plugin allows Elasticsearch to find the master-eligible nodes in a cluster running on AWS EC2 by querying the AWS API for the addresses of the EC2 instances running these nodes.
Install the ElasticSearch EC2 Discovery Plugin
$ cd /home/ec2-user/elasticsearch-7.2.0/bin
$ sudo ./elasticsearch-plugin install discovery-ec2
Enabling EC2 discovery
To enable EC2 discovery, configure Elasticsearch to use the ec2 seed hosts provider in config/elasticsearch.yml:
discovery.seed_providers: ec2
Configuring EC2 discovery
EC2 discovery supports a number of settings. Some settings are sensitive and must be stored in the Elasticsearch keystore. For example, to authenticate using a particular access key and secret key, add these keys to the keystore by running the following commands:
$ bin/elasticsearch-keystore add discovery.ec2.access_key
$ bin/elasticsearch-keystore add discovery.ec2.secret_key
Configuring EC2 discovery
EC2 discovery supports a number of settings. Some settings are sensitive and must be stored in the Elasticsearch keystore. For example, to authenticate using a particular access key and secret key, add these keys to the keystore by running the following commands:
$ bin/elasticsearch-keystore add discovery.ec2.access_key
$ bin/elasticsearch-keystore add discovery.ec2.secret_key
In detailed discription is found in this reference
– https://blog.francium.tech/install-aws-ec2-discovery-plugin-in-elasticsearch-5a973348cad9
– https://www.elastic.co/guide/en/elasticsearch/plugins/current/discovery-ec2-usage.html#discovery-ec2-usage
- DevOps Certified Professionals (DCP)
- Site Reliability Engineering Certified Professionals (SRECP)
- Master in DevOps Engineering (MDE)
- DevSecOps Certified Professionals (DSOCP)
URL - https://www.devopsschool.com/certification/
My Linkedin - https://www.linkedin.com/in/rajeshkumarin
- Apache Lucene Query Example - April 8, 2024
- Google Cloud: Step by Step Tutorials for setting up Multi-cluster Ingress (MCI) - April 7, 2024
- What is Multi-cluster Ingress (MCI) - April 7, 2024
