Elasticsearch Lab#3 – DSL Query Collection Part#1

Reference Sample Data & Query – https://github.com/devopsschool-demo-labs-projects/elasticsearch Rajesh KumarI’m a DevOps/SRE/DevSecOps/Cloud Expert passionate about sharing knowledge and experiences. I have worked at Cotocus. I share tech blog at DevOps School,…

Read more »

What is seq_no and primary_term in elasticsearch?

the _seq_no and _primary_term as parameter needed to implement the optimistic locking. Elasticsearch keeps tracks of the sequence number and primary term of the last operation to have changed each…

Read more »

Elasticsearch Lab#2 – Bulk indexing document using mget & bulk api

Sample Data Reference Sample Data & Query – https://github.com/devopsschool-demo-labs-projects/elasticsearch Rajesh KumarI’m a DevOps/SRE/DevSecOps/Cloud Expert passionate about sharing knowledge and experiences. I have worked at Cotocus. I share tech blog at…

Read more »

Elasticsearch Lab#1 – CRUD

Reference Sample Data & Query – https://github.com/devopsschool-demo-labs-projects/elasticsearch Rajesh KumarI’m a DevOps/SRE/DevSecOps/Cloud Expert passionate about sharing knowledge and experiences. I have worked at Cotocus. I share tech blog at DevOps School,…

Read more »

Example of Elastic Logstash pipeline input, filter and output

Rajesh KumarI’m a DevOps/SRE/DevSecOps/Cloud Expert passionate about sharing knowledge and experiences. I have worked at Cotocus. I share tech blog at DevOps School, travel stories at Holiday Landmark, stock market…

Read more »

Elasticsearch Cluster Setup and Configuration Tutorial step by step using AWS ec2

Rajesh KumarI’m a DevOps/SRE/DevSecOps/Cloud Expert passionate about sharing knowledge and experiences. I have worked at Cotocus. I share tech blog at DevOps School, travel stories at Holiday Landmark, stock market…

Read more »

Elasticsearch QUERY Parameters Examples with Explanation

Rajesh KumarI’m a DevOps/SRE/DevSecOps/Cloud Expert passionate about sharing knowledge and experiences. I have worked at Cotocus. I share tech blog at DevOps School, travel stories at Holiday Landmark, stock market…

Read more »

Elasticsearch-ELK Errors and Solutions Database

Elastic Search Error – Node is started with node.data=false, but has shard data Elastic Search Error – max virtual memory areas vm.max_map_count [65530] is too low, increase to at least…

Read more »

ElasticSearch Error – filebeat – filebeat.yml – did not find expected key

Errors Solutions Check filebeat.yml: Only one file beat output is supported …check #output.elasticsearch is comments Rajesh KumarI’m a DevOps/SRE/DevSecOps/Cloud Expert passionate about sharing knowledge and experiences. I have worked at…

Read more »

ElasticSearch Error – filebeat – filebeat.yml – more than one namespace configured accessing ‘output’

Errors Answer Check output.logstash in filebeat.yml: should be uncommented. Rajesh KumarI’m a DevOps/SRE/DevSecOps/Cloud Expert passionate about sharing knowledge and experiences. I have worked at Cotocus. I share tech blog at…

Read more »

ElasticSearch Error – this can cause resize pauses and prevents mlockall from locking the entire heap

Error Message Solutions Setting the parameter -Xms to the same value of -Xmx in /etc/elasticsearch/jvm.options solved the problem 🙂 Thanks Rajesh KumarI’m a DevOps/SRE/DevSecOps/Cloud Expert passionate about sharing knowledge and…

Read more »

Elasticsearch Command Lines and their Usage Guide

elasticsearch elasticsearch-certgen elasticsearch-certutil elasticsearch-cli elasticsearch-croneval elasticsearch-env elasticsearch-env-from-file elasticsearch-keystore elasticsearch-migrate elasticsearch-node elasticsearch-plugin elasticsearch-saml-metadata elasticsearch-setup-passwords elasticsearch-shard elasticsearch-sql-cli elasticsearch-sql-cli-7.6.0.jar elasticsearch-syskeygen elasticsearch-users x-pack-env x-pack-security-env x-pack-watcher-env Rajesh KumarI’m a DevOps/SRE/DevSecOps/Cloud Expert passionate about sharing knowledge…

Read more »

How to enable Elastic Search REST API Accessible using IP Address?

By default http transport and internal elasticsearch transport only listens to localhost. If you want to access Elasticsearch from the host other than localhost then try adding following configurations in…

Read more »
kibana-7-2-install-and-configuration-in-rhel-7-centos

Kibana 7.2 Install and configuration in RHEL 7/Centos

Install and Setup JDK 8+ Download and Install Kibana 7.X Configure Kibana 7.X How to start kibana service? Rajesh KumarI’m a DevOps/SRE/DevSecOps/Cloud Expert passionate about sharing knowledge and experiences. I…

Read more »
elasticsearch-plugin

How to use elastic search EC2 discovery plugin?

Problem To increase the performance in Elasticsearch, many times we add new nodes to Elasticsearch cluster and manually change the configuration file with new IP address, etc. so that it…

Read more »
elasticsearch-error-node

Elastic Search Error – Node is started with node.data=false, but has shard data

[2019-07-11T09:59:18,013][ERROR][o.e.b.Bootstrap ] [es-master01] Exception java.lang.IllegalStateException: Node is started with node.data=false, but has shard data: [/home/ec2-user/elasticsearch-7.2.0/data/nodes/0/indices/6XsNftnbSbOnzPK3wOKc 0w/0]. Use ‘elasticsearch-node repurpose’ tool to clean up at org.elasticsearch.env.NodeEnvironment.ensureNoShardData(NodeEnvironment.java:1065) ~[elasticsearch-7.2.0.jar:7.2.0] at org.elasticsearch.env.NodeEnvironment.(NodeEnvironment.java:324) ~[elasticsearch-7.2.0.jar:7.2.0] Solution…

Read more »
elastic-search-error-max-virtual-memory-areas

Elastic Search Error – max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]

Errors [2]: max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144] Solution To make it persistent, you can add this line: $ sudo vi /etc/sysctl.conf…

Read more »
elastic-search-error-max-max-file-descriptors

Elastic Search Error – max file descriptors [4096] for elasticsearch process is too low, increase to at least [65535]

Elastic Search Error – max file descriptors [4096] for elasticsearch process is too low, increase to at least [65535] Solution Temporary to the current login session To set ulimit value…

Read more »
sample-data-elastic-search

How to create sample data to perform searches using elastic search?

Step 1 – Go to: http://www.json-generator.com/ Step 2 – Download and save as customers_full.json Step 3 – Open customers_full.json in sublimetext Remove the array brackets Find-Replace },{ with }\n{ in…

Read more »
application-x-ndjson-and-application-json

What is difference between application/x-ndjson and application/json?

Lets understand what is json? JSON stands for JavaScript Object Notation JSON is a lightweight format for storing and transporting data JSON is often used when data is sent from…

Read more »
elasticsearch-error-bulk-request

Elastic Search Error – The bulk request must be terminated by a newline

FIX – Add a New line in customers_full.json Rajesh KumarI’m a DevOps/SRE/DevSecOps/Cloud Expert passionate about sharing knowledge and experiences. I have worked at Cotocus. I share tech blog at DevOps…

Read more »
elasticsearch-error-illegal

ElasticSearch Error – illegal_argument_exception – Rejecting mapping update to type

########### COMMAND ################# ########### OUTPUT ########### Solution Multiple mapping types are not supported in indices created in 6.0 The ability to have multiple mapping types per index has been removed…

Read more »
elasticsearch-error-content-type-header

ElasticSearch Error – Content-Type header [application/x-www-form-urlencoded] is not supported

To fix this, add curl option -H ‘Content-Type: application/json’. This error is due to strict content-type checking introduced in ElasticSearch 6.0 Solution Rajesh KumarI’m a DevOps/SRE/DevSecOps/Cloud Expert passionate about sharing…

Read more »

Understanding the Update API in Elasticsearch

The update API allows to update(also allows to delete, or ignore the operation). a document based on a script provided. The operation gets the document from the index, runs the…

Read more »

Understanding Elasticsearch Index Mapping & Schema

A schema is a description of one or more fields that describes the document type and how to handle the different fields of a document. The schema in Elasticsearch is…

Read more »

Find out elastic Search health using _cat/health

To check elasticsearch cluster health by using (CURL) and Cluster API “cat” provieded by elasticsearch The cat API print the info in JSON format, compact and aligned text which is…

Read more »

Understanding Elasticsearch Keywords and Terminology

Understanding Elasticsearch Keywords and Terminology Elasticsearch Terminology Description Cluster “A cluster is a collection of one or more nodes (servers) that together holds your entire data and provides federated indexing…

Read more »

Crash Course of Elasticsearch in 10 mins

What is Elasticsearch? Elasticsearch is fast, horizontally scalable open source search engine. It provides HTTP API for storing and indexing JSON documents and with default configuration it behaves a little…

Read more »

Elasticsearch Web References & Cheatsheet

Elasticsearch official documentation https://www.elastic.co/guide/en/elasticsearch/reference/6.x/index.html Very good Elasticsearch Guide on Github https://github.com/dzharii/awesome-elasticsearch Elasticsearch Official forum https://discuss.elastic.co/ Stackoverflow elasticsearch handlers http://stackoverflow.com/questions/tagged/elasticsearch Jolicode elasticsearch Cheatsheet http://elasticsearch-cheatsheet.jolicode.com/ Elasticsearch DevOpsSchool reference https://www.devopsschool.com/blog/category/elastic-stack/ https://www.devopsschool.com/tutorial/elastic/ Rajesh KumarI’m…

Read more »

Web GUI tools for Elasticsearch cluster

elasticsearch-head is a web front end for browsing and interacting with an Elastic Search cluster. There are multiple ways of running elasticsearch-head. Running as a plugin of ElasticSearch (this is…

Read more »