Elasticsearch is a distributed store for JSON documents, and almost everything surprising about it follows from a single property: the files it searches are immutable. An index is divided into shards, each shard is a Lucene index, and each Lucene index is a set of segments written once and never edited. A delete only writes a tombstone, an update writes a fresh document and hides the old one, and background merges reclaim the space later. Because a segment never changes, it can be moved onto cheaper storage or pushed into object storage and still searched from there — which is precisely what makes a multi-year retention window affordable rather than theoretical.
The mapping is the contract signed before any of that happens. Field types, whether a field is indexed at all, whether doc values exist for sorting and aggregating, and whether a keyword sub-field is available for exact matching are all fixed when the index is created; changing them means building a new index and reindexing behind an alias. For a log platform the mapping decides how many gigabytes a day costs. For an application search feature it decides what is matchable at all, because text fields pass through an analysis chain of character filters, a tokeniser and token filters, and the query is analysed the same way before the two are compared.
Reading is done through the JSON Query DSL, where the split between query context, scored with BM25, and filter context, which answers a cacheable yes or no, is the largest performance lever most teams never pull. Above search sits the aggregation framework — metric and bucket aggregations that nest — which is how the same corpus answers analytical questions. Operationally, primary shard count is fixed at creation, node roles decide what each node may do, and lifecycle policy, rollover and snapshots are what keep an index estate from growing without limit, whether the cluster runs on your own hardware or as a managed deployment.
Why this skill matters now
In Pune the money behind Elasticsearch usually comes from an obligation rather than a feature request. The banking, insurance and shared-services operations around Kharadi, Yerwada and Magarpatta keep log and audit data for a period a regulator chose, not one an architect negotiated, and the engineering question that follows is uncomfortable: what does a retained day cost, and will a query against month eleven still return before the auditor loses patience. That pushes a local batch straight into lifecycle tiers, searchable snapshots, shard arithmetic and restore rehearsal — the material that decides whether a long window is genuinely queryable or merely stored.
The product engineering side of the city pulls the same tool in the opposite direction. Consumer and business software teams in Baner, Balewadi, Viman Nagar and Kalyani Nagar are judged on relevance: names and addresses that arrive transliterated or in mixed script, synonyms nobody maintained, scoring a product owner disagrees with, and faceted browse that has to stay fast while the catalogue grows. Advertising and clickstream teams with engineering here add sustained ingest, where bulk sizing, refresh interval and rollover stop being configuration trivia.
Deployment splits along the same line and doubles the syllabus. Regulated estates run self-managed clusters with no outbound path, so node roles, discovery, rolling upgrades, heap sizing and recovering a red cluster are unavoidable skills. Product teams tend to sit in a managed deployment close to the Mumbai region and hire for mapping, relevance and cost instead. Local postings mirror the split precisely — Elasticsearch named beside Logstash, Beats and Kibana in one column, and beside Java or Spring in the other.