Private corporate batches, live online cohorts and 1-on-1 mentoring in pipelines that parse and enrich at scale — inputs, grok, filters, conditionals, outputs, persistent queues and throughput tuning — taught by a practitioner who runs it in production.
20 years across DevOps, SRE and Security · 10,000+ engineers trained · Trained teams at JPMorgan Chase, Verizon, Nokia and the World Bank
Principal DevOps Engineer & Architect
Rajesh teaches Logstash from the execution model outward: what a batch is, how pipeline workers process events in parallel, why filter order changes both correctness and throughput, and what @metadata is for. Sessions build real pipelines rather than snippets — file and Beats inputs with automatic config reload, grok patterns constructed incrementally against live sample data and then made efficient, dissect where grok is overkill, date filters that set the event's actual time rather than its ingest time, mutate for typing and renaming, and geoip and useragent enrichment. The harder material is covered directly: conditionals and multiple pipelines, multiline events and stack traces, the Elasticsearch output with data streams and index naming, persistent queues and the dead letter queue for durability, and the tuning work — workers, batch size, heap and monitoring — that decides whether a pipeline keeps up under load. The choice between a Beat processor, an Elasticsearch ingest pipeline and Logstash is treated as a design decision with stated criteria.
Twenty years across DevOps, SRE and Security, in principal and architect roles at PayPay, SoftwareAG, ServiceNow, JDA Software, Intuit, Adobe and others. He has trained engineers at JPMorgan Chase, Verizon, Nokia, the World Bank, VMware, Oracle, Mercedes-Benz and Airbus — more than 10,000 people personally. He teaches what he runs, not what he reads.
You are booked with a named engineer, and that is who turns up. Marketplaces and larger providers rotate whoever is free, so the person who sold you the agenda is rarely the person teaching it.
The same trainer is available for the next engagement, which matters when a team builds on what it learned last time.
Your batch is assigned a named trainer before it starts, and that is who teaches it. See the full faculty.
Engagements are matched on the tool, not the calendar. For Logstash that means a trainer who has run it in production — pipelines that parse and enrich at scale — inputs, grok, filters, conditionals, outputs, persistent queues and throughput tuning — rather than whoever is free that week. You are told who is teaching before you commit, and that person is on the discovery call that shapes the agenda.
Where a batch is large enough to need a second trainer, the pairing is declared up front. The lead trainer stays accountable for the syllabus and the assessment either way.
Principal DevOps Engineer & Architect
Twenty years across DevOps, SRE and Security in principal and architect roles at PayPay, SoftwareAG, ServiceNow, JDA Software, Intuit, Adobe, IBM/Emptoris, Ness, MindTree and Accenture. He has trained more than 10,000 engineers personally, at organisations including JPMorgan Chase, Verizon, Nokia, the World Bank, VMware, Oracle, Mercedes-Benz and Airbus. He teaches what he runs, not what he reads.
Teams of 8–30
Custom agenda, your timezone, onsite or online, NDA-friendly.
Request a quoteIndividual engineers
A private instructor and a curriculum built around your goal.
₹99,999Individuals who want peers
Scheduled batch, max 8 to 10 hours of live instruction.
₹34,999A private batch starts with a discovery call. We look at the stack you actually run — the CI system, the cloud, the constraints — and map the agenda onto it, so examples use your topology rather than a generic one.
Delivery is onsite at your premises, live online, or hybrid, scheduled around your release calendar rather than ours. Batches run 8 to 30 engineers.
Every attendee leaves with recordings, slides, lab repositories and a completion certificate. You receive an attendance and assessment report. Invoicing supports PO and GST.
Talk to us about a private Logstash batchLogstash is a server-side data processing pipeline. It takes events in from many sources, transforms them, and sends them out to one or more destinations — most commonly Elasticsearch, but equally a file, Kafka, an object store or another Logstash instance. It is part of the Elastic Stack alongside Elasticsearch and Kibana, and its role in that stack is the transformation tier: the place where an unstructured log line becomes a document with typed, named, queryable fields.
A Logstash configuration has exactly three sections and the whole tool follows from them. Inputs pull or receive events — from Beats, files, syslog, HTTP, a JDBC query, Kafka and around fifty other plugins. Filters transform each event: grok and dissect for parsing, mutate for renaming and type conversion, date for turning a timestamp string into the event's actual time, json and csv and kv for structured payloads, and geoip, useragent and translate for enrichment from external data. Outputs deliver the result, with conditionals allowing one pipeline to route different events to different destinations. Every event carries a set of fields plus @metadata, a namespace that exists during processing and is dropped before delivery.
Underneath that simple model sits the execution engine that determines whether a pipeline survives production. Logstash runs pipeline workers in parallel over batches, so filter ordering, batch size and worker count directly set throughput. Persistent queues trade disk for durability across restarts, and the dead letter queue captures events the output rejected instead of silently discarding them. Multiple pipelines can run in one process, and pipeline-to-pipeline communication lets you build a fan-in or fan-out topology rather than one monolithic configuration.
Log volume grows faster than the budgets that pay for it, and almost every cost lever sits in the ingest path: what you ship, what you parse, what you enrich, what you drop, and what shape the resulting document has. A team that cannot modify its pipelines confidently ends up indexing everything at full fidelity and paying for it in storage and cluster hardware.
The second reason is that parsing is where data quality is decided, permanently. An event that arrives with its timestamp unparsed, its numeric fields typed as strings and its source identifier under a different name from every other data source will never join, aggregate or correlate properly, and no dashboard built on top can fix it. Reindexing is possible but expensive, so the filter block is one of the highest-leverage pieces of configuration in an observability platform.
Third, Logstash is now one option among several rather than the default. Beats processors handle simple cases at the edge, Elasticsearch ingest pipelines handle many others inside the cluster, and OpenTelemetry collectors and Fluentd/Fluent Bit compete directly. Knowing when Logstash earns its memory footprint — heavy parsing, enrichment from external sources, buffering, fan-out, protocol translation — and when a lighter option is correct is exactly the judgement organisations need and rarely have.
What Logstash is for and where it belongs. The transformation tier of an ingest architecture, why a shipper and a processor are separate concerns, an honest comparison against Beats processors, Elasticsearch ingest pipelines, Fluentd and OpenTelemetry collectors, and the criteria for choosing between them.
Topics: Introduction to the course and its lab environment · Introduction to Logstash and the problem it solves · Logstash in the Elastic Stack: where it sits and what feeds it · Shipper versus processor: why the tiers are separate · Logstash versus Beat processors and Elasticsearch ingest pipelines · Logstash versus Fluentd, Fluent Bit and OpenTelemetry collectors · When Logstash earns its memory footprint, and when it does not · Typical ingest architectures and their trade-offs
Running Logstash properly from the start. Installation on Mac, Linux and Windows and from a container; the directory layout and the difference between logstash.yml, pipelines.yml and jvm.options; then the execution model — batches, pipeline workers, ordering — which is the mental model everything else in the course depends on.
Topics: Installing Logstash on Mac and Linux · Installing Logstash on Windows · Running Logstash in Docker and on Kubernetes · Directory layout and where configurations live · logstash.yml, pipelines.yml and jvm.options · JVM heap sizing for Logstash · Understanding the Logstash execution model · Pipeline workers, batch size and batch delay · Event ordering guarantees and what you actually get · Running from the command line versus as a service · Plugin management: list, install, update, remove · Offline plugin packs for restricted environments
The three-section configuration, built from nothing. Processing a first event from stdin to stdout, the event structure and its standard fields, stashing an event into Elasticsearch, then reading real log files with the file input, sincedb position tracking, and automatic configuration reload so the edit-test loop is fast.
Topics: The three sections: input, filter, output · Processing our first event from stdin to stdout · The event structure: fields, @timestamp, @version, tags · Stashing your first event into Elasticsearch · Codecs: plain, line, json, multiline · The file input, start_position and sincedb · Automatic config reload and the fast edit-test loop · Outputting events to a file · The stdout output and the rubydebug codec · Testing a configuration with --config.test_and_exit · Reading Logstash's own logs and interpreting startup errors
Getting data in from everywhere it lives. The Beats input as the standard shipper interface, file and syslog for hosts, HTTP for webhooks and APIs, JDBC for pulling from databases, Kafka for buffered high-volume streams — plus stitching several inputs and outputs into one pipeline and tagging events so filters can tell them apart.
Topics: The beats input and its role with Filebeat and friends · The file input at scale: many files, rotation and sincedb · Working with HTTP input for webhooks and APIs · Handling JSON input and the json codec · The syslog, tcp and udp inputs · The jdbc input for pulling from databases on a schedule · The kafka input for buffered, replayable streams · Stitching together multiple input and output plugins · Tagging and typing events at the input · add_field, tags and identifying an event's source · Input back-pressure and what happens when the pipeline stalls
The core skill. Building grok patterns incrementally against real sample data, the standard pattern library, writing custom patterns, and the performance work that stops a greedy pattern from consuming a whole core. Then dissect for fixed-position formats, and the parsers for already-structured payloads — json, csv, kv and xml.
Topics: Parsing logs with Logstash: the general approach · Grok syntax, semantics and the standard pattern library · Building a grok pattern incrementally against sample data · Finishing and hardening a grok pattern · Custom patterns and pattern directories · Grok performance: anchoring, greediness and pattern ordering · grok failure tags and handling unparsed events · The dissect filter and when it beats grok · The json filter and nested structures · The csv and kv filters · The xml filter · Parsing Apache and nginx access logs end to end · Parsing requests, response codes and byte counts into typed fields
Turning parsed strings into a usable document. Accessing and manipulating field values with the field reference syntax, mutate for renaming, converting and removing; then the date filter, which decides whether an event is stored at the time it happened or the time it was ingested; and enrichment with geoip, useragent, translate and lookups — plus @metadata for fields that must not survive to the output.
Topics: Accessing field values and the field reference syntax · The mutate filter: rename, convert, gsub, strip, remove_field · Type conversion and why string-typed numbers break aggregations · Formatting dates and the date filter · Setting the time of the event rather than of ingestion · Time zones, epoch formats and multiple date patterns · Geographical data enrichment with geoip · Parsing user agents · The translate filter and lookup dictionaries · The ruby filter for logic no plugin covers · Working with @metadata and transient fields · The fingerprint filter and deterministic document IDs · The drop and prune filters for volume control · Elastic Common Schema field naming
Structuring configuration so it stays maintainable. Conditional statements for branching filters and outputs, pipeline organisation across files, running multiple pipelines from one process, and pipeline-to-pipeline communication for fan-in and fan-out. Then multiline handling — the correct way at the shipper and the workable way in Logstash — and parsing stack traces with grok.
Topics: Introduction to conditional statements · Working with conditionals in filters and outputs · Comparison operators, regex matching and field existence tests · Conditional routing to different outputs · Splitting configuration across multiple files and load order · Running multiple pipelines with pipelines.yml · Pipeline-to-pipeline communication: the distributor and collector patterns · Handling multiline events · Handling multiline events the easy way, at the shipper · Parsing stack traces with grok · The split and aggregate filters for one-to-many and many-to-one events · Keeping a large configuration reviewable
Getting events out reliably. The Elasticsearch output in depth — hosts, authentication, data streams, index naming, templates and document IDs — plus the other outputs worth knowing. Then durability: memory versus persistent queues, the dead letter queue for events Elasticsearch rejected, retry behaviour, and what actually happens to in-flight events when Logstash restarts.
Topics: Sending processed events to Elasticsearch · Elasticsearch output: hosts, sniffing, authentication and TLS · Data streams versus time-based index naming · Index templates and managing the mapping from the pipeline side · Document IDs, deduplication and idempotent delivery · The file, kafka, s3 and http outputs · Conditional and multiple outputs from one pipeline · Memory queues versus persistent queues · Sizing and monitoring a persistent queue · The dead letter queue and reprocessing rejected events · Retry behaviour, back-pressure and at-least-once semantics · What happens to in-flight events on restart · Mapping conflicts and how they surface as rejections
Making a pipeline survive production. Finding the bottleneck rather than guessing at it; tuning workers, batch size and heap; the filter costs that dominate CPU; then the operational surface — the monitoring APIs, ingest lag, alerting, running Logstash on containers and Kubernetes, securing it, and upgrading it alongside the rest of the stack.
Topics: Finding the bottleneck: input, filter, output or downstream · Tuning pipeline workers, batch size and batch delay · JVM heap, garbage collection and off-heap usage · Filter cost: grok, ruby, geoip and where CPU goes · Horizontal scaling and load balancing across instances · Buffering with Kafka in front of Logstash · The node stats and hot threads monitoring APIs · Stack monitoring and pipeline visualisation · Detecting and alerting on ingest lag · Testing configurations in CI before deployment · Running Logstash in containers and on Kubernetes · Securing Logstash: TLS, credentials and the keystore · Upgrades and version compatibility with Elasticsearch · Common production failure modes and their signatures
We rebuild the agenda around the tools you actually run.
Request a custom agendaConstruct a pattern incrementally against a real unmatched log format, harden it against a hundred sample lines, then anchor it and measure the throughput improvement.
Take raw Apache access logs from file input through grok, date, geoip and useragent to typed, ECS-named documents in Elasticsearch, with unparsed lines handled explicitly.
Reassemble multi-line Java exceptions two ways — at the shipper and in Logstash — parse the exception class and message out, and compare the reliability of each approach.
Use conditionals and pipeline-to-pipeline communication to split a mixed event stream across three outputs, then refactor a monolithic config into a distributor pattern.
Enable persistent queues, force a mapping rejection into the dead letter queue, terminate Logstash under load, then prove exactly which events survived and reprocess the rest.
Profile a pipeline falling behind its input, locate the bottleneck from the monitoring APIs, tune workers, batching and filters, and prove the improvement with measurements.
Billed yearly at ₹9,996
Max 8 to 10 hours of live instruction
Full program, private instructor
8–30 engineers · custom agenda · onsite or online · PO and GST invoicing
Refunds. If we cancel or postpone a cohort, you get a full refund within 15 days. There is no money-back guarantee otherwise.
Terms. Course material remains licensed to the attendee. Read the terms.
Your data. We don't share it with third parties. Privacy policy.
Certificate of completion
Very good training session. Well explained from the basics to the complex concepts. Also tried to cover practicals and demos within the 3 hour sessions. The learning content and videos are of a great deal of help.
Basics explanation was exemplary from Rajesh where he dealt with complicated topics to be simple. Great learning stuff personally for me.
Very detailed explanation and has lots of patience in attending the questionnaire. Thanks again for your wonderful sessions.
Good discussion, helped us to understand different tools in SRE.
Got good lab sessions which kept the new DevOps tool learnings to the point and it helped a lot in my career.
I took Terraform training with the tutor named Mithilesh. I requested to tailor the course curriculum for my needs. He did an excellent job of showing me how to write the Terraform script per the instructions provided.
| What matters | YouTube + blogs | Generic online course | Freelance marketplace | DevOpsSchool |
|---|---|---|---|---|
| Named practitioner | No | Rarely | Varies per booking | Yes — same trainer each time |
| Production experience | Unknown | Unknown | Unverified | 20 years, named employers |
| Custom agenda | No | No | Sometimes | Built from your stack |
| Onsite delivery | No | No | Sometimes | Yes |
| Lab environment | None | Sandbox that expires | Varies | Your own cloud — skill goes with you |
| Assessment | None | Quiz | Rarely | Assignments + capstone per module |
| Per-attendee certificates | No | Sometimes | Rarely | Yes |
| Corporate invoicing | No | Limited | Varies | PO and GST |
| Post-training support | None | Forum, time-limited | None | Lifetime forum access |
Tell us the team, the stack and the timeline. You'll get a straight answer, not a sales sequence.
Talk to an advisorPrefer to call or email?
More ways to reach us on the contact page.