---

1. What is dotnet-trace?

dotnet-trace is a .NET Core/ .NET CLI diagnostic tool that collects runtime traces from your app using EventPipe.

Think of it as:

• ✅ A lightweight profiler you can run in production (no restart needed)
• ✅ A way to capture CPU, GC, exceptions, thread activity, etc.
• ✅ A tool that generates .nettrace files you can open in PerfView, Visual Studio, or convert to speedscope for flame charts.

It works for:

• .NET Core 3.0+
• .NET 5/6/7/8/9 apps
  On Windows / Linux / macOS.

---

2. When to use dotnet-trace (Use Cases)

Use dotnet-trace whenever you need to understand what your .NET process is doing internally:

🔹 Common scenarios

1. High CPU investigation
   • CPU 80–90% on production app
   • You want to know which methods are consuming CPU.
2. GC / Memory issues
   • Frequent GC
   • High allocation rate
   • Pauses impacting latency.
3. Random hangs / delays
   • Requests are slow, but traditional logging doesn’t tell you why.
4. Startup performance issues
   • App is slow to start; you want to see what happens during startup.
5. Comparing before vs after optimizations
   • Capture a trace before tuning,
   • Capture another trace after changes,
   • Compare CPU/GC patterns.
6. Lab / Training
   • Teaching .NET runtime internals
   • Showing real-world behavior of GC, threads, etc.

---

3. Where to use dotnet-trace

You can use it on:

• Local developer machine – debugging / demos
• Staging / UAT – reproduce production-like load and trace
• Production – short, targeted traces when you hit an incident
  • It’s sampling-based (for CPU) and event-based, so usually safe if you keep duration limited.

Works with:

• Kestrel apps (ASP.NET Core)
• Worker services, console apps
• Windows services, Linux systemd services
• Apps running in Docker / Kubernetes (attach using PID inside container).

---

4. How to install dotnet-trace

You already ran this, but let’s document it properly.

4.1 Prerequisites

• .NET SDK installed (you already have dotnet --version working).

4.2 Install as a global tool

dotnet tool install --global dotnet-trace
Code language: PHP (php)

If already installed and you want to update:

dotnet tool update --global dotnet-trace
Code language: PHP (php)

Verify:

dotnet-trace --version
dotnet-trace --help

---

5. Key Concepts (Before Commands)

A few basics:

• Process ID (PID) – you attach to a running process using --process-id <pid>.
• EventPipe – internal mechanism used to stream runtime events.
• Profiles – preconfigured sets of providers/events:
  • cpu-sampling
  • gc-collect
  • gc-verbose
• .nettrace file – the output that you later inspect or convert.

---

6. Basic Commands (dotnet-trace CLI)

6.1 List processes you can trace

dotnet-trace ps

Typical output:

12345   Orders.Api
30280   Payments.Worker
Code language: CSS (css)

Use the PID in collect.

---

6.2 Collect a basic trace from a running app

dotnet-trace collect --process-id 30280

Default output (e.g., trace.nettrace) will be generated in the current directory.
Press Ctrl + C to stop collection.

---

6.3 Save trace with custom filename

dotnet-trace collect \
  --process-id 30280 \
  --output C:\traces\ordersapi_2025-11-25.nettrace
Code language: CSS (css)

---

6.4 Use a profile (recommended)

CPU-focused trace:

dotnet-trace collect \
  --process-id 30280 \
  --profile cpu-sampling \
  --duration 00:02:00
Code language: CSS (css)

This:

• Focuses on CPU events
• Samples for 2 minutes

GC-focused trace:

dotnet-trace collect \
  --process-id 30280 \
  --profile gc-collect \
  --duration 00:01:00
Code language: CSS (css)

---

6.5 Start a process and trace it from the beginning

dotnet-trace collect -- dotnet Orders.Api.dll
Code language: CSS (css)

• -- separates dotnet-trace arguments from the child process command.
• dotnet-trace will exit with the app’s exit code.

---

6.6 List available profiles

dotnet-trace list-profiles
Code language: PHP (php)

You’ll see built-in profiles with descriptions.

---

6.7 Convert traces to another format (for flame charts)

Convert .nettrace → speedscope:

dotnet-trace convert trace.nettrace --format speedscope
Code language: CSS (css)

This produces something like:

trace.speedscope.json
Code language: CSS (css)

You can open that on speedscope.app by uploading the JSON file to visualize CPU flame charts.

---

6.8 Generate textual reports

dotnet-trace report trace.nettrace --report-type cpu
Code language: CSS (css)

Common report types:

• cpu
• gc-collect
• gc-verbose

Example:

dotnet-trace report trace.nettrace --report-type gc-verbose
Code language: CSS (css)

---

7. Step-by-Step: End-to-End Workflow (Basic)

Let’s build a simple “incident to analysis” flow.

Scenario: High CPU on Orders.Api

1. Identify PID dotnet-trace ps Assume you see: 30280 Orders.Api
2. Collect CPU trace for 2 minutes dotnet-trace collect \ --process-id 30280 \ --profile cpu-sampling \ --duration 00:02:00 \ --output C:\traces\orders_highcpu.nettrace
3. Convert to speedscope & analyze flame chartdotnet-trace convert C:\traces\orders_highcpu.nettrace --format speedscope This generates a .speedscope.json.
   Then:
   • Open browser → speedscope.app
   • Upload the file
   • Look at:
     • Top hot methods
     • Call stacks leading to those methods
     • Identify which controller / EF query / third-party library eats CPU.
4. Optional: Generate CPU text report dotnet-trace report C:\traces\orders_highcpu.nettrace --report-type cpu You get a summary of top CPU-consuming call stacks in text form.
5. Apply Fix → Re-run trace
   • Optimize that method / query
   • Repeat trace to verify improvement.

---

8. Step-by-Step: GC / Memory Analysis Workflow

Scenario: GC is frequently running and latency increases.

1. Capture GC-focused trace dotnet-trace collect \ --process-id 30280 \ --profile gc-verbose \ --duration 00:01:30 \ --output C:\traces\orders_gcverbose.nettrace
2. Generate GC report dotnet-trace report C:\traces\orders_gcverbose.nettrace --report-type gc-verbose
3. Understand the output (key elements)

Typical info:

• Number of Gen 0 / Gen 1 / Gen 2 collections
• LOH (Large Object Heap) allocation / collections
• Pause times
• Allocation patterns per thread or per call stack.

Things to look for:

• Very frequent Gen 2 collections → large survival → potential memory pressure
• Large LOH activity → big arrays / big strings (e.g., big JSON / images)
• Which methods are allocating heavily.

4. Fix candidates:

• Reduce allocations in hot loops
• Pool reusable objects
• Avoid large temporary lists/arrays
• Fix unbounded caching.

---

9. How to Understand dotnet-trace Results (Conceptually)

9.1 CPU Sampling

From CPU report and flame charts, you’ll see:

• Which methods appear on top (most time spent)
• Call chain: e.g.,
  Controller → Service → Repository → EF Core → SQL

Interpretation:

• The width (in flame chart) or top % (in report) = how expensive it is.
• If business logic / EF query dominates:
  • Optimize queries, add indexes, reduce synchronous blocking, etc.

---

9.2 GC/Memory

From GC reports:

• High number of collections in short duration → heavy allocations
• Long pause times → GC pauses affecting latency
• Many Gen 2 collections → objects living too long / large memory usage

You correlate:

• When GC happens vs request latency
• Allocation-heavy methods → fix them.

---

9.3 Using PerfView or Visual Studio

Although dotnet-trace is a CLI collector, the .nettrace file can be:

• Opened in PerfView
• Imported into Visual Studio Performance Profiler (depending on version)

In those tools, you get:

• Graphs for CPU over time
• GC, threads, exceptions
• Call tree, hot paths, etc.

---

10. Advanced Usage

10.1 Combining with Kubernetes / Docker

Inside a container:

1. Exec into the pod: kubectl exec -it orders-api-pod-xyz -- /bin/bash
2. Install dotnet-trace in the container (or via a base image).
3. List processes: dotnet-trace ps
4. Collect from the .NET process inside the container.

You can then copy the .nettrace file out using:

kubectl cp <namespace>/<pod>:/path/inside/container/trace.nettrace ./trace.nettrace
Code language: HTML, XML (xml)

---

10.2 Filtering Providers (Custom EventPipe configs)

For very advanced scenarios, you can specify providers manually with --providers, but most of the time, profiles are enough. If you want ultra-deep control:

• Custom providers (e.g., Microsoft-AspNetCore-Server-Kestrel, System.Net.Http)
• Specific keywords and levels for tracing.

Example (indicative):

dotnet-trace collect \
  --process-id 30280 \
  --providers "Microsoft-AspNetCore-Server-Kestrel:0xFFFFFFFF:4"
Code language: CSS (css)

(You’ll usually only do this when you exactly know which ETW/EventSource providers you want.)

---

10.3 Automating in a Script

You can create a script, e.g., capture_cpu_trace.ps1:

$pid = (dotnet-trace ps | Select-String "Orders.Api").ToString().Split(" ", [System.StringSplitOptions]::RemoveEmptyEntries)[0]

dotnet-trace collect `
  --process-id $pid `
  --profile cpu-sampling `
  --duration 00:02:00 `
  --output "C:\traces\orders_cpu_$((Get-Date).ToString('yyyyMMdd_HHmmss')).nettrace"
Code language: JavaScript (javascript)

This lets you quickly capture CPU traces without manually checking PID each time.

---

11. Advantages of dotnet-trace

✅ 1. No code changes required

You attach to running processes. No need to modify source/logging.

✅ 2. Low overhead (safe for short runs in production)

Designed for live diagnostics with reasonable performance overhead when used carefully.

✅ 3. Works everywhere .NET runs

Windows, Linux, macOS, Docker, K8s.

✅ 4. Deep runtime insights

CPU, GC, threads, allocations, exceptions, etc., from the runtime level.

✅ 5. Integrates with other tools

• PerfView
• Visual Studio profiler
• speedscope (flame charts)
• Other trace analysis tools that support .nettrace or converted formats.

✅ 6. Perfect for training / labs

You can demonstrate effect of code changes, GC settings, async vs sync, etc., visually.

---

12. Quick dotnet-trace Cheatsheet (Summary)

# List .NET processes
dotnet-trace ps

# Collect basic trace
dotnet-trace collect --process-id <pid>

# Collect CPU trace for 2 min
dotnet-trace collect --process-id <pid> --profile cpu-sampling --duration 00:02:00 --output cpu.nettrace

# Collect GC-verbose trace
dotnet-trace collect --process-id <pid> --profile gc-verbose --duration 00:01:00 --output gc.nettrace

# Run an app and trace it
dotnet-trace collect -- dotnet MyApp.dll

# List built-in profiles
dotnet-trace list-profiles

# Convert to speedscope
dotnet-trace convert cpu.nettrace --format speedscope

# CPU report
dotnet-trace report cpu.nettrace --report-type cpu

# GC-verbose report
dotnet-trace report gc.nettrace --report-type gc-verbose
Code language: CSS (css)

---