DOTNET: Capstone Project for DOTNET Performance Optimization

1. Recommended GitHub Application

Chosen app: NimblePros/eShopOnWeb

Repo: https://github.com/NimblePros/eShopOnWeb (GitHub)

Why this one:

• Official Microsoft reference app, now community-maintained by NimblePros.
• ASP.NET Core MVC app (classic web app) + PublicApi project (API you can treat as “microservice”).
• Uses Entity Framework Core with SQL Server support, with explicit steps to configure SQL Server in appsettings.json and apply EF migrations. (GitHub)
• Main branch is currently ASP.NET Core 9.0. By the time .NET 10 is out, updating target frameworks will be straightforward; the project already tracks latest .NET versions. (GitHub)

We’ll treat:

• src/Web = front-end web app
• src/PublicApi = microservice/API (this is what you’ll mainly hammer with k6, dotnet-counters, Datadog, etc.)

2. Capstone: High-Level Structure

Your “DOTNET Performance Optimization Capstone” will use eShopOnWeb and walk students through:

1. Setup & Run the App
   • Clone repo, configure SQL Server, run migrations, run Web + PublicApi.
2. Baseline & Live Health
   • Use dotnet-counters to watch CPU, GC, allocations, thread pool.
   • Problem it solves: “what’s happening right now in this process?”
3. CPU Profiling
   • Use dotTrace to find slow endpoints in PublicApi / Web.
   • Problem it solves: “where is the time going?”
4. Memory Profiling
   • Use dotMemory to detect high allocations, possible leaks.
   • Problem it solves: memory leaks, LOH bloat, allocation hotspots.
5. Deep Runtime Analysis
   • Use PerfView on traces to understand GC, CPU, allocations in detail.
   • Problem it solves: advanced GC/CPU investigation (“truth machine”).
6. Micro-benchmarks
   • Extract a core hot function into a separate class library and benchmark with BenchmarkDotNet.
   • Problem it solves: LINQ vs loops, Span vs array, etc.
7. Load Testing
   • Use k6 to generate load on PublicApi (simulate catalog browse / add to basket).
   • Observe impact via dotnet-counters, PerfView, Datadog.
8. Datadog Monitoring & Dashboard
   • Install Datadog Agent on Windows. (docs.datadoghq.com)
   • Install Datadog .NET Tracer for automatic instrumentation. (docs.datadoghq.com)
   • Create custom dashboard showing latency, errors, CPU, GC, SQL time.

I’ll now go step-by-step.

3. Step 0 – Prerequisites

On your Windows machine:

.NET & Tools

• .NET SDK: Latest current (today that’s .NET 9; use 9.x SDK).
• Visual Studio 2022 or Rider:
  • Workloads: ASP.NET and web development, .NET desktop development.
• SQL Server:
  • SQL Server Developer Edition or SQL Server Express (localdb is fine).
• Git.

Performance Tools

• dotnet-counters (dotnet tool if not already included): dotnet tool install -g dotnet-counters
• dotTrace & dotMemory: install via JetBrains Toolbox (or ReSharper Ultimate).
• PerfView: download PerfView.exe from the official GitHub releases page.
• BenchmarkDotNet: as a NuGet package (later, in your benchmark project).

Load Testing & Monitoring

• k6 (Windows):
  • Follow official guide: download MSI or use choco install k6. (Grafana Labs)
• Datadog account (trial is fine). (DEV Community)
• Datadog Agent for Windows:
  • Download & install the Agent (MSI) from Datadog or use the datadog-installer-x86_64.exe. (docs.datadoghq.com)
• Datadog .NET Tracer for automatic instrumentation of ASP.NET Core. (docs.datadoghq.com)

4. Step 1 – Clone & Run eShopOnWeb with SQL Server

4.1 Clone the repo

git clone https://github.com/NimblePros/eShopOnWeb.git
cd eShopOnWeb
Code language: PHP (php)

Solution files:

• eShopOnWeb.sln – main solution.
• Key projects under src/:
  • Web – ASP.NET Core MVC app.
  • PublicApi – ASP.NET Core Web API.

4.2 Configure SQL Server

The README has a “Configuring the sample to use SQL Server” section. (GitHub)

1. Open src/Web/appsettings.json.
2. Verify connection strings point to your local SQL Server, for example:

"ConnectionStrings": {
  "CatalogConnection": "Server=localhost;Database=CatalogDb;User Id=sa;Password=YourStrong!Passw0rd;TrustServerCertificate=True;",
  "IdentityConnection": "Server=localhost;Database=IdentityDb;User Id=sa;Password=YourStrong!Passw0rd;TrustServerCertificate=True;"
}
Code language: JavaScript (javascript)

For localdb:

"CatalogConnection": "Server=(localdb)\\mssqllocaldb;Database=CatalogDb;Trusted_Connection=True;MultipleActiveResultSets=true"
Code language: JavaScript (javascript)

3. Make sure "UseOnlyInMemoryDatabase" in appsettings.json is false (or remove the setting) so EF uses SQL Server. (GitHub)

4.3 Apply EF Core migrations

From the src/Web folder:

cd src/Web

dotnet restore
dotnet tool restore   # ensures dotnet-ef is available for this repo

dotnet ef database update -c catalogcontext -p ../Infrastructure/Infrastructure.csproj -s Web.csproj
dotnet ef database update -c appidentitydbcontext -p ../Infrastructure/Infrastructure.csproj -s Web.csproj
Code language: PHP (php)

This creates and seeds:

• Catalog database (products, baskets, etc.)
• Identity database (users, roles). (GitHub)

4.4 Run Web & PublicApi

You need both running:

Terminal 1 (PublicApi)

cd src/PublicApi
dotnet run
# Usually listens on https://localhost:5002 or similar (check console output)
Code language: PHP (php)

Terminal 2 (Web)

cd src/Web
dotnet run --launch-profile https
# Typically: https://localhost:5001
Code language: PHP (php)

Open browser:

• Store front: https://localhost:5001/
• Admin (Blazor) UI: https://localhost:5001/admin (GitHub)

Default seeded login (per README):

• demouser@microsoft.com / (check README or seed code for the password; often Pass@word1 or similar in the sample).

Now the app is running and ready for your capstone labs.

5. Step 2 – Live Runtime Health with dotnet-counters

Problem: “Is my service healthy right now? CPU, GC, allocations, thread pool?”

5.1 Find the process ID

Run Web + PublicApi. Then:

dotnet-counters ps

You’ll see something like:

12345  eShopOnWeb.Web
12360  eShopOnWeb.PublicApi
Code language: CSS (css)

5.2 Monitor PublicApi under light load

Start monitoring:

dotnet-counters monitor --process-id 12360 System.Runtime Microsoft-AspNetCore-Server-Kestrel
Code language: CSS (css)

Open browser and hit:

• https://localhost:5001/catalog
• Or call PublicApi endpoints directly (e.g. /api/catalogitems – check controllers).

Watch:

• CPU Usage (%)
• GC Heap Size
• Gen 0/1/2 collections / sec
• ThreadPool Thread Count
• Requests per second (from Kestrel counters)

5.3 Capture a short session

For a 60-second capture:

dotnet-counters collect --process-id 12360 --duration 60 --output publicapi-counters.json
Code language: CSS (css)

Capstone deliverable:
Students must:

• Capture baseline metrics (no load).
• Trigger manual browsing.
• Compare counters (CPU, allocations, GC) before & during load.
• Document “normal healthy baseline” for this app.

6. Step 3 – CPU Profiling with dotTrace

Problem: “Why is this endpoint slow? Where is the time going?”

6.1 Scenario to profile

Pick one PublicApi endpoint, for example:

• GET /api/catalogitems (list products).
• GET /api/catalogitems?pageIndex=10&pageSize=20 (heavier query).

6.2 Running dotTrace

1. Open dotTrace.
2. Choose:
   • Profile | .NET Core Application.
   • Target: src/PublicApi/bin/.../PublicApi.dll or launch via solution in VS/Rider with dotTrace integration.
   • Profiling type: start with Timeline or Sampling.
3. Start profiling.
4. While profiling, run:
   • Browser: call the endpoint repeatedly, or
   • k6 (later step) to generate load.
5. Stop profiling in dotTrace.

6.3 Analyze

In the dotTrace UI:

• Sort by “Hot Spots” or Call Tree → look at:
  • EF Core query methods (ToListAsync, Where, Include, etc.).
  • Any custom business methods doing heavy CPU.
• Identify:
  • Expensive LINQ queries.
  • Over-use of AutoMapper, string concatenation, etc.

6.4 Code optimization example

Students can apply simple optimizations like:

• Replace ToList() then filtering in memory with filtering in the database.
• Use AsNoTracking() on read-only EF queries.
• Cache reference data in memory if it’s safe.

Re-profile with dotTrace and show:

• Reduced total CPU time.
• Reduced time in specific methods.

Capstone deliverable:
Before/after screenshots of dotTrace with explanation of:

1. Hot path before optimization.
2. Code change.
3. Hot path after optimization (time reduced by X%).

7. Step 4 – Memory Profiling with dotMemory

Problem: “Why is memory/GC so high? Any leaks or heavy allocations?”

7.1 Start dotMemory session

1. Open dotMemory.
2. Attach to Web or PublicApi process.
3. Generate some load:
   • Browse catalog / admin.
   • Or run a small k6 test for 1–2 minutes.
4. Take a “Get Snapshot” in dotMemory.

7.2 Analyze snapshot

Look for:

• Top types by bytes (large arrays, strings, DTOs).
• Large Object Heap (LOH) usage.
• Retention paths:
  • Are large collections retained by static caches or DI singletons?

7.3 Optional: Introduce intentional tiny “leak”

For training, you can introduce a simple bug, e.g.:

public static List<string> DebugCache = new List<string>();

public Task SomeServiceMethod(...)
{
    DebugCache.Add(Guid.NewGuid().ToString());
    ...
}
Code language: PHP (php)

After traffic, memory will grow; dotMemory will show the static list retaining objects.

Capstone deliverable:

• Snapshot analysis: top 3 types by size.
• Explanation of whether memory usage is reasonable or suspicious.
• (Optional) detection & fix of intentional “leak,” with before/after snapshots.

8. Step 5 – Deep GC/CPU Analysis with PerfView

Problem: “I need full low-level view of GC, CPU, allocations, threads.”

8.1 Collect trace with PerfView

1. Run PerfView.exe as Administrator.
2. Collect → Collect…
3. Configure:
   • Check CPU Samples, .NET Alloc, Thread Time, GC.
4. In “Process Filter”, specify PublicApi;Web to avoid noise.
5. Click Start Collection.
6. Generate traffic (browser or k6).
7. Click Stop Collection.

PerfView creates a .etl.zip file.

8.2 Analyze

Open the trace:

• View → CallTree
  • Look at Inclusive CPU (%); identify top methods.
• View → GCStats
  • GC pause times.
  • Gen 2 / LOH activity.
• View → Events → GC/AllocationTick
  • Allocation hotspots per type/method.

Compare before and after your earlier optimizations:

• Less CPU in certain methods.
• Less GC time / fewer Gen 2 collections.
• Lower allocation rates.

Capstone deliverable:

• Short report (one slide or doc section) summarizing PerfView findings:
  • “Top 3 CPU-heavy methods.”
  • “GC characteristics under load.”
  • “Impact of optimization X on GC stats.”

9. Step 6 – Micro-benchmarks with BenchmarkDotNet

Problem: “Is optimization A actually faster than B? LINQ vs loop, etc.”

9.1 Create benchmark project

In solution root:

dotnet new console -n EShop.Benchmarks
cd EShop.Benchmarks
dotnet add package BenchmarkDotNet
Code language: JavaScript (javascript)

Edit Program.cs to something like:

using BenchmarkDotNet.Attributes;
using BenchmarkDotNet.Running;
using System.Linq;

public class CatalogFilterBenchmarks
{
    private readonly List<int> _data;

    public CatalogFilterBenchmarks()
    {
        _data = Enumerable.Range(1, 100_000).ToList();
    }

    [Benchmark]
    public int LinqWhereCount()
    {
        return _data.Where(x => x % 2 == 0).Count();
    }

    [Benchmark]
    public int ForLoopCount()
    {
        var count = 0;
        for (int i = 0; i < _data.Count; i++)
        {
            if (_data[i] % 2 == 0) count++;
        }
        return count;
    }
}

public class Program
{
    public static void Main(string[] args)
    {
        BenchmarkRunner.Run<CatalogFilterBenchmarks>();
    }
}
Code language: PHP (php)

Run:

dotnet run -c Release

You’ll get:

• Mean execution times.
• Std deviation.
• Allocations per operation.

9.2 Link to real code

For stronger capstone:

• Extract a real method from eShopOnWeb (e.g., a pricing calculation, filtering logic, mapping).
• Create CurrentImplementation vs OptimizedImplementation.
• Benchmark both with BenchmarkDotNet.

Capstone deliverable:

• Benchmark table showing:
  • Methods A vs B.
  • Time & allocations.
• Narrative: “We chose B because it’s 30% faster with 50% fewer allocations.”

10. Step 7 – Load Testing with k6

Goal: Generate load against PublicApi / Web, correlate with dotnet-counters, PerfView, Datadog.

10.1 Install k6 (Windows)

Follow the k6 docs (choco, MSI or binary). (Grafana Labs)

Verify:

k6 version

10.2 Create test script eshop-load.js

Example script hitting a catalog endpoint:

import http from 'k6/http';
import { check, sleep } from 'k6';

export let options = {
  vus: 50,
  duration: '2m',
  thresholds: {
    http_req_duration: ['p(95)<500'], // 95% < 500ms
    http_req_failed: ['rate<0.01'],   // < 1% errors
  },
};

const BASE_URL = 'https://localhost:5002'; // PublicApi HTTPS port

export default function () {
  let res = http.get(`${BASE_URL}/api/catalogitems?pageIndex=0&pageSize=20`);
  
  check(res, {
    'status is 200': (r) => r.status === 200,
  });
  
  sleep(1);
}
Code language: JavaScript (javascript)

If HTTPS/localhost causes TLS issues, you can run k6 with --insecure-skip-tls-verify.

10.3 Run the test

k6 run eshop-load.js
Code language: CSS (css)

While k6 runs, also:

• dotnet-counters monitor on PublicApi.
• Optionally, another PerfView trace capture.

Capstone deliverable:

• k6 output screenshots (RPS, latency, failures).
• Counters snapshot (GC, CPU).
• Discussion: “Under 50 VUs, 95p latency = X ms, error rate Y%.”

11. Step 8 – Datadog Agent & Dashboard

Now we’ll wire the app into Datadog APM + metrics.

11.1 Install Datadog Agent on Windows

1. Sign in to Datadog.
2. Go to Integrations → Agent → Windows.
3. Download the installer (or datadog-installer-x86_64.exe) and run it. (docs.datadoghq.com)
4. During setup, supply your Datadog API key.
5. After install:
   • Confirm service is running.
   • In Datadog UI, your host should appear under Infrastructure → Host Map.

11.2 Install Datadog .NET Tracer

Follow the “Tracing .NET Core Applications” documentation: (docs.datadoghq.com)

On Windows the common approach is:

1. Download & run the latest Datadog .NET Tracer MSI (from Datadog docs).
2. This sets environment variables and profiler DLLs system-wide.

Key environment variables (usually set by installer, but you can override per app):

DD_SERVICE=eshop-publicapi
DD_ENV=perf-lab
DD_VERSION=1.0.0
DD_AGENT_HOST=localhost
DD_TRACE_ENABLED=true
DD_LOGS_INJECTION=true
Code language: JavaScript (javascript)

For Web app:

DD_SERVICE=eshop-web
DD_ENV=perf-lab
DD_VERSION=1.0.0
DD_AGENT_HOST=localhost

Restart your Web & PublicApi processes after installing tracer / setting env vars.

11.3 Generate traffic and verify traces

1. Run Web + PublicApi.
2. Run your k6 test again.
3. In Datadog:
   • Go to APM → Services.
   • You should see services like eshop-publicapi, eshop-web appear.
   • Click into a service, open some traces, and inspect:
     • Request path.
     • DB spans (SQL statements, durations).
     • Error spans (if any).

Datadog docs for .NET tracing show how DB spans and HTTP spans are captured automatically via automatic instrumentation. (docs.datadoghq.com)

11.4 Create a Datadog Dashboard

In Datadog:

1. Dashboards → New Dashboard.
2. Add widgets:
   • Timeseries: avg:aspnet_core.request.duration{service:eshop-publicapi}
   • Timeseries: sum:aspnet_core.request.hits{service:eshop-publicapi} (RPS)
   • Query: sum:system.cpu.user{host:YOUR_HOST} (CPU).
   • aspnet_core.requests.errors for error rate.
   • If the tracer exposes GC metrics (via runtime integration), show:
     • GC pause time.
     • Gen 2 collections.
3. Add a Top List for:
   • top(avg:trace.http.request.duration{service:eshop-publicapi} by {resource})
   • (Shows slowest endpoints.)

Capstone deliverable:

• A screenshot of the Datadog dashboard.
• Explanation of key charts:
  • CPU vs request rate during k6.
  • 95p latency & error rate.
  • Slowest endpoints list.

12. How It All Ties Together (Capstone Storyline)

To help you present this as a coherent capstone, here’s the narrative:

1. Setup & Baseline
   • Run eShopOnWeb with SQL Server.
   • Confirm basic functionality.
2. Observe Live Health (dotnet-counters)
   • Identify baseline CPU, GC, allocations under light load.
3. Generate Load (k6)
   • Simulate real traffic to PublicApi.
4. Investigate CPU (dotTrace)
   • Profile slow endpoints.
   • Optimize EF queries / business logic.
5. Investigate Memory (dotMemory)
   • Look for high allocations or leaks.
   • Fix or justify memory patterns.
6. Deep-Dive (PerfView)
   • Verify GC/CPU behavior at low level.
   • Validate impact of optimization.
7. Micro-Benchmarks (BenchmarkDotNet)
   • Benchmark critical internal methods to choose best implementation.
8. End-to-End Observability (Datadog)
   • Visualize everything in one dashboard: latency, errors, CPU, GC, SQL.
   • Correlate k6 load, runtime counters, and APM traces.

By the end, students experience the full lifecycle:

Bug / performance problem → measurement (counters, traces, profilers) → hypothesis → code & DB change → benchmark → load test → monitor in Datadog.

Rajesh Kumar

I’m a DevOps/SRE/DevSecOps/Cloud Expert passionate about sharing knowledge and experiences. I have worked at Cotocus. I share tech blog at DevOps School, travel stories at Holiday Landmark, stock market tips at Stocks Mantra, health and fitness guidance at My Medic Plus, product reviews at TrueReviewNow , and SEO strategies at Wizbrand.

Do you want to learn Quantum Computing?

Please find my social handles as below;

Rajesh Kumar Personal Website
Rajesh Kumar at YOUTUBE
Rajesh Kumar at INSTAGRAM
Rajesh Kumar at X
Rajesh Kumar at FACEBOOK
Rajesh Kumar at LINKEDIN
Rajesh Kumar at WIZBRAND

Rajesh Kumar DailyLogs