Azure Foundry Agent Service Tutorial: Architecture, Pricing, Use Cases, and Hands-On Guide for AI + Machine Learning

1. Introduction

What this service is

Foundry Agent Service is an Azure-managed service for building, configuring, and operating AI “agents” that can follow instructions, use tools, and (optionally) retrieve knowledge from enterprise data sources—all while integrating with Azure identity, governance, and monitoring.

Simple explanation (one paragraph)

If you want a chat assistant that does more than answer questions—such as looking up information, calling APIs, and producing consistent, auditable outcomes—Foundry Agent Service provides the managed “agent runtime” and lifecycle tools so you can create and run those agents in Azure instead of stitching everything together yourself.

Technical explanation (one paragraph)

Technically, Foundry Agent Service provides an agent orchestration layer that sits between your application and foundation models. It typically manages agent configuration (instructions, tools, knowledge connections), stateful conversations (threads/sessions), execution (tool calls, retrieval/grounding), and operational controls (authentication, logging/telemetry integration, governance). Your model usage and data access are mediated through Azure-based connections and policies, and you operate agents as project resources within Azure’s AI platform.

What problem it solves

Teams commonly struggle with: – Turning a model into a reliable system that can use tools, retrieve knowledge, and follow policies – Managing agent state (conversation history, memory, tool outputs) in a scalable and secure way – Integrating with Azure enterprise requirements: Microsoft Entra ID, RBAC, logging, network controls, cost management, and compliance

Foundry Agent Service addresses these by providing a managed way to build and run agents as first-class Azure AI resources.

Naming note (important): Microsoft’s AI platform branding has evolved (for example, “Azure AI Studio” vs “Azure AI Foundry”, and “Agent Service” vs “Agents”). Use “Foundry Agent Service” as the primary name, and verify the latest naming, feature availability, and GA/preview status in the official docs:
https://learn.microsoft.com/search/?terms=Foundry%20Agent%20Service%20Azure

2. What is Foundry Agent Service?

Official purpose

Foundry Agent Service exists to help you build AI agents that can: – Follow system-level instructions – Use tools (for example, calling APIs, functions, or connected services) – Optionally ground responses in enterprise data (for example, via retrieval) – Maintain conversation state in a managed way – Be operated securely in Azure with identity, governance, and monitoring

Because terminology and product boundaries can change, treat this as the practical “what it does” definition and verify the exact scope in the current product documentation: https://learn.microsoft.com/search/?terms=Foundry%20Agent%20Service%20overview

Core capabilities (practical view)

Common capabilities associated with an Azure-managed agent service include: – Agent definition: name, instructions, model selection, tool configuration – Stateful conversations: threads/sessions and message history – Tool calling orchestration: structured requests from the model to tools, and returning tool results – Knowledge grounding: retrieval from configured data sources (when supported) – Safety and policy integration: aligning outputs with content filtering/safety policies (often via Azure AI Content Safety or platform policies) – Operationalization: logging, tracing, and controlled deployment across environments

Verify in official docs which of the above are supported in your region, subscription, and service version.

Major components (conceptual model)

While exact terms vary, agent services usually include:

Component	What it represents	Why it matters
Project / workspace	A logical container for AI assets (agents, connections, evaluations)	Enables separation by team/app/environment
Agent	The configured AI “worker” (instructions + tools + model)	Reusable unit of behavior
Thread / session	A stateful conversation container	Enables multi-turn workflows
Messages	Inputs/outputs in a thread	Auditing and replay
Runs / executions	A single agent execution cycle (model + tools)	Orchestrates tool usage and final output
Connections	Links to Azure resources (models, data sources)	Centralizes credentials, governance

Service type

Foundry Agent Service is a managed AI agent orchestration service in Azure, aligned to the AI + Machine Learning category. It is not simply a model endpoint; it is the layer that coordinates tools, state, and policies around a model.

Scope: regional vs global, project-scoped vs subscription-scoped

In Azure, most AI services are regional and tied to a project/workspace or a resource in a resource group. In practice: – Expect agents to be created within an Azure AI project/workspace (project-scoped) – Expect operations and data residency to follow the region of the underlying AI project and connected resources

Because scope and region support can change, confirm for your environment: https://learn.microsoft.com/search/?terms=Azure%20AI%20Foundry%20regions%20availability

How it fits into the Azure ecosystem

Foundry Agent Service typically sits at the center of an “agentic” solution and integrates with: – Model providers in Azure (commonly Azure OpenAI models, and potentially other hosted models depending on platform support) – Enterprise data (commonly via Azure AI Search for retrieval, or storage/databases via tools) – Compute for tools: Azure Functions, Container Apps, AKS, App Service – Security: Microsoft Entra ID, Managed Identities, Key Vault, Private Link/VNet (where supported) – Observability: Azure Monitor, Log Analytics, Application Insights (depending on integration)

3. Why use Foundry Agent Service?

Business reasons

Faster time-to-value: agent building blocks (instructions, tools, state) reduce custom orchestration code
Consistency: standard agent definitions and projects help teams reuse patterns across apps
Governance: centrally managed connections and access control reduce shadow AI deployments
Safer enterprise rollout: easier to apply policies, monitor usage, and manage secrets

Technical reasons

Stateful interactions: built-in constructs for multi-turn conversations
Tool orchestration: structured tool calls reduce prompt-only “best effort” integrations
Composable design: connect models + data + tools with fewer moving parts than DIY frameworks
Azure-native integration: aligns with Entra ID, RBAC, and standard deployment patterns

Operational reasons

Observability-ready: clearer auditing of agent runs, tool usage, and failures (capabilities vary)
Environment separation: dev/test/prod projects with different connections and policies
Change control: agent versions/config changes can be managed like application artifacts

Security/compliance reasons

Centralized identity: prefer Entra ID and Managed Identities over scattered API keys
Least-privilege access: RBAC-based control to projects and connected resources
Data governance: clearer boundaries for where prompts, outputs, and tool data flows

Scalability/performance reasons

Managed scaling: you scale agent requests like an API rather than scaling custom orchestrators
Backpressure and limits: service and model rate limits can be handled more predictably (still plan for retries)

When teams should choose it

Choose Foundry Agent Service when: – You need an agent that calls tools (APIs, functions, workflows) – You need multi-turn state and consistent conversation handling – You must satisfy Azure enterprise controls (RBAC, network boundaries, monitoring) – You want a managed way to build agentic apps without assembling a full custom framework

When teams should not choose it

Consider alternatives when: – You only need single-shot text generation (a simple model endpoint may be enough) – You require fully offline/on-prem orchestration (self-managed frameworks may fit better) – You need features not yet supported in your region or compliance boundary (for example, strict residency constraints or missing private networking features—verify availability) – You want deep customization of agent internals beyond what a managed service exposes

4. Where is Foundry Agent Service used?

Industries

Financial services (policy Q&A, internal support, document workflows)
Healthcare and life sciences (clinical ops knowledge assistants—ensure compliance)
Retail and e-commerce (customer support automation, catalog helpers)
Manufacturing (maintenance copilots, SOP assistants)
Energy (field ops troubleshooting assistants)
Public sector (case worker guidance with strict governance)
Technology and SaaS (developer support, internal productivity copilots)

Team types

Application developers adding “agentic features” to products
Platform teams standardizing AI adoption with guardrails
Data/ML teams operationalizing retrieval and evaluation
Security and compliance teams governing model and data access
DevOps/SRE teams running production agent endpoints

Workloads and architectures

Internal copilots (HR, IT helpdesk, finance ops)
Customer service agent assist
Document processing pipelines with human-in-the-loop review
Workflow automation (ticketing, approvals, knowledge retrieval)
API orchestration and data enrichment

Real-world deployment contexts

Production: strong RBAC, audited tool calls, controlled data connections, monitoring/alerting, environment separation
Dev/test: rapid experimentation with sandbox data and smaller models; cost controls and quotas

5. Top Use Cases and Scenarios

Below are realistic scenarios where Foundry Agent Service is commonly a good fit. For each, assume the agent uses a supported model through Azure, and uses tools/knowledge connections where appropriate.

1) IT Helpdesk Triage Agent

Problem: L1 support is overwhelmed; many tickets are repetitive.
Why this service fits: Stateful conversation + tool calls to ticketing systems.
Example: Agent asks clarifying questions, checks outage status via an API tool, and drafts a ticket with structured fields.

2) Customer Support Order Lookup Agent

Problem: Customers ask “Where is my order?” and “Can I change my address?”.
Why this service fits: Tool calling to order systems; consistent instructions for safe actions.
Example: Agent calls an order-status API tool, returns ETA, and escalates if the order is in an exception state.

3) Policy and Compliance Q&A (Grounded)

Problem: Employees need accurate policy answers; hallucinations are risky.
Why this service fits: Retrieval/grounding from approved documents; auditable sources.
Example: Agent answers travel expense policy questions and cites the exact policy sections retrieved.

4) Incident Response Assistant (On-call Copilot)

Problem: During incidents, responders waste time finding runbooks and past incidents.
Why this service fits: Tool access to monitoring/knowledge systems; stateful threads for incident timeline.
Example: Agent fetches relevant runbooks, queries recent alerts, and summarizes likely root causes.

5) Sales Enablement Agent

Problem: Sales needs fast, accurate product answers tailored to customers.
Why this service fits: Grounded retrieval from approved collateral; tool to CRM for customer context.
Example: Agent pulls account notes and produces a tailored pitch and objection handling guide.

6) Developer Documentation Assistant

Problem: Engineers lose time searching internal docs and code examples.
Why this service fits: Retrieval from docs; tool to internal Git search service.
Example: Agent answers “How do I rotate secrets for service X?” with steps and links to internal docs.

7) Procurement and Vendor Risk Assistant

Problem: Vendor assessments require repeated checks against requirements.
Why this service fits: Tool calls to questionnaires; grounded retrieval from policy templates.
Example: Agent drafts a vendor risk summary and flags missing evidence for human review.

8) HR Onboarding Assistant

Problem: New hires ask the same onboarding questions.
Why this service fits: Grounding from curated onboarding knowledge; consistent tone and disclaimers.
Example: Agent provides day-1 checklist, benefits links, and helps schedule IT setup steps via tools.

9) Finance Close Assistant

Problem: Month-end close involves repetitive reconciliations and explanations.
Why this service fits: Tooling to pull ledger data; constrained instructions to avoid unauthorized actions.
Example: Agent fetches KPIs and drafts variance commentary with traceable calculations.

10) Field Service Troubleshooting Agent

Problem: Technicians need quick guidance for equipment faults.
Why this service fits: Grounded SOP retrieval; tool to query asset history.
Example: Agent asks for model/serial, fetches maintenance history, and recommends next steps with safety warnings.

11) Legal Contract Review Assistant (Assistive)

Problem: Lawyers need faster first-pass reviews.
Why this service fits: Tooling for clause library retrieval; auditable suggestions.
Example: Agent highlights risky clauses and suggests alternative language, clearly labeling as non-final.

12) Product Ops / Release Notes Generator

Problem: Release notes are time-consuming and inconsistent.
Why this service fits: Tool to pull merged PR titles/issue summaries; consistent templates.
Example: Agent produces customer-friendly release notes with categorized changes and links.

6. Core Features

Because feature availability can vary by region and release stage, verify each item against the official docs for Foundry Agent Service: https://learn.microsoft.com/search/?terms=Foundry%20Agent%20Service%20features

1) Managed agent definitions

What it does: Stores agent configuration (instructions, model choice, tool definitions, optional knowledge settings) as a managed resource.
Why it matters: Avoids hardcoding prompts/tool schemas in multiple apps.
Practical benefit: Teams can reuse and version agent behavior across environments.
Caveats: Versioning/publishing workflows vary—verify how changes propagate to production.

2) Stateful conversations (threads/sessions)

What it does: Maintains conversation context in a structured way.
Why it matters: Many real workflows need multi-turn clarification and memory.
Practical benefit: Better user experience; fewer “start over” interactions.
Caveats: State retention duration, storage location, and export/audit features vary—verify retention and compliance behavior.

3) Tool calling orchestration

What it does: Enables the model to request structured tool invocations; your system executes the tool and returns results.
Why it matters: Reliable integration with external systems (CRM, ERP, ticketing, databases).
Practical benefit: More deterministic workflows than prompt-only “please call this API”.
Caveats: You must secure and validate tool calls; never let the model execute privileged actions without authorization checks.

4) Connections to Azure resources

What it does: Centralizes references to model endpoints and data sources.
Why it matters: Credentials and governance can be managed once.
Practical benefit: Faster onboarding, consistent security posture.
Caveats: Connection types and auth modes differ—verify supported connectors.

5) Grounding / retrieval integration (when supported)

What it does: Augments responses using enterprise knowledge sources (commonly via a search index).
Why it matters: Reduces hallucinations and improves factuality.
Practical benefit: Answers can reference approved documents and up-to-date data.
Caveats: Retrieval quality depends heavily on chunking, indexing, and query tuning.

6) Safety controls integration

What it does: Applies content safety filtering and policy controls to prompts and outputs.
Why it matters: Required for many enterprise deployments.
Practical benefit: Reduce harmful content and policy violations.
Caveats: Safety filtering can block legitimate content; build user-friendly fallbacks and human escalation.

7) Evaluation and testing hooks (platform-dependent)

What it does: Helps test agent behavior with test sets, metrics, or structured evaluations.
Why it matters: Agents regress when prompts/tools change.
Practical benefit: More reliable releases, fewer production surprises.
Caveats: Evaluation features may live in the broader Azure AI Foundry platform rather than the agent runtime itself—verify exact location.

8) Enterprise identity (Entra ID) and RBAC alignment

What it does: Aligns access control with Azure identity and resource governance.
Why it matters: Avoids sharing long-lived API keys across teams.
Practical benefit: Centralized access reviews and auditing.
Caveats: Some integrations may still require keys; store them in Key Vault and use Managed Identity where possible.

9) Observability integration (logging/metrics)

What it does: Emits operational signals about requests, failures, and latencies (capabilities vary).
Why it matters: Production operations require visibility.
Practical benefit: Faster troubleshooting; SLO/SLA tracking.
Caveats: Beware logging sensitive prompts/outputs; configure redaction and access controls.

10) Environment separation and governance

What it does: Enables separate projects/resources per environment.
Why it matters: Prevents test data and dev prompts from affecting production.
Practical benefit: Safer deployments and cost controls.
Caveats: Ensure consistent configuration promotion (IaC) and avoid manual drift.

7. Architecture and How It Works

High-level architecture

At a high level: 1. Your application sends a user message to Foundry Agent Service. 2. The agent runtime determines what to do next based on instructions and context. 3. The runtime calls the configured model. 4. If the model requests a tool call, the runtime (or your app, depending on the pattern) invokes the tool and returns results. 5. The agent produces a final response, optionally grounded by data retrieval.

Request/data/control flow (typical)

Control plane: Create/update agents, configure tools, set connections, manage access.
Data plane: Send messages, run agent executions, receive responses, fetch run logs/messages.

Integrations with related services (common Azure patterns)

Model layer: Azure-hosted foundation models (commonly Azure OpenAI).
Official entry point for Azure OpenAI docs: https://learn.microsoft.com/azure/ai-services/openai/
Retrieval: Azure AI Search for indexing and retrieval.
Docs: https://learn.microsoft.com/azure/search/
Tool execution: Azure Functions / Container Apps / AKS / Logic Apps (depending on your design).
Secrets: Azure Key Vault. Docs: https://learn.microsoft.com/azure/key-vault/
Identity: Microsoft Entra ID + Managed Identity. Docs: https://learn.microsoft.com/entra/identity/
Monitoring: Azure Monitor + Application Insights. Docs: https://learn.microsoft.com/azure/azure-monitor/

Dependency services

Foundry Agent Service rarely exists alone. Plan for: – A model provider resource (often Azure OpenAI) with deployed models – A data source (optional) for grounding (often Azure AI Search + storage) – Compute for tools (optional) and secured connectivity to internal systems – Monitoring and log storage (Log Analytics workspace)

Security/authentication model (typical Azure implementation)

Human admin access: Entra ID sign-in to Azure portal / Azure AI Foundry portal
App-to-service auth: Entra ID (recommended) or API keys (where applicable)
Tool auth: Managed Identity from tool compute (Function/Container) to downstream resources (SQL, Storage, Key Vault)

Verify exactly which auth modes Foundry Agent Service supports in your tenant and region.

Networking model

Networking options vary by service maturity: – Public endpoints with strong auth (common default) – Private networking via Private Link / VNet integration (sometimes available, sometimes limited; verify)

Start with public endpoints + Entra ID and then harden to private networking if and when available for your compliance needs.

Monitoring/logging/governance considerations

Capture request counts, latency, error rates
Correlate agent run IDs with application request IDs
Decide whether to log prompts/outputs; if you do, implement:
PII redaction
restricted access (RBAC)
retention limits
encryption and audit trails

Simple architecture diagram (Mermaid)

flowchart LR
  U[User] --> A[Your App<br/>Web/API]
  A --> F[Foundry Agent Service]
  F --> M[Model Endpoint<br/>(e.g., Azure OpenAI)]
  F --> T[Tools<br/>(Functions/APIs)]
  F --> R[Retrieval Source<br/>(e.g., Azure AI Search)]
  F --> A
  A --> U

Production-style architecture diagram (Mermaid)

flowchart TB
  subgraph Client
    U[Users]
    B[Browser/Mobile]
    U --> B
  end

  subgraph AppTier["Application Tier (Your Subscription)"]
    APIM[API Management<br/>(optional)]
    WEB[App Service / Container Apps]
    FUNC[Azure Functions<br/>Tool Endpoints]
    KV[Key Vault]
    MON[App Insights / Azure Monitor]
  end

  subgraph AIFoundry["Azure AI Foundry (Project)"]
    FAS[Foundry Agent Service]
    CONN[Connections<br/>(Models/Data)]
  end

  subgraph DataTier["Data Tier"]
    AIS[Azure AI Search<br/>Index]
    STG[Storage Account<br/>(Docs)]
    SQL[(Azure SQL / Cosmos DB<br/>(optional))]
  end

  subgraph ModelTier["Model Tier"]
    AOAI[Azure OpenAI Resource<br/>(Deployed Model)]
  end

  B --> APIM --> WEB
  WEB --> FAS
  FAS --> AOAI
  FAS --> AIS
  WEB --> FUNC
  FUNC --> SQL
  FUNC --> KV
  WEB --> KV
  WEB --> MON
  FUNC --> MON

  STG --> AIS
  FAS --> CONN

8. Prerequisites

Account/subscription/tenant requirements

An active Azure subscription
Access to Azure AI Foundry / Foundry Agent Service in your tenant (availability varies)
If using Azure OpenAI as the model provider: an Azure OpenAI-enabled subscription/region (access requirements may apply)

Start here to locate the current official onboarding path: https://learn.microsoft.com/search/?terms=Azure%20AI%20Foundry%20get%20started

Permissions / IAM roles

You typically need: – Resource Group: Contributor (or Owner) to create resources – Azure AI project/workspace: permissions to create agents and connections – Model resource (Azure OpenAI): permissions to deploy and use models – Azure AI Search (optional): permissions to create index and query

Because role names differ across services and evolve, verify the required built-in roles in docs for: – Azure AI Foundry project access – Azure OpenAI RBAC (if using Entra-based auth) – Azure AI Search RBAC

Billing requirements

A subscription with billing enabled
Budget/alerts recommended for cost containment

CLI/SDK/tools

For this tutorial, you should have: – Azure CLI: https://learn.microsoft.com/cli/azure/install-azure-cli – Git (optional) – Python 3.10+ (optional, for scripting data upload) – Access to Azure portal and the Azure AI Foundry portal (often at https://ai.azure.com/ — verify in your tenant)

Region availability

Foundry Agent Service availability is region-dependent.
Your model (for example, Azure OpenAI deployments) is also region-dependent.

Verify region support before provisioning: https://learn.microsoft.com/search/?terms=Azure%20AI%20Foundry%20region%20availability

Quotas/limits

Plan for: – Model token and rate limits – Search query limits (if using Azure AI Search) – Agent/project quotas (if applicable)

Always check: – Quotas in Azure portal for the relevant service – Product documentation for service-specific caps

Prerequisite services (for the lab)

Minimum viable lab: – Resource Group – Azure AI Foundry project/workspace with Foundry Agent Service access – A supported model connection (commonly via Azure OpenAI)

Optional (recommended for grounding lab): – Azure AI Search – Storage account (for documents)

9. Pricing / Cost

Current pricing model (how to think about it)

Foundry Agent Service cost is typically a combination of: 1. Model usage costs (tokens, input/output) from the underlying model provider (commonly Azure OpenAI) 2. Tool execution costs (Azure Functions/Container Apps compute, database queries, API Management calls) 3. Retrieval costs (Azure AI Search indexing + query units; storage for documents) 4. Observability costs (Log Analytics ingestion, retention, Application Insights telemetry)

Whether Foundry Agent Service itself has an additional standalone charge (per agent, per run, per request) depends on its current SKU/GA status. Verify on the official pricing page (if available) or the Azure Pricing Calculator.

Official pricing sources

Azure Pricing Calculator: https://azure.microsoft.com/pricing/calculator/
Azure OpenAI pricing (official): https://azure.microsoft.com/pricing/details/cognitive-services/openai-service/
Azure AI Search pricing (official): https://azure.microsoft.com/pricing/details/search/

For Foundry Agent Service pricing specifically, use Microsoft documentation search to find the current pricing page (if published): https://learn.microsoft.com/search/?terms=Foundry%20Agent%20Service%20pricing

Pricing dimensions (common)

Tokens: input + output tokens per model call
Number of model calls per conversation (agents often do multiple calls per user request)
Tool calls: quantity and duration (compute time), plus downstream API costs
Retrieval queries: per request, potentially multiple queries per response
Index size and replicas/partitions for search
Logs: volume of telemetry and retention duration

Free tier

Some Azure services offer free tiers; Azure OpenAI typically does not include a general free tier.
Preview services sometimes have limited free usage, but do not assume this—verify in official docs.

Cost drivers (what actually increases your bill)

Multi-step agent runs
One “user question” can become: – model call to interpret request – retrieval query – model call to write grounded answer – one or more tool calls + follow-up model call
Large context windows
Long conversations and large retrieved document chunks increase input tokens.
High concurrency
Parallel agent runs may cause rate-limit retries and additional costs if not controlled.
Verbose logging
Logging full prompts/outputs can dramatically increase Log Analytics ingestion.

Hidden/indirect costs

Networking egress if tools call external endpoints or cross-region resources
Private networking features (Private Link) may add costs
CI/CD runners and test environments
Data preparation and indexing pipelines (compute)

Network/data transfer implications

Keep model, agent runtime, retrieval, and tool compute in the same region when possible.
Minimize cross-region calls to reduce latency and egress.

How to optimize cost

Use smaller/cheaper models for most traffic; reserve larger models for escalation paths
Apply retrieval only when needed; avoid retrieving for trivial questions
Limit maximum tool calls per run; add guardrails
Summarize or truncate conversation history intentionally
Use caching for stable answers and repeated retrieval queries
Configure log sampling and reduce prompt/response logging in production

Example low-cost starter estimate (non-numeric)

A “starter” setup usually includes: – One small chat model deployment – A single agent – Low-volume dev/test usage – Optional basic Azure AI Search tier for small documents

Your primary costs will likely be: – Model tokens – Search indexing/query (if used) – Minimal monitoring ingestion

Because prices vary by region/model/tier, build an estimate in the calculator using: – Expected daily users – Avg messages per session – Avg tokens per message (input + output) – Tool calls per message (if any) – Search queries per message (if any)

Example production cost considerations (non-numeric)

For production: – Provision for peak concurrency and higher token throughput – Add redundancy for retrieval and tools – Budget for monitoring retention, audit needs, and alerting – Expect higher per-request cost due to multi-step orchestration

10. Step-by-Step Hands-On Tutorial

This lab is designed to be beginner-friendly and low-risk. It focuses on creating a basic agent and testing it safely. Because portal workflows and API surfaces can change, you will copy the exact endpoint/code snippets from your Azure AI Foundry environment where available.

Objective

Create a “Policy Helper” agent using Foundry Agent Service in Azure, connect it to a chat model, test it in the Azure AI Foundry playground, and validate secure access patterns. Optionally, add a retrieval source using Azure AI Search.

Lab Overview

You will: 1. Create an Azure resource group. 2. Ensure you have a model endpoint available (commonly Azure OpenAI). 3. Create or open an Azure AI Foundry project that supports Foundry Agent Service. 4. Create an agent with clear instructions and a narrow scope. 5. Test the agent in the portal and validate behavior. 6. (Optional) Add grounding via Azure AI Search and validate citations/grounding behavior. 7. Clean up resources.

Step 1: Create a resource group

What you’ll do: Create a dedicated resource group so cleanup is easy.

Open a terminal and sign in:

az login
az account show

Set variables (edit location to a region that supports your services):

RG="rg-foundry-agent-lab"
LOC="eastus"   # change to a supported region
az group create -n "$RG" -l "$LOC"

Expected outcome: Resource group created successfully.

Verification:

az group show -n "$RG" --query "{name:name, location:location, provisioningState:properties.provisioningState}"

Step 2: Ensure you have a supported model endpoint (commonly Azure OpenAI)

What you’ll do: Confirm you have access to a deployed chat model that Foundry Agent Service can use.

You have two common paths:

Path A (existing Azure OpenAI resource)

In Azure portal, find your Azure OpenAI resource.
Confirm you have at least one deployed chat model in a supported region.

Path B (create Azure OpenAI resource)

Provisioning Azure OpenAI can require approval depending on your subscription. Follow the official docs: https://learn.microsoft.com/azure/ai-services/openai/how-to/create-resource

Expected outcome: You have a model deployment ready to use.

Verification checklist: – You know the resource name and region – You can see at least one deployment for a chat model – You have permissions to use it

Model names and availability change frequently. Choose a generally available chat-capable model in your region and subscription and verify it works in a simple test in the Azure OpenAI playground (if applicable).

Step 3: Create/open an Azure AI Foundry project with Foundry Agent Service

What you’ll do: Use Azure AI Foundry to create a project that will hold your agent.

Open the Azure AI Foundry portal (commonly): https://ai.azure.com/
If your tenant uses a different entry point, locate it via Azure portal or docs search: https://learn.microsoft.com/search/?terms=Azure%20AI%20Foundry%20portal
Create a new Project (or select an existing one dedicated to this lab).
Confirm the project is in the same region (or a compatible region) as your model and any data services.

Expected outcome: You can access a project dashboard where you can create agents and manage connections.

Verification: – You can navigate to sections like Agents, Connections, or Models (names vary). – You can create new assets inside the project.

Step 4: Create a connection to your model provider (if required)

What you’ll do: Ensure the project can use your model deployment.

In the project: 1. Go to Connections (or equivalent). 2. Add a connection to your model provider resource (commonly Azure OpenAI). 3. Select the specific deployment you want to use.

Expected outcome: Your project can reference the deployment when creating an agent.

Verification: – The connection shows as “Healthy/Connected” (wording varies). – You can see the model deployment as selectable when configuring the agent.

If you don’t see your deployment, it is usually due to region mismatch, missing permissions, or the deployment being in a different subscription. Fix those before proceeding.

Step 5: Create your first agent in Foundry Agent Service

What you’ll do: Create an agent with narrow scope and safe instructions.

Navigate to Agents (or equivalent).
Select Create agent.
Configure fields similar to: – Name: policy-helper – Instructions (example—keep it short and strict):
- Only answer questions about the “Contoso Travel Policy (Lab)”.
- If you don’t have enough information, ask a clarifying question.
- Do not invent policy rules.
- If asked for legal advice, respond that you are not a lawyer and recommend contacting the compliance team.
Choose your model deployment from the connection you created.
Save the agent.

Expected outcome: The agent is created and appears in your project.

Verification: – The agent details page shows the configured model and instructions. – You can open a test/playground experience for the agent.

Step 6: Test the agent in the portal (baseline behavior)

What you’ll do: Validate that instructions are followed before adding data sources.

In the agent playground/test UI: 1. Ask a question the agent should refuse (because we haven’t provided policy docs yet), for example:
“What’s the hotel reimbursement limit?”

Ask a question outside scope:
“Write me a poem about space.”

Expected outcome: – For policy-specific questions, the agent should ask for policy text or admit it lacks information (depending on how it’s designed). – For out-of-scope questions, the agent should refuse or redirect based on your instructions.

Verification tip: If it confidently invents policy limits, tighten the instructions and retest.

Step 7 (Optional, recommended): Add grounding with Azure AI Search

This step depends on whether your Foundry Agent Service experience supports adding a retrieval/grounding source directly. If your UI supports it, do the following.

7A) Create Azure AI Search

Create an Azure AI Search service in the same region (or a supported region):

Azure portal path: – Create a resource → Azure AI Search

Docs: https://learn.microsoft.com/azure/search/search-create-service-portal

Choose a cost-effective tier for a lab (verify tier availability and features).

Expected outcome: Search service created.

7B) Create an index and load a small policy document

There are multiple ways (portal, REST API, SDK). The simplest for beginners is portal-based import, but it depends on supported data sources.

A minimal approach: – Create a small text document named contoso-travel-policy.txt with a few bullet rules (no sensitive data). – Upload it into a storage account container. – Use supported ingestion/indexing flow to create an index.

Docs entry point: https://learn.microsoft.com/azure/search/search-what-is-azure-search

Expected outcome: You have an index containing your policy text.

7C) Connect the agent to the search index (grounding)

In your agent configuration: 1. Add a knowledge source / grounding data source. 2. Select your Azure AI Search index via a connection. 3. Configure retrieval parameters if exposed (topK, filters, etc.).

Expected outcome: The agent can answer policy questions using retrieved content.

Verification: Ask: “What’s the hotel reimbursement limit?”
– Expected: It answers using your uploaded policy text (and may show citations depending on UI/support).

If citations are not shown, you can still validate grounding by changing the policy text and verifying the answer changes accordingly.

Step 8: Validate access and least privilege (quick check)

What you’ll do: Confirm only authorized users can access the project/agent.

In Azure portal, review project/resource IAM: – Ensure only intended users/groups have access.
If your organization requires it: – Use a dedicated Entra ID group for AI project admins – Separate “agent developers” from “agent operators”

Expected outcome: Access is restricted to your lab account/team.

Validation

Use this checklist:

Validation item	How to verify	Expected result
Agent exists	Agent list in project	`policy-helper` visible
Baseline instruction following	Ask out-of-scope questions	Refusal/redirect per instructions
Grounding works (optional)	Ask policy question answered by uploaded doc	Answer matches doc; minimal hallucination
Permissions	Try from unauthorized account (if possible)	Access denied

Troubleshooting

Issue: “I can’t see Foundry Agent Service / Agents in the portal”

Common causes: – Feature not enabled in your tenant/region – You are using a project type that doesn’t support agents – Lack of permissions

Fix: – Verify availability and onboarding in docs search:
https://learn.microsoft.com/search/?terms=Foundry%20Agent%20Service%20availability – Confirm you are Owner/Contributor on the project/resource group.

Issue: “Model deployment not selectable”

Common causes: – Region mismatch – Missing permissions to the model resource – Unsupported model type for the agent runtime

Fix: – Ensure the model deployment is in a supported region and you have access. – Test the model in its own playground first (where applicable).

Issue: “Grounding returns irrelevant snippets”

Fixes: – Improve chunking and indexing strategy – Add metadata filters – Reduce topK or adjust query strategy – Ensure the document text is clean and unambiguous

Issue: “Costs are higher than expected”

Common causes: – Too many tool calls or retrieval steps per user message – Large conversation history included each time – Verbose logging to Log Analytics

Fix: – Limit tool calls, truncate/summarize history, reduce logging volume, and set budgets/alerts.

Cleanup

To avoid ongoing charges, remove lab resources.

If you created a dedicated resource group for everything, delete it:

az group delete -n "$RG" --yes --no-wait

If you used shared resources (existing Azure OpenAI), only remove: – The agent and project assets created for the lab – Optional Azure AI Search and storage resources created specifically for this lab

Expected outcome: No billable lab resources remain.

11. Best Practices

Architecture best practices

Design for “agent steps”: Assume one user input may trigger multiple model calls, retrieval calls, and tool calls.
Separate concerns:
Agent: reasoning + orchestration
Tools: deterministic actions and data access
Retrieval: factual grounding
Fail safely: If a tool fails, return a safe error and ask the user for next steps rather than guessing.
Prefer structured outputs for tool calls and intermediate steps.

IAM/security best practices

Use Microsoft Entra ID for human and workload identity where supported.
Use Managed Identity for tool compute to access Key Vault, Search, Storage, databases.
Apply least privilege to:
project access
model usage
search query permissions
tool endpoints and downstream APIs
Separate roles:
Agent developers (can edit instructions/tools)
Operators (can view health/metrics)
Security reviewers (can audit logs)

Cost best practices

Choose the smallest model that meets quality targets; route complex queries to larger models.
Implement:
token limits
max tool calls
timeouts and retries with jitter
Cache stable results (policy answers, product specs) where allowed.
Use budgets, alerts, and tagging for chargeback.

Performance best practices

Keep agent, model, retrieval, and tools in the same region.
Reduce token usage:
concise system instructions
summarize long threads
keep retrieved chunks small but meaningful
Tune retrieval:
relevant fields
filters
topK based on latency targets

Reliability best practices

Build idempotent tools (safe to retry).
Implement circuit breakers for downstream systems.
Provide graceful degradation:
If retrieval fails, respond with “I can’t access the knowledge base right now.”
If tool fails, offer manual steps or escalation.

Operations best practices

Track:
request rate
latency
tool error rate
retrieval latency
model throttling/rate limiting
Add run correlation IDs across app logs and agent runs.
Establish incident playbooks for:
rate limit spikes
tool endpoint outages
search index failures

Governance/tagging/naming best practices

Tag resources: app, env, owner, costCenter, dataClass
Naming conventions:
Resource group: rg-{app}-{env}-{region}
Agent: {app}-{purpose}-{env}
Promote configuration through environments using IaC when supported; avoid manual edits in production.

12. Security Considerations

Identity and access model

Prefer Entra ID authentication and RBAC.
Restrict who can:
create/update agents (prompt/tool changes are effectively “code changes”)
create connections (these can unlock data/model access)
read logs (may contain sensitive content)

Encryption

Azure services encrypt data at rest by default in most cases; confirm for each connected service:
Azure AI Search
Storage accounts
Log Analytics
Any state stores used by agent workflows (if applicable)

Verify encryption and customer-managed key (CMK) options where needed: https://learn.microsoft.com/azure/security/fundamentals/encryption-overview

Network exposure

Start with public endpoints only if:
strong auth is enforced (Entra ID)
IP restrictions are applied at ingress layers (APIM, App Gateway) where appropriate
For regulated environments:
Evaluate Private Link/VNet integration availability for Foundry Agent Service and dependencies (verify support)
Keep data sources private where possible

Secrets handling

Store secrets in Azure Key Vault.
Avoid embedding keys in:
agent instructions
code repositories
CI logs
Use Managed Identity to fetch secrets at runtime.

Audit/logging

Decide what to log:
For debugging: sample prompts/outputs in dev only
For production: metadata + redacted content
Restrict log access with RBAC.
Implement retention policies and deletion workflows aligned with compliance.

Compliance considerations

Data classification: determine if prompts and outputs may contain PII/PHI.
Residency: ensure region selection meets your legal requirements.
Vendor risk: document where data flows (model provider, retrieval store, tool endpoints).

Common security mistakes

Allowing the model to call tools that perform privileged actions without authorization checks
Logging full prompts/outputs containing PII to shared workspaces
Using one shared API key across many apps/teams
Connecting agents directly to production databases without a read-only, audited API layer

Secure deployment recommendations

Put tools behind an authenticated API layer (APIM, OAuth, mTLS if needed).
Validate tool call parameters server-side.
Use allowlists for tool actions (for example, read-only vs write).
Add human approval for high-risk actions (refunds, account changes).

13. Limitations and Gotchas

Because the service evolves, treat these as common gotchas and verify specifics in official docs.

Known limitations (typical for managed agent services)

Region availability: may be limited to certain regions.
Feature gaps: some features may be preview-only or not available in sovereign clouds.
Networking: Private Link/VNet support may be limited or require specific configuration.
Observability: full tracing of tool calls and retrieval steps may require additional setup.

Quotas

Model rate limits and token throughput limits often dominate.
Search service quotas apply if using retrieval.
Agent/project limits may exist (number of agents, runs, threads)—verify.

Regional constraints

Your agent runtime, model resource, and retrieval should be co-located when possible.
Some model deployments are only available in certain regions.

Pricing surprises

Agents can multiply costs by doing multiple calls per user message.
Retrieval + tool calls add costs beyond model tokens.
Logging and retention can become significant.

Compatibility issues

Some models may not support tool calling consistently.
Some tool schemas may cause unreliable tool selection if too complex.

Operational gotchas

Downstream tool failures can cause user-visible agent failures unless you handle errors gracefully.
Silent policy blocks (content safety) can appear as “empty” or refused outputs—monitor for safety-trigger rates.

Migration challenges

If you started with an open-source agent framework, migrating to a managed service may require:
refactoring tool interfaces
changing how state is stored
adapting to platform evaluation/monitoring tooling

Vendor-specific nuances

Azure RBAC, Entra ID integration, and enterprise governance are strengths—but they require careful role design and operational discipline.

14. Comparison with Alternatives

Nearest services in Azure

Azure OpenAI (direct model endpoints): best for simple chat/completions without managed agent orchestration.
Azure AI Search + custom RAG: best when you want full control over retrieval and prompting.
Microsoft Copilot Studio: best for low-code business copilots with connectors and governance (different target audience).
Azure Bot Service: best for traditional bot channel integration; you can still use agents/models behind it.

Nearest services in other clouds

AWS Bedrock Agents: managed agent orchestration in AWS.
Google Vertex AI Agent Builder: managed agent-building experience in Google Cloud.
OpenAI Assistants API (non-Azure): similar concepts, but different governance and Azure integration story.

Open-source / self-managed alternatives

Semantic Kernel, LangChain, LlamaIndex, AutoGen: flexible frameworks for building agents, but you operate orchestration, state, security integration, and monitoring yourself.

Comparison table

Option	Best For	Strengths	Weaknesses	When to Choose
Foundry Agent Service (Azure)	Enterprise-grade agent apps with Azure governance	Managed agent lifecycle, Azure integration, project-based organization	Availability and features may vary; platform constraints	When you want managed orchestration + Azure controls
Azure OpenAI (direct)	Simple chat, single-turn generation	Simpler architecture, fewer moving parts	You build state + tools orchestration yourself	When you don’t need a managed agent runtime
Custom RAG (Azure AI Search + app logic)	Highly customized retrieval and prompting	Maximum control, transparent retrieval pipeline	More engineering, more ops	When you need bespoke retrieval logic and full control
Copilot Studio	Business users, low-code copilots	Fast to build, connectors, governance	Less code-level control; different runtime model	When low-code is primary requirement
AWS Bedrock Agents	AWS-first organizations	Managed agents in AWS ecosystem	Cross-cloud governance complexity	When your workloads and governance are AWS-centric
Vertex AI Agent Builder	Google Cloud-first organizations	Integrated agent building in GCP	Cross-cloud governance complexity	When your workloads and governance are GCP-centric
Semantic Kernel / LangChain (self-managed)	Teams needing portability and customization	Flexible, open ecosystem	You own security, scale, state, monitoring	When you need maximum portability and control

15. Real-World Example

Enterprise example: Bank internal policy copilot with audited tool calls

Problem: Employees need accurate answers about policies (travel, procurement, security). Hallucinations and unauthorized data access are unacceptable.
Proposed architecture:
Foundry Agent Service hosts the policy agent
Model connection to an approved enterprise model deployment
Grounding via Azure AI Search indexing approved policy documents stored in a secured storage account
Tools:
- Read-only API to policy exception workflow status
- Ticket creation tool for compliance questions (human review)
Security:
- Entra ID group-based access to the agent
- Managed Identity from tools to Key Vault and databases
Monitoring:
- Azure Monitor dashboards for usage, failures, safety blocks
Why Foundry Agent Service was chosen:
Managed state and tool orchestration reduces custom code
Strong alignment with Azure IAM/governance and enterprise monitoring
Expected outcomes:
Reduced compliance team load from repetitive questions
Better accuracy due to grounding
Auditable trail of tool calls and data access patterns

Startup/small-team example: E-commerce support agent with order lookup

Problem: A small support team can’t handle growing ticket volume; they need automated order status responses and returns guidance.
Proposed architecture:
Foundry Agent Service for agent configuration and state
Model connection to a cost-effective chat model deployment
Tool: HTTPS function to query the order system (read-only), behind API Management
Optional retrieval: index the FAQ and returns policy in Azure AI Search
Basic monitoring and budget alerts
Why Foundry Agent Service was chosen:
Avoid building custom orchestration and state management
Faster iteration on agent instructions and behavior
Expected outcomes:
Faster first response time
Fewer repetitive tickets
Controlled costs by limiting tool calls and using a smaller model

16. FAQ

Is Foundry Agent Service the same as calling a chat model directly?
No. Calling a model directly gives you text generation. Foundry Agent Service focuses on agent configuration, stateful interactions, and orchestrating tools/grounding around model calls.
Do I need Azure OpenAI to use Foundry Agent Service?
Often, a model provider is required, and Azure OpenAI is a common choice. Supported model providers depend on your Azure AI Foundry environment—verify in official docs.
Can I run agents privately (no public internet)?
It depends on whether Foundry Agent Service supports Private Link/VNet integration in your region and SKU. Verify current networking features and design with APIM + private tools/data sources.
Where is conversation state stored?
Managed agent services typically store thread/session state as part of the service. Retention and residency must be verified in the official documentation for compliance.
Does it support grounding with my documents?
Many Azure agent patterns use Azure AI Search for retrieval/grounding. Whether it’s integrated directly in your Foundry Agent Service UI/API depends on feature availability—verify.
How do I prevent hallucinations?
Use grounding with curated sources, constrain instructions, require citations, and design the agent to ask clarifying questions when data is missing.
How do I keep the agent from performing dangerous actions?
Put privileged actions behind secured tools, validate requests server-side, enforce allowlists, and require human approval for high-risk steps.
What is the biggest production risk?
Uncontrolled tool access (security) and unpredictable cost (multi-step runs) are two of the most common risks.
How do I estimate cost?
Estimate model tokens per user message multiplied by the average number of model calls per run, then add retrieval queries and tool compute costs. Use Azure Pricing Calculator.
Should I log prompts and outputs?
In production, avoid logging sensitive content unless necessary. If you must, redact PII, restrict access, and define retention policies.
How do I do environment separation (dev/test/prod)?
Use separate Azure AI Foundry projects and separate model deployments and data connections per environment.
Can multiple apps share one agent?
Yes, but it can complicate change management and access control. Many teams prefer per-app agents or strict versioning/promotion processes.
Does Foundry Agent Service execute my tools automatically?
Tool execution patterns differ: some platforms orchestrate tool calls but you still host the tool endpoint. Verify how tool invocation and authentication are implemented.
What happens if my tool endpoint is down?
The agent run may fail or degrade. Implement retries, fallbacks, and clear user messaging, and monitor tool error rates.
How do I roll out prompt/tool changes safely?
Treat agent configuration as code: use approvals, staged rollouts, test suites, and monitoring for regressions.
Can I use it for regulated data (PII/PHI)?
Possibly, but only after validating data handling, residency, retention, and compliance controls for every connected service (model, retrieval, logs). Engage your compliance team early.

17. Top Online Resources to Learn Foundry Agent Service

Use the table below as a curated starting point. Because naming and URLs can evolve, the Microsoft Learn search links are included as stable entry points.

Resource Type	Name	Why It Is Useful
Official documentation (search)	Microsoft Learn search: Foundry Agent Service	Fastest way to find the current overview, quickstarts, and API references: https://learn.microsoft.com/search/?terms=Foundry%20Agent%20Service%20Azure
Official portal	Azure AI Foundry portal	Where you create projects, connections, and agents (tenant-dependent): https://ai.azure.com/
Official docs (Azure OpenAI)	Azure OpenAI documentation	Model deployments, auth, quotas, pricing considerations: https://learn.microsoft.com/azure/ai-services/openai/
Official pricing	Azure OpenAI pricing	Model token pricing (key input to agent cost): https://azure.microsoft.com/pricing/details/cognitive-services/openai-service/
Official pricing	Azure AI Search pricing	Retrieval cost driver: https://azure.microsoft.com/pricing/details/search/
Pricing tool	Azure Pricing Calculator	Build a scenario-based estimate: https://azure.microsoft.com/pricing/calculator/
Official docs (Search)	Azure AI Search documentation	Indexing, retrieval tuning, security: https://learn.microsoft.com/azure/search/
Official docs (Identity)	Microsoft Entra ID documentation	Auth and enterprise access control patterns: https://learn.microsoft.com/entra/identity/
Official docs (Key Vault)	Azure Key Vault documentation	Secret management best practices: https://learn.microsoft.com/azure/key-vault/
Official docs (Monitoring)	Azure Monitor documentation	Logging, metrics, alerts for production operations: https://learn.microsoft.com/azure/azure-monitor/

18. Training and Certification Providers

The following providers may offer training relevant to Azure, AI + Machine Learning, and agentic solutions. Verify current syllabi and delivery modes on each website.

Institute	Suitable Audience	Likely Learning Focus	Mode	Website URL
DevOpsSchool.com	DevOps engineers, architects, developers	Cloud DevOps, Azure fundamentals, deployment practices that support AI apps	Check website	https://www.devopsschool.com/
ScmGalaxy.com	Beginners to intermediate	Software configuration management, DevOps foundations useful for operating AI services	Check website	https://www.scmgalaxy.com/
CLoudOpsNow.in	Cloud ops and platform teams	Cloud operations practices, monitoring, reliability patterns	Check website	https://www.cloudopsnow.in/
SreSchool.com	SREs, ops leads, platform engineers	SRE practices, observability, incident response for production AI systems	Check website	https://www.sreschool.com/
AiOpsSchool.com	Ops + AI practitioners	AIOps concepts, operating AI/ML services with automation	Check website	https://www.aiopsschool.com/

19. Top Trainers

These sites are presented as training resources/platforms. Verify current offerings directly.

Platform/Site	Likely Specialization	Suitable Audience	Website URL
RajeshKumar.xyz	DevOps/cloud training content	Beginners to intermediate engineers	https://rajeshkumar.xyz/
devopstrainer.in	DevOps tooling and practices	DevOps engineers and sysadmins	https://www.devopstrainer.in/
devopsfreelancer.com	Practical DevOps consulting/training content	Teams needing hands-on guidance	https://www.devopsfreelancer.com/
devopssupport.in	DevOps support and training-style resources	Ops teams and engineers	https://www.devopssupport.in/

20. Top Consulting Companies

These companies may help with cloud architecture, DevOps enablement, and productionizing AI workloads. Validate scope, references, and contracts directly.

Company	Likely Service Area	Where They May Help	Consulting Use Case Examples	Website URL
cotocus.com	Cloud/DevOps consulting	Architecture, migrations, automation	Azure landing zone setup, CI/CD for AI apps, monitoring rollout	https://cotocus.com/
DevOpsSchool.com	DevOps and cloud consulting/training	Platform enablement, engineering upskilling	Operating model for AI projects, DevSecOps pipelines, IaC standards	https://www.devopsschool.com/
DEVOPSCONSULTING.IN	DevOps consulting	Delivery pipelines, reliability practices	SRE-aligned ops for AI services, cost controls, observability	https://www.devopsconsulting.in/

21. Career and Learning Roadmap

What to learn before this service

Azure fundamentals – Resource groups, regions, RBAC, networking basics
Identity and security – Microsoft Entra ID, Managed Identity, Key Vault
API fundamentals – REST, auth, retry strategies, rate limits
Basics of LLMs – Tokens, context windows, temperature, system vs user prompts
RAG fundamentals (optional but recommended) – Embeddings, indexing, retrieval evaluation, chunking strategies

What to learn after this service

Production RAG optimization and evaluation
Observability for AI systems (tracing, redaction, governance)
Advanced tool security: authorization, approvals, policy engines
CI/CD for agent configuration (if supported): promotion, rollback, regression testing
Multi-agent patterns (only if supported and needed)

Job roles that use it

Cloud Solutions Architect (AI)
AI Engineer / Applied AI Engineer
Platform Engineer (AI enablement)
DevOps Engineer / SRE supporting AI services
Security Engineer (AI governance)
Backend Developer integrating agent endpoints

Certification path (if available)

There may not be a Foundry Agent Service–specific certification. Typical relevant Azure certifications to consider: – Azure fundamentals and architecture certifications – Azure AI-focused certifications (verify current Microsoft certification lineup)

Start here: https://learn.microsoft.com/credentials/

Project ideas for practice

Build an internal “Runbook Agent” grounded in Markdown runbooks with a safe escalation workflow
Create a “Customer Support Agent” with read-only tools and strict authorization checks
Build evaluation harnesses: regression tests for 30 common queries and track answer quality over time
Implement cost dashboards: tokens/user/day, tool calls/user/day, retrieval queries/user/day

22. Glossary

Term	Definition
Agent	An AI-driven component configured with instructions, tools, and (optionally) knowledge sources to complete tasks.
Tool calling	A pattern where the model requests a structured call to an external function/API, which your system executes.
Grounding	Constraining responses using retrieved enterprise data to reduce hallucinations.
RAG (Retrieval-Augmented Generation)	Architecture that retrieves relevant documents and supplies them to the model to improve accuracy.
Thread / Session	A stateful conversation container holding message history and execution context.
Run / Execution	A single cycle where the agent processes messages, calls tools, and returns an output.
RBAC	Role-Based Access Control in Azure.
Managed Identity	An Azure identity for workloads that allows access to resources without storing secrets.
Azure AI Search	Azure service for indexing and searching content, commonly used for RAG.
Azure Monitor / Application Insights	Azure observability services for metrics, logs, and distributed tracing.
Token	The basic unit of text processed by many language models; used for billing and context limits.
Rate limit	Service-enforced limit on requests/tokens per time window.
Least privilege	Security principle of granting only the minimum permissions required to perform a task.

23. Summary

Foundry Agent Service (Azure) is a managed way to build and operate AI agents—systems that combine a foundation model with stateful conversation handling, tool usage, and (optionally) grounded enterprise knowledge. It matters because most production AI solutions require more than a model endpoint: they need secure integrations, reliable orchestration, governance, and observability.

From a cost perspective, the biggest drivers are model tokens and the “multipliers” introduced by agent steps (retrieval + tool calls + additional model calls), plus indirect costs like search, tool compute, and logging. From a security perspective, success depends on Entra ID/RBAC, tight tool authorization, careful logging/redaction, and disciplined environment separation.

Use Foundry Agent Service when you want Azure-native agent orchestration with enterprise controls. If you only need simple text generation, call a model endpoint directly; if you need maximum orchestration control, consider a self-managed framework—accepting the added operational burden.

Next step: follow the official Microsoft Learn documentation trail for the exact Foundry Agent Service capabilities in your region and build a small grounded agent with one safe, read-only tool: https://learn.microsoft.com/search/?terms=Foundry%20Agent%20Service%20Azure

rajeshkumar

Category