Category
Storage
1. Introduction
- What this service is
-
Archive Storage in Azure Storage is the long-term, lowest-cost access tier for Azure Blob Storage. It’s designed for data you rarely read but must keep for months or years (compliance, audit, backups, research datasets, raw logs, etc.).
-
One-paragraph simple explanation
-
Think of Archive Storage as putting files into a deep, low-cost vault. You can store a lot of data cheaply, but when you need it back, you must “request” it and wait while Azure brings it online before you can download or process it.
-
One-paragraph technical explanation
-
Azure implements Archive Storage as the Archive access tier for block blobs stored in an Azure Storage account. Blobs in the archive tier are offline: you can’t read them immediately. To access data, you must rehydrate (change the tier to Hot/Cool) and wait until the blob becomes available. Costs typically shift from “pay more for storage” (Hot) to “pay less for storage but more for retrieval/rehydration and potential early-deletion charges” (Archive).
-
What problem it solves
- It reduces long-term storage cost while still keeping data in Azure with enterprise-grade durability, encryption, identity controls, and governance—making it a strong fit for regulatory retention, cold backups, and data archives where access is infrequent but retention is mandatory.
2. What is Archive Storage?
Important naming clarification (read this first):
In Azure, “Archive Storage” is not typically a standalone service you deploy as its own resource. It most commonly refers to the Archive access tier of Azure Blob Storage within an Azure Storage account. You manage it through Blob Storage features (tiers, lifecycle rules, rehydration), not by creating a separate “Archive Storage instance.”
Official purpose
- Provide low-cost, long-term retention for blob data that is rarely accessed and can tolerate hours of retrieval latency.
Core capabilities
- Store blob data in an Archive access tier with very low $/GB-month compared to Hot.
- Move data to Archive manually or automatically using lifecycle management policies.
- Retrieve archived data by rehydrating it to Hot/Cool (rehydration time varies; verify current expectations in official docs).
- Apply standard Azure Storage controls: encryption, RBAC, SAS, private endpoints, logging/monitoring, immutability (where applicable), and governance.
Major components
- Azure Storage account (the resource you create)
- Blob service within the storage account
- Containers (like folders at the top level)
- Block blobs (the objects/files)
- Access tiers: Hot, Cool, and Archive (Archive is the focus here)
- Lifecycle management rules (optional but common in real systems)
- Rehydration workflow (tier change back to Hot/Cool)
Service type
- A storage tier (Archive) within Azure Blob Storage (object storage).
Scope (subscription/regional)
- Deployed in an Azure region as part of a Storage account.
- Managed at the subscription level (billing, policies) and resource group level (lifecycle, locks, access).
- Access is controlled via Azure AD identities and/or storage keys/SAS, with networking controls at the storage account boundary.
How it fits into the Azure ecosystem
Archive Storage is usually one component in broader Azure data platforms and operations:
- Data pipelines: Azure Data Factory / Synapse / Databricks land data in Hot/Cool, then lifecycle it to Archive.
- Security and compliance: Microsoft Purview for cataloging; Azure Policy for enforcement; immutable blob storage for retention (verify scenario support).
- Backup/DR patterns: application or database exports stored cheaply for long periods; retrieval only during audit or restore events.
- Observability: logs exported to Blob Storage, then archived.
Official docs starting points (verify latest details here):
– Blob access tiers overview: https://learn.microsoft.com/azure/storage/blobs/access-tiers-overview
– Azure Blob Storage documentation: https://learn.microsoft.com/azure/storage/blobs/
3. Why use Archive Storage?
Business reasons
- Lower long-term cost for data you must keep but rarely use.
- Avoid on-prem tape/library operations and reduce data center footprint.
- Support retention-driven needs (audits, legal, financial records) while keeping data in the same cloud ecosystem as your workloads.
Technical reasons
- Native tiering within the same storage platform (Blob Storage), avoiding data moves to a completely different product.
- Durability and redundancy options (LRS/ZRS/GRS variants depending on region/account capabilities—verify for your account).
- Integrates with lifecycle policies, immutability, and standard blob APIs/SDKs.
Operational reasons
- You can automate archiving using policy-based lifecycle management rather than manual runs.
- Monitoring and auditing integrate with Azure Monitor and storage diagnostics.
- Works well with Infrastructure as Code (Bicep/Terraform) and standard deployment pipelines.
Security/compliance reasons
- Supports encryption at rest by default, with options such as customer-managed keys (CMK) depending on account configuration (verify requirements).
- Access can be constrained via Azure AD RBAC, private endpoints, firewall rules, and SAS.
- Optional immutability policies for WORM-style retention (validate feature availability and constraints in official docs).
Scalability/performance reasons
- Scales like Blob Storage scales—suitable for very large datasets.
- Archive is not about performance; it’s about cost. The “performance” consideration is actually rehydration latency and retrieval behavior.
When teams should choose it
- Data is accessed rarely (weeks/months).
- Retrieval time of hours is acceptable.
- Long retention is required (compliance, backups, historical raw telemetry).
- You want to keep data in Azure with cloud-native governance.
When they should not choose it
- You need frequent reads, low-latency access, or interactive analytics directly on stored objects.
- You can’t tolerate retrieval delays.
- Your workload repeatedly rehydrates the same data—costs can exceed Cool/Hot quickly.
- You’re storing short-lived data: archive tiers often have minimum storage duration and early deletion charges (verify exact current terms in the official pricing/docs).
4. Where is Archive Storage used?
Industries
- Finance: audit trails, transaction archives, compliance exports
- Healthcare/Life sciences: retention of records and research datasets
- Government/Public sector: regulated record retention
- Media/Entertainment: raw footage archives and project assets
- Manufacturing/IoT: historical sensor/telemetry archives
- Retail: historical orders and event logs
Team types
- Cloud platform teams managing enterprise storage
- Security/compliance teams enforcing retention and auditability
- Data engineering teams implementing lake/landing zones
- DevOps/SRE teams archiving logs and backup artifacts
- Application teams storing long-lived exports and documents
Workloads
- Long-term log retention (after hot analytics window)
- Backup exports (VM/app/database exports)
- Compliance data retention (WORM/immutability scenarios—verify)
- Dataset snapshots for reproducible science/ML
Architectures
- Data lake zones (landing → curated → archive)
- Event/log pipelines with lifecycle movement
- Backup/restore workflows with infrequent retrieval
- Multi-region DR patterns (depending on replication strategy)
Real-world deployment contexts
- Production: lifecycle policies, private endpoints, RBAC, encryption policies, monitoring, and documented retrieval runbooks.
- Dev/test: verifying retention policy behavior and retrieval time/cost modeling; you typically don’t archive much in dev unless testing compliance workflows.
5. Top Use Cases and Scenarios
Below are realistic scenarios where Azure Archive Storage (Archive tier in Blob Storage) fits well.
1) Compliance record retention
- Problem: Regulations require keeping records for 7–10 years, but they’re rarely accessed.
- Why Archive Storage fits: Low storage cost with enterprise security and retention controls.
- Example: Quarterly financial statements exported to PDFs/CSVs and archived; retrieval happens only during audits.
2) Security log long-term retention
- Problem: Keep security logs for investigations, but only recent logs are queried frequently.
- Why it fits: Store last 30–90 days in Hot/Cool; archive older logs.
- Example: Exported firewall/proxy logs stored in Blob; lifecycle rules move older blobs to Archive.
3) Backup export repository
- Problem: You keep weekly/monthly backup exports but rarely restore from older ones.
- Why it fits: Archive reduces cost for older restore points.
- Example: Database full backups copied to Blob; after 45 days, moved to Archive.
4) Historical IoT telemetry archives
- Problem: Massive time-series telemetry is useful for long-term trend studies, not daily operations.
- Why it fits: Archive stores large volumes cheaply; rehydrate only for investigations.
- Example: Raw device telemetry Parquet files archived after 60 days.
5) Media raw footage archiving
- Problem: Raw video is large; edits happen early, then footage is retained long-term.
- Why it fits: Archive minimizes storage costs once production ends.
- Example: 4K raw files archived with occasional retrieval for remastering.
6) Data lake “cold zone” for reproducibility
- Problem: You must preserve original datasets to reproduce analyses.
- Why it fits: Archive stores immutable-ish snapshots cheaply (immutability features depend on configuration; verify).
- Example: Monthly dataset snapshots stored and archived; rehydrated for audits.
7) Legal hold and eDiscovery source preservation
- Problem: A legal case requires preserving documents and communications exports.
- Why it fits: Archive reduces cost while maintaining governance, with retention/hold controls (verify the exact legal-hold/immutability mechanics for your scenario).
- Example: Exported mailboxes and case files stored in Blob and archived.
8) Application-generated documents (rare access)
- Problem: You generate invoices/receipts and must store them long-term, but users rarely download older documents.
- Why it fits: Hot for recent months; Archive for older years.
- Example: Invoices > 12 months moved to Archive; rehydrated on-demand.
9) Long-term build artifacts / release archives
- Problem: Keep old release binaries for compliance or rollback but rarely use them.
- Why it fits: Archive keeps artifacts cheap; retrieval is occasional.
- Example: Quarterly releases archived after 90 days.
10) Research and genomics data retention
- Problem: Large research files must be retained for long periods; access is sporadic.
- Why it fits: Archive is designed for deep storage; retrieval is planned.
- Example: Sequencing output stored and later rehydrated for meta-analysis.
11) Post-incident forensic snapshots
- Problem: Preserve evidence after an incident; access is rare but must be durable.
- Why it fits: Archive stores forensic packages cost-effectively.
- Example: Disk images and investigation exports archived after case closure.
12) Cross-team shared archive repository
- Problem: Multiple teams need a central archive with strict access controls.
- Why it fits: Central storage account + RBAC + private endpoints + lifecycle.
- Example: Organization-wide “Archive” subscription stores long-term exports with standardized naming and tags.
6. Core Features
This section focuses on features that matter most when using Archive Storage in Azure (Archive tier in Blob Storage). Where a feature is a broader Blob Storage capability, it’s called out as such.
1) Archive access tier (offline storage)
- What it does: Stores blobs in the Archive tier for lowest storage cost.
- Why it matters: Major cost reduction for long-lived, rarely accessed data.
- Practical benefit: You can keep years of data without paying Hot-tier rates.
- Limitations/caveats:
- Archive blobs are offline and can’t be read until rehydrated.
- Expect rehydration latency (often hours; verify current SLA/behavior in official docs).
2) Rehydration (Archive → Hot/Cool)
- What it does: Changes a blob’s tier from Archive to Hot or Cool to make it readable again.
- Why it matters: It’s the gateway to retrieving archived data.
- Practical benefit: You only pay retrieval/rehydration when needed.
- Limitations/caveats:
- Rehydration takes time and may have separate pricing dimensions (operation + data retrieval).
- Some workflows need a runbook to manage “request → wait → verify → download.”
3) Lifecycle management policies (automated tiering)
- What it does: Moves blobs automatically based on rules (age, prefix, blob index tags, etc.—exact rule options depend on current platform features; verify).
- Why it matters: Eliminates manual archiving and ensures cost targets are met.
- Practical benefit: Data lands in Hot/Cool for ingestion and then automatically archives.
- Limitations/caveats:
- Policies typically evaluate on a schedule (not immediate).
- Misconfigured rules can move important data to Archive unexpectedly—use prefixes/tags and guardrails.
4) Tier at the blob level (fine-grained control)
- What it does: Lets you set Archive tier per blob rather than per container/account.
- Why it matters: You can mix hot and cold objects in one container without splitting data.
- Practical benefit: Keep metadata/manifest files hot while archiving large payloads.
- Limitations/caveats:
- The account “default access tier” typically applies to Hot/Cool, while Archive is an explicit blob-level choice.
5) Redundancy options (durability tradeoffs)
- What it does: Storage accounts can be configured with redundancy (LRS/ZRS/GRS variants depending on region and account type).
- Why it matters: Determines durability and availability characteristics of archived data.
- Practical benefit: Choose cost vs resilience per business requirements.
- Limitations/caveats:
- Not all redundancy modes may support all tiering features uniformly; verify Archive tier support for your redundancy choice in official docs.
6) Encryption at rest (Microsoft-managed or customer-managed keys)
- What it does: Encrypts data by default; some configurations support CMK via Azure Key Vault.
- Why it matters: Protects archived data (which is often sensitive).
- Practical benefit: Meet compliance requirements without additional tooling.
- Limitations/caveats:
- CMK requires operational discipline (key rotation, access policies, availability planning).
7) Identity and access (Azure AD RBAC, SAS, storage keys)
- What it does: Control access using Azure AD roles, time-bound SAS tokens, or account keys.
- Why it matters: Archive data often has strict access constraints.
- Practical benefit: Least privilege with auditable access patterns.
- Limitations/caveats:
- Storage keys are powerful and hard to govern—prefer RBAC/SAS where possible.
8) Networking controls (private endpoints, firewall, trusted services)
- What it does: Restricts Blob endpoint access to private networks and approved IPs.
- Why it matters: Reduces data exfiltration risk.
- Practical benefit: Archive repositories can be private-only.
- Limitations/caveats:
- Private endpoints require DNS planning and can affect automation if not designed properly.
9) Data protection (soft delete, versioning, immutability)
- What it does: Helps protect against accidental deletion/overwrite.
- Why it matters: Archive datasets are often “set and forget,” making accidental delete extremely costly.
- Practical benefit: Recover from operator mistakes and ransomware-like deletion events.
- Limitations/caveats:
- The exact interactions between versioning/immutability and tiering can be nuanced—validate with official docs and a non-production test.
10) Observability (metrics + logs)
- What it does: Exposes metrics and logs through Azure Monitor and diagnostic settings.
- Why it matters: You need visibility into capacity, transactions, errors, and unexpected retrieval spikes.
- Practical benefit: Detect “rehydration storms,” misconfigured lifecycle, or unauthorized access attempts.
- Limitations/caveats:
- Logging has cost and retention considerations.
7. Architecture and How It Works
High-level service architecture
At a high level, Archive Storage is:
- A storage account hosting Blob Storage
- One or more containers
- Blobs stored in tiers: Hot/Cool/Archive
- A management plane for policies (lifecycle), security, and networking
- A data plane for blob operations (PUT/GET/list/tier change)
Request/data/control flow
-
Ingestion path (data plane)
Applications, pipelines, or users upload blobs to a container. Typically, the blob starts in Hot or Cool. -
Archiving path (control + data plane)
– A lifecycle policy or manual operation changes the tier of older blobs to Archive. – Once archived, the blob becomes offline. -
Retrieval path (rehydration + data plane)
– A user/app requests rehydration (tier change to Hot/Cool). – Azure begins rehydration. During this time, the blob remains unavailable for reads. – After rehydration completes, the blob can be read normally.
Integrations with related services
Common integrations around Archive Storage include:
- Azure Data Factory: move/copy data and orchestrate archiving and retrieval workflows.
- Azure Functions / Logic Apps: automate rehydration requests upon ticket approval.
- Azure Key Vault: CMK encryption key storage (if using CMK).
- Azure Monitor / Log Analytics: storage diagnostics, alerting on transactions and egress.
- Microsoft Purview: cataloging and governance across data lake zones.
- Azure Policy: enforce “no public access,” required private endpoints, required tags, etc.
Dependency services
- Azure Storage account is the primary dependency.
- Optional: Key Vault, VNets/Private DNS, Log Analytics workspace, Event Grid (for blob events), Policy.
Security/authentication model
- Azure AD (recommended): assign roles like Storage Blob Data Reader/Contributor to identities (users, managed identities).
- SAS tokens: scoped, time-limited access for external systems.
- Shared key (account key): powerful, should be restricted and rotated.
Networking model
- Public endpoint with firewall rules, or
- Private endpoint to keep traffic on private IPs, plus DNS configuration.
Monitoring/logging/governance considerations
- Track:
- Storage capacity by tier
- Transactions (especially tier changes and reads)
- Egress data
- Authorization failures
- Govern:
- Naming/tagging conventions
- Lifecycle policy review process
- Access review and key rotation
- Cost alerts for retrieval spikes
Simple architecture diagram (Mermaid)
flowchart LR
U[User/App] -->|Upload (PUT)| B[Azure Blob Storage<br/>Hot/Cool]
B -->|Lifecycle rule or manual tier change| A[Archive Storage<br/>(Archive tier)]
U -->|Rehydrate request| A
A -->|After rehydration completes| B
U -->|Download (GET)| B
Production-style architecture diagram (Mermaid)
flowchart TB
subgraph VNET[Virtual Network]
subgraph SUBNET1[App Subnet]
APP[App / Data Pipeline<br/>(VM, AKS, ADF IR, etc.)]
end
subgraph SUBNET2[Private Endpoint Subnet]
PE[Private Endpoint<br/>for Blob]
end
DNS[Private DNS Zone<br/>privatelink.blob.core.windows.net]
end
APP -->|Private DNS resolves| DNS
APP -->|HTTPS via Private Link| PE --> SA[(Storage Account<br/>Blob Storage)]
SA --> CON[Container(s)]
CON --> HOT[Hot/Cool blobs]
CON --> ARC[Archive blobs]
POL[Lifecycle Management Policy] --> SA
KV[Azure Key Vault<br/>(CMK optional)] --> SA
MON[Azure Monitor / Log Analytics] <-->|Diagnostics & Metrics| SA
GOV[Azure Policy / Tags / Locks] --> SA
ARC -->|Rehydrate to Hot/Cool| HOT
8. Prerequisites
Account/subscription/tenant requirements
- An active Azure subscription with billing enabled.
- Ability to create:
- Resource groups
- Storage accounts
- Role assignments (if using Azure AD RBAC)
- Optional: private endpoints, Key Vault, Log Analytics
Permissions / IAM roles
To complete the lab using Azure CLI, you typically need: – At minimum: permissions to create resource group and storage account (e.g., Contributor on a resource group). – For data-plane operations using Azure AD: – Storage Blob Data Contributor (or higher) on the storage account or container scope.
If you use account keys, you don’t need data-plane RBAC, but you must be allowed to list keys (management-plane permission).
Billing requirements
- Storage accounts and Blob Storage are usage-based.
- Archive tier has low storage cost but can have:
- Retrieval/rehydration costs
- Transaction costs
- Early deletion charges (verify current terms)
CLI/SDK/tools needed
- Azure CLI: https://learn.microsoft.com/cli/azure/install-azure-cli
- Optional:
- AzCopy for fast transfers: https://learn.microsoft.com/azure/storage/common/storage-use-azcopy-v10
- PowerShell Az module (optional)
- Python/Node/.NET SDK (optional)
Region availability
- Blob Storage is available in many regions, but features vary.
- Verify Archive tier availability for your chosen region and redundancy via official docs.
Quotas/limits
Storage accounts have quotas/limits on: – Request rate patterns – Throughput – Capacity (practically large, but account-level limits exist) – Object size limits for block blobs and upload methods
Verify current limits in the official Azure Storage scalability and performance targets documentation: – https://learn.microsoft.com/azure/storage/common/scalability-targets-standard-account
Prerequisite services
For the core tutorial: – Only an Azure Storage account is required.
Optional (for production patterns): – Key Vault (for CMK) – VNet + Private Endpoint + Private DNS – Log Analytics / Azure Monitor alerts
9. Pricing / Cost
Azure Archive Storage pricing is primarily part of Azure Blob Storage pricing. Pricing varies by region, redundancy, and sometimes by performance/feature choices, so avoid hardcoding numbers—use official sources.
Official pricing and estimation:
– Azure Blob Storage pricing: https://azure.microsoft.com/pricing/details/storage/blobs/
– Azure Pricing Calculator: https://azure.microsoft.com/pricing/calculator/
Pricing dimensions (what you pay for)
Common cost dimensions for Archive tier solutions include:
-
Data stored (GB-month) in Archive tier – The main savings lever: Archive storage per GB-month is typically much cheaper than Hot.
-
Write operations / transactions – Uploads, list operations, metadata operations, tier changes—charged per operation class (verify current transaction categories on the pricing page).
-
Data retrieval and read operations – Reading archived data generally requires rehydration first; retrieval costs can be meaningful.
-
Rehydration (Archive → Hot/Cool) – Rehydration may involve:
- An operation cost for tier change
- Data retrieval costs
- Potential priority options (Standard/High) with different cost/time characteristics (verify current behavior)
-
Early deletion charges / minimum storage duration – Archive tier is designed for long retention and can include a minimum storage duration and early deletion fees if you delete or move data out of Archive too soon.
– Verify the current minimum duration for Archive in official docs/pricing for your region and account type. -
Data transfer (egress) – Data transferred out of Azure (internet egress) is billed. – Data transferred between regions can be billed depending on replication and architecture. – Data transferred within the same region between many Azure services is often free or discounted, but do not assume—verify your exact path.
-
Redundancy premium – Geo-redundant options (GRS/RA-GRS variants) cost more than LRS. – Archive tier costs still depend on redundancy chosen.
Cost drivers (what surprises teams)
- Frequent rehydration: If users “browse” archived data regularly, Archive can become more expensive than Cool.
- Poor lifecycle rules: Accidentally archiving active datasets leads to expensive retrieval and operational disruption.
- Egress costs: Restores to on-prem or other clouds can be costly.
- Minimum duration / early deletion: Deleting or moving blobs too early can create unexpected charges.
- Monitoring/logging retention: Diagnostic logs written to Log Analytics have ingestion and retention costs.
How to optimize cost (practical guidance)
- Use prefixes/tags to clearly separate archive-eligible data.
- Keep a manifest/index in Hot/Cool so you don’t have to list large archive containers repeatedly.
- Avoid repeated rehydration by:
- Rehydrating to Cool and keeping it there for a period if you expect repeated access
- Copying rehydrated data to a “working” container and leaving the original archived
- Implement approval workflows for rehydration (ticket-based).
- Set cost alerts for retrieval spikes and egress.
- Choose redundancy based on business requirements; don’t default to geo-redundant if not needed.
Example low-cost starter estimate (no fabricated numbers)
A minimal lab setup might include: – 1 Storage account (Standard) – A few MB to a few GB of data stored in Archive – Very few operations – No private endpoint, no Key Vault, no Log Analytics
In this setup: – Storage cost is extremely low. – Main costs come from any rehydration and retrieval you perform during testing.
Use the Azure Pricing Calculator with: – Data stored in Archive tier (GB) – Expected rehydrations per month – Expected data retrieved (GB) – Transaction volume (reads/writes/list) – Region + redundancy
Example production cost considerations
In production, costs are usually dominated by: – Total archived TB/PB – Geo-redundancy premium (if used) – Retrieval patterns (investigations, audits, restores) – Egress and cross-region copies – Observability costs (if you centralize logs)
A good production cost model includes: – Separate estimates for “steady state” (mostly storage) and “incident mode” (large retrieval). – A retrieval budget and operational controls to prevent runaway costs.
10. Step-by-Step Hands-On Tutorial
Objective
Create an Azure Storage account, upload a blob, move it into Archive Storage (Archive tier), attempt access (and observe behavior), then rehydrate it back to a readable tier and download it.
Lab Overview
You will: 1. Create a resource group and storage account. 2. Create a blob container and upload a small test file. 3. Set the blob’s tier to Archive. 4. Verify that reads are blocked while archived. 5. Request rehydration to Hot (or Cool), monitor status, and download after rehydration completes. 6. Clean up resources.
This lab is designed to be safe and low-cost: – Uses a small blob. – Uses locally generated test data. – Deletes everything at the end.
Step 1: Sign in and set variables
1) Sign in:
az login
az account show
2) Set variables (choose a unique storage account name; must be globally unique and 3–24 lowercase letters/numbers):
export LOCATION="eastus"
export RG="rg-archive-storage-lab"
export SA="archivestorage$RANDOM$RANDOM" # may still collide; adjust if needed
export CONTAINER="archive-lab"
export FILE="hello-archive.txt"
3) Create a small test file:
echo "Hello from Azure Archive Storage lab: $(date -u)" > "$FILE"
ls -l "$FILE"
Expected outcome: You are logged in, and a local file exists.
Step 2: Create a resource group and storage account
1) Create the resource group:
az group create --name "$RG" --location "$LOCATION"
2) Create a Storage account (General Purpose v2, Standard LRS):
az storage account create \
--name "$SA" \
--resource-group "$RG" \
--location "$LOCATION" \
--kind StorageV2 \
--sku Standard_LRS \
--https-only true \
--allow-blob-public-access false
3) Confirm the account exists:
az storage account show --name "$SA" --resource-group "$RG" --query "{name:name,location:location,kind:kind,sku:sku.name}" -o table
Expected outcome: A StorageV2 account exists with Standard_LRS.
Step 3: Create a container and upload a blob
For simplicity in a lab, we’ll use a storage account key. (In production, prefer Azure AD RBAC + managed identities where possible.)
1) Fetch a key into a variable:
export SA_KEY=$(az storage account keys list -g "$RG" -n "$SA" --query "[0].value" -o tsv)
2) Create a container:
az storage container create \
--name "$CONTAINER" \
--account-name "$SA" \
--account-key "$SA_KEY"
3) Upload the blob:
az storage blob upload \
--container-name "$CONTAINER" \
--account-name "$SA" \
--account-key "$SA_KEY" \
--file "$FILE" \
--name "$FILE"
4) Verify blob properties (including access tier):
az storage blob show \
--container-name "$CONTAINER" \
--account-name "$SA" \
--account-key "$SA_KEY" \
--name "$FILE" \
--query "{name:name, tier:properties.accessTier, size:properties.contentLength}" -o table
Expected outcome: The blob exists and is currently in Hot or Cool (commonly Hot by default).
Step 4: Move the blob to Archive Storage (Archive tier)
Set the blob tier to Archive:
az storage blob set-tier \
--container-name "$CONTAINER" \
--account-name "$SA" \
--account-key "$SA_KEY" \
--name "$FILE" \
--tier Archive
Verify:
az storage blob show \
--container-name "$CONTAINER" \
--account-name "$SA" \
--account-key "$SA_KEY" \
--name "$FILE" \
--query "{name:name, tier:properties.accessTier, archiveStatus:properties.archiveStatus}" -o table
Expected outcome: tier shows Archive. archiveStatus is typically empty unless rehydration is in progress.
Step 5: Attempt to download while archived (observe expected failure)
Try to download:
az storage blob download \
--container-name "$CONTAINER" \
--account-name "$SA" \
--account-key "$SA_KEY" \
--name "$FILE" \
--file "downloaded-$FILE"
Expected outcome: The download should fail because the blob is in the Archive tier (offline). The error message may indicate the blob must be rehydrated first.
This is a key operational concept: Archive Storage is not directly readable.
Step 6: Request rehydration (Archive → Hot) and monitor status
1) Request rehydration to Hot:
az storage blob set-tier \
--container-name "$CONTAINER" \
--account-name "$SA" \
--account-key "$SA_KEY" \
--name "$FILE" \
--tier Hot
2) Check the blob’s archive status:
az storage blob show \
--container-name "$CONTAINER" \
--account-name "$SA" \
--account-key "$SA_KEY" \
--name "$FILE" \
--query "{name:name, tier:properties.accessTier, archiveStatus:properties.archiveStatus}" -o table
While rehydration is pending, archiveStatus may indicate a rehydration state (exact values can vary; use the output as your source of truth).
3) Poll until rehydration completes (simple loop):
while true; do
STATUS=$(az storage blob show \
--container-name "$CONTAINER" \
--account-name "$SA" \
--account-key "$SA_KEY" \
--name "$FILE" \
--query "properties.archiveStatus" -o tsv)
TIER=$(az storage blob show \
--container-name "$CONTAINER" \
--account-name "$SA" \
--account-key "$SA_KEY" \
--name "$FILE" \
--query "properties.accessTier" -o tsv)
echo "$(date -u) tier=$TIER archiveStatus=${STATUS:-<none>}"
# When archiveStatus is empty and tier is Hot/Cool, it is typically available.
if [ -z "$STATUS" ] && [ "$TIER" != "Archive" ]; then
break
fi
sleep 60
done
Expected outcome: Eventually, the blob becomes readable again. Rehydration can take time (often hours); do not assume it will finish during a short lab window. If you need immediate validation, keep the file small and be prepared to wait—or treat this step as a “requested rehydration” demonstration and proceed later to download.
Step 7: Download after rehydration completes
Once rehydration is complete, download:
az storage blob download \
--container-name "$CONTAINER" \
--account-name "$SA" \
--account-key "$SA_KEY" \
--name "$FILE" \
--file "downloaded-$FILE" \
--overwrite true
Compare content:
diff "$FILE" "downloaded-$FILE" && echo "Downloaded file matches original."
Expected outcome: The download succeeds and the file matches.
Validation
Use these checks to validate your work:
1) Confirm tier transitions:
az storage blob show \
--container-name "$CONTAINER" \
--account-name "$SA" \
--account-key "$SA_KEY" \
--name "$FILE" \
--query "{tier:properties.accessTier, archiveStatus:properties.archiveStatus, lastModified:properties.lastModified}" -o table
2) Confirm the blob exists in your container:
az storage blob list \
--container-name "$CONTAINER" \
--account-name "$SA" \
--account-key "$SA_KEY" \
--output table
Troubleshooting
Error: “The specified account name is already taken”
- Storage account names must be globally unique.
- Fix: change
$SAand re-run Step 2.
Error: Authorization failure (403)
- If using
--account-key, ensure$SA_KEYis set correctly. - If using Azure AD auth (
--auth-mode login), ensure you have the correct Storage Blob Data Contributor role at the right scope.
Download fails because blob is in Archive
- That is expected.
- Fix: run rehydration (
set-tierto Hot/Cool) and wait untilarchiveStatusclears.
Rehydration seems stuck
- Rehydration can take significant time depending on settings and platform conditions.
- Verify in official docs what the expected rehydration time is and whether a priority option is available for your account. Also confirm you requested rehydration to Hot/Cool.
Lifecycle policy doesn’t move blobs immediately (if you try it)
- Lifecycle policies are evaluated on a schedule, not instantly.
- Validate the rule scope (prefix, blob type, days since creation/last modified).
Cleanup
To avoid ongoing charges, delete the resource group:
az group delete --name "$RG" --yes --no-wait
Expected outcome: All lab resources are deleted (storage account, container, blobs).
11. Best Practices
Architecture best practices
- Design a tiering strategy:
- Hot for active ingestion and recent access
- Cool for infrequent but still online access
- Archive for offline deep retention
- Keep indexes/manifests in Hot/Cool so you can find what you need without scanning archived datasets.
- Create a rehydration workflow (ticket/approval + automation) so retrieval is controlled and auditable.
- Consider separating concerns:
- One storage account for “archive vault” data with stricter network rules
- Another for active data lake zones
IAM/security best practices
- Prefer Azure AD RBAC and managed identities over account keys.
- Use SAS only when needed; keep SAS:
- Short-lived
- Narrowly scoped (container/blob)
- Minimum permissions (read only for retrieval)
- Run regular access reviews for archive repositories.
Cost best practices
- Use lifecycle rules to move data to Archive based on clear criteria (prefix/tag + age).
- Set budgets and alerts specifically for:
- Data retrieval/egress
- Transaction spikes
- Avoid frequent rehydration; if a dataset becomes “semi-active,” keep it in Cool.
Performance best practices
- Archive isn’t for performance; optimize the system:
- Store small metadata in Hot/Cool
- Batch rehydration requests
- Use AzCopy for large transfers once rehydrated
Reliability best practices
- Choose redundancy (LRS/ZRS/GRS variants) based on RTO/RPO requirements.
- Document restore steps and run periodic restore drills (rehydrate + download + validate).
Operations best practices
- Implement naming conventions:
- Storage account:
st<org><env><region><purpose> - Container:
archive-<domain>(e.g.,archive-finance) - Blob path:
domain/system/year=YYYY/month=MM/day=DD/... - Use tags on the storage account for:
- Cost center
- Data classification
- Owner team
- Retention policy ID
- Enable and centralize diagnostics carefully (balance visibility with log cost).
Governance best practices
- Use Azure Policy to enforce:
- Public access disabled
- HTTPS-only
- Private endpoints (where required)
- Required tags
- Use resource locks for critical archive accounts to prevent accidental deletion (ensure your process supports intentional teardown when needed).
12. Security Considerations
Identity and access model
- Azure AD RBAC (recommended):
- Assign least-privilege roles:
- Storage Blob Data Reader for read-only retrieval (after rehydration)
- Storage Blob Data Contributor for upload and tier changes
-
Use managed identities for automation (Functions, ADF, VMs, AKS workloads).
-
Shared Access Signatures (SAS):
- Use for temporary external access.
-
Prefer User Delegation SAS (Azure AD-backed) where supported and appropriate—verify current support and constraints for your environment.
-
Account keys
- Highly privileged; anyone with a key can access data depending on settings.
- Use only when necessary; rotate keys and store them in secure systems (Key Vault).
Encryption
- Azure Storage encrypts data at rest by default.
- For stronger controls:
- Use Customer-Managed Keys (CMK) with Key Vault (requires governance/availability planning).
- For data in transit:
- Enforce HTTPS-only (recommended).
Network exposure
- For sensitive archives:
- Use private endpoints and disable public network access where feasible.
- Restrict outbound and inbound via firewalls and NSGs in surrounding architecture.
- Ensure DNS resolution for
privatelink.blob.core.windows.netis correct.
Secrets handling
- Don’t embed SAS tokens or keys in code repositories.
- Use Key Vault and workload identities.
- Log redaction: make sure pipelines don’t print secrets.
Audit/logging
- Enable diagnostic settings for:
- Authentication failures
- Write/delete/tier-change operations
- Suspicious access patterns
- Send logs to a secure Log Analytics workspace with retention aligned to your audit needs.
Compliance considerations
- Archive use cases often involve regulated data:
- Retention requirements
- Encryption requirements
- Access logging requirements
- Validate whether you need immutability/WORM controls and confirm how they interact with tiering in your exact configuration (verify in official docs).
Common security mistakes
- Leaving public access enabled on storage accounts or containers.
- Using long-lived SAS tokens with broad permissions.
- Sharing account keys across teams.
- No monitoring for unexpected retrieval/egress (data exfiltration risk).
- No resource locks or deletion protection for critical archives.
Secure deployment recommendations
- Use:
- RBAC + managed identities
- Private endpoints for sensitive archives
- Policy enforcement + tags
- Budget alerts for retrieval/egress
- Regular access reviews and key rotation (if keys are used at all)
13. Limitations and Gotchas
Archive Storage is extremely useful, but it has sharp edges. Plan for these.
-
Offline nature – You cannot directly read an archived blob until it is rehydrated.
-
Rehydration time – Retrieval can take a significant amount of time (often hours).
– Verify current expected rehydration times and options in official docs. -
Minimum storage duration / early deletion charges – Archive tier commonly has a minimum duration policy and charges if you delete or move data too early.
– Verify current terms in pricing/docs for your region. -
Cost unpredictability during incidents – During audits or incidents, retrieval and egress can spike. If not budgeted and controlled, cost can surprise you.
-
Lifecycle policy safety – Mis-scoped lifecycle rules can move active data to Archive, causing application failures.
-
Not suitable for analytics directly – Archived blobs aren’t available for interactive analytics without rehydration.
-
Tooling assumptions – Some tools assume data is always online; ensure your tooling handles archive errors and rehydration workflows.
-
Feature compatibility nuances – Some Blob Storage features may have specific constraints when blobs are in Archive (or when combined with versioning/immutability/replication).
– Validate with official docs and run a proof-of-concept. -
Redundancy and region constraints – Availability of certain redundancy modes and features can vary by region and account type.
– Verify your exact target region and SKU. -
Operational friction – You need a runbook: how to identify what to retrieve, request rehydration, track status, and complete retrieval.
14. Comparison with Alternatives
Archive Storage is one option in a spectrum of storage and archival solutions.
Comparison table
| Option | Best For | Strengths | Weaknesses | When to Choose |
|---|---|---|---|---|
| Azure Archive Storage (Blob Archive tier) | Deep, long-term retention with rare access | Lowest blob storage cost tier, integrates with Azure Storage security/governance, lifecycle automation | Offline; rehydration delay; retrieval/early deletion costs | You need long retention and can tolerate hours to retrieve |
| Azure Blob Storage Cool tier | Infrequent access but still online | Lower cost than Hot, immediate reads | More expensive than Archive for long retention | You need online access with lower frequency |
| Azure Blob Storage Hot tier | Frequent access / active workloads | Best performance and lowest access costs | Highest storage cost | Active datasets, web content, frequent reads |
| Azure Files / Azure File Sync | Lift-and-shift file shares | SMB/NFS-like semantics (depending on config) | Not an archive tier solution; different pricing/semantics | Legacy apps needing file shares |
| Azure Backup (vault-based) | Managed backups for Azure workloads | Policy-driven, restore workflows, central management | Not a general-purpose object archive; different retrieval patterns | You want managed backups rather than building your own |
| AWS S3 Glacier / Glacier Deep Archive | Archive in AWS | Mature archive classes, similar offline retrieval model | Different ecosystem; migration needed | Your platform is primarily AWS |
| Google Cloud Storage Archive/Coldline | Archive in GCP | Integrated with GCS ecosystem | Different ecosystem; migration needed | Your platform is primarily GCP |
| On-prem object storage (e.g., MinIO) + cold disks/tape | Data sovereignty, on-prem constraints | Full control, may reduce cloud egress | Ops burden, durability/process risk, scaling limits | You must keep data on-prem and accept operational overhead |
15. Real-World Example
Enterprise example: Regulated audit archive for financial exports
- Problem
-
A financial institution must retain monthly and quarterly reports and supporting datasets for 7+ years. Access is rare, but audits require retrieving specific months quickly (within a day is acceptable).
-
Proposed architecture
- Storage account (Archive repository) with:
- Private endpoint + private DNS
- Azure AD RBAC (separate roles for writers vs auditors)
- Lifecycle:
- Hot for 30 days (ingestion/validation window)
- Cool for 11 months
- Archive after 12 months
- Diagnostic logs to Azure Monitor / Log Analytics
- Optional: Key Vault for CMK encryption (if required by policy)
-
Retrieval workflow:
- Auditor requests dataset by period
- Automation triggers rehydration for relevant prefixes
- After rehydration completes, a time-bound SAS is issued for download
-
Why Archive Storage was chosen
- Lowest cost for multi-year retention with Azure-native governance.
-
Offline retrieval fits audit timelines.
-
Expected outcomes
- Significant reduction in storage spend versus keeping everything Hot/Cool.
- Controlled retrieval events with logging and approvals.
- Reduced risk of public exposure via private networking and RBAC.
Startup/small-team example: Low-cost long-term backups of generated artifacts
- Problem
-
A small SaaS team wants to keep monthly exports and old customer attachments for compliance and customer support, but access is rare and budgets are tight.
-
Proposed architecture
- Single Storage account (Standard LRS) with:
- Containers:
active/andarchive/ - Simple lifecycle rules that move
archive/blobs to Archive after a short buffer period - Basic RBAC for the team
- Containers:
-
Retrieval:
- Support engineer runs a script to rehydrate and fetch a specific blob when needed
-
Why Archive Storage was chosen
-
Minimal operational overhead (still Blob Storage) and very low long-term cost.
-
Expected outcomes
- Lower monthly storage bills.
- Clear operational behavior (“rehydrate first”) that can be documented in a runbook.
16. FAQ
-
Is “Archive Storage” a separate Azure service I deploy?
Usually no. In Azure, “Archive Storage” commonly refers to the Archive access tier for Azure Blob Storage within a Storage account. -
Can I read an archived blob immediately?
No. Archive blobs are offline. You must rehydrate them to Hot or Cool before reading. -
How long does rehydration take?
It can take hours. The exact time depends on platform behavior and options available. Verify current expectations in official docs. -
Does Archive Storage reduce costs automatically?
Only if you actually move data into Archive and avoid frequent retrieval. Use lifecycle management and good data classification. -
Should I archive everything by default?
No. Archive is not appropriate for active datasets. Use Hot/Cool for online needs and Archive for deep retention. -
What’s the difference between Cool and Archive?
Cool is online but cheaper than Hot; Archive is offline and cheaper than Cool but requires rehydration to access. -
Can I set a container default tier to Archive?
Typically, the default account access tier is Hot or Cool. Archive is generally a blob-level tier choice. Verify current platform behavior in docs. -
What happens if an app tries to read an archived blob?
The read will fail until the blob is rehydrated. -
Can lifecycle rules move blobs to Archive automatically?
Yes, lifecycle management can automate tiering. Validate your rule scope and timing. -
Are there minimum retention periods for Archive tier?
Archive tier often has a minimum duration and early deletion charges. Verify the current terms on the pricing/docs for your region. -
Is Archive Storage good for ransomware protection?
It can help reduce exposure of older data (since it’s offline), but it’s not sufficient alone. Use RBAC, immutability (if required), soft delete/versioning, and monitoring. -
Can I use private endpoints with archived blobs?
Yes—private endpoints apply at the storage account endpoint level, regardless of tier. -
Do I need special SDKs to use Archive tier?
No. You use standard Azure Blob Storage APIs/SDKs and set the blob tier. -
Can I perform server-side copy operations from archived blobs?
Some operations may be constrained by the offline nature of Archive. Verify specific operation support (copy, snapshot behaviors) in official docs. -
How do I prevent accidental archiving of active data?
Use clear prefixes/tags, separate containers, test lifecycle rules, and add monitoring/alerts on tier changes. -
What’s the best way to design retrieval workflows?
Treat retrieval as a controlled operation: identify target blobs, request rehydration, wait for completion, then provide time-bound access (SAS) or copy to a working container.
17. Top Online Resources to Learn Archive Storage
| Resource Type | Name | Why It Is Useful |
|---|---|---|
| Official documentation | Azure Blob Storage documentation – https://learn.microsoft.com/azure/storage/blobs/ | Canonical docs for Blob Storage concepts, APIs, security, and operations |
| Official documentation | Access tiers overview – https://learn.microsoft.com/azure/storage/blobs/access-tiers-overview | Core concepts for Hot/Cool/Archive, tiering, and behavior |
| Official documentation | AzCopy documentation – https://learn.microsoft.com/azure/storage/common/storage-use-azcopy-v10 | Practical tooling for moving large datasets efficiently |
| Official pricing | Azure Blob Storage pricing – https://azure.microsoft.com/pricing/details/storage/blobs/ | Authoritative pricing dimensions and tier costs by redundancy/region |
| Cost estimation | Azure Pricing Calculator – https://azure.microsoft.com/pricing/calculator/ | Build realistic estimates for storage, retrieval, and data transfer |
| Architecture center | Azure Architecture Center – https://learn.microsoft.com/azure/architecture/ | Reference architectures and design guidance for production systems |
| Official documentation | Storage scalability and performance targets – https://learn.microsoft.com/azure/storage/common/scalability-targets-standard-account | Helps architects plan limits, throughput expectations, and account design |
| Official tutorials | Azure Storage samples (GitHub org) – https://github.com/Azure/azure-sdk-for-python/tree/main/sdk/storage | SDK samples (language-specific) for blob operations (verify updated paths per language) |
| Official videos | Microsoft Azure YouTube – https://www.youtube.com/@MicrosoftAzure | Webinars and walkthroughs; search within for “Blob access tiers” and “lifecycle management” |
| Community learning | Microsoft Q&A (Azure Storage tag) – https://learn.microsoft.com/answers/tags/189/azure-storage | Real-world troubleshooting patterns (validate answers against official docs) |
18. Training and Certification Providers
| Institute | Suitable Audience | Likely Learning Focus | Mode | Website URL |
|---|---|---|---|---|
| DevOpsSchool.com | DevOps engineers, cloud engineers, SREs, platform teams | Azure + DevOps practices, automation, operations, CI/CD, IaC | Check website | https://www.devopsschool.com/ |
| ScmGalaxy.com | Beginners to intermediate engineers | DevOps fundamentals, tooling, cloud introductions | Check website | https://www.scmgalaxy.com/ |
| CLoudOpsNow.in | Cloud ops practitioners | Cloud operations, monitoring, governance, reliability | Check website | https://www.cloudopsnow.in/ |
| SreSchool.com | SREs, operations teams | Reliability engineering, incident response, monitoring | Check website | https://www.sreschool.com/ |
| AiOpsSchool.com | Ops + data/automation learners | AIOps concepts, automation for operations, monitoring analytics | Check website | https://www.aiopsschool.com/ |
19. Top Trainers
| Platform/Site | Likely Specialization | Suitable Audience | Website URL |
|---|---|---|---|
| RajeshKumar.xyz | DevOps/cloud training content | Beginners to intermediate practitioners | https://rajeshkumar.xyz/ |
| devopstrainer.in | DevOps training programs | Engineers seeking structured DevOps learning | https://www.devopstrainer.in/ |
| devopsfreelancer.com | Freelance DevOps help/training | Teams needing hands-on guidance | https://www.devopsfreelancer.com/ |
| devopssupport.in | DevOps support and learning | Ops teams needing implementation support | https://www.devopssupport.in/ |
20. Top Consulting Companies
| Company Name | Likely Service Area | Where They May Help | Consulting Use Case Examples | Website URL |
|---|---|---|---|---|
| cotocus.com | Cloud/DevOps consulting (verify offerings) | Architecture, migration planning, operationalization | Designing an archive strategy, lifecycle policies, and secure network patterns | https://cotocus.com/ |
| DevOpsSchool.com | DevOps and cloud consulting/training (verify offerings) | DevOps processes, automation, platform engineering | Implementing IaC for storage accounts, RBAC, monitoring, cost governance | https://www.devopsschool.com/ |
| DEVOPSCONSULTING.IN | DevOps consulting (verify offerings) | Delivery pipelines, cloud ops, automation | Building automated archive/rehydration workflows and operational runbooks | https://www.devopsconsulting.in/ |
21. Career and Learning Roadmap
What to learn before Archive Storage
- Azure fundamentals:
- Subscriptions, resource groups, regions
- IAM basics (Azure AD, RBAC)
- Azure Storage fundamentals:
- Storage accounts, containers, blobs
- Authentication methods (RBAC vs SAS vs keys)
- Networking basics:
- Private endpoints, DNS, firewall rules (for secure storage)
What to learn after Archive Storage
- Lifecycle automation at scale:
- Azure Policy enforcement
- Tag-driven governance
- Data governance:
- Microsoft Purview cataloging and classification
- Security hardening:
- CMK with Key Vault, key rotation strategies
- Monitoring + alerting patterns for storage access
- Data pipelines:
- Data Factory/Synapse patterns for landing zones and tier transitions
- DR and resilience:
- Redundancy choices and multi-region design tradeoffs
Job roles that use it
- Cloud Solution Architect
- Platform Engineer
- DevOps Engineer / SRE
- Security Engineer (data protection and governance)
- Data Engineer (data lake cost optimization)
- Cloud Operations Engineer
Certification path (Azure)
Archive Storage is not usually tested as a standalone product, but it appears within storage and architecture objectives: – AZ-104 (Azure Administrator): storage accounts, access control, networking basics – AZ-305 (Azure Solutions Architect Expert): architecture tradeoffs, governance, security, cost – Security-focused tracks can also be relevant (for governance and data protection)
Always verify current certification objectives on Microsoft Learn: – https://learn.microsoft.com/credentials/
Project ideas for practice
- Implement lifecycle policies for a “data lake zones” layout (hot → cool → archive).
- Build a rehydration automation workflow (Function/Logic App) triggered by an approval ticket.
- Implement private endpoint + private DNS and validate access from a locked-down VNet.
- Create cost alerts for retrieval and egress; simulate retrieval spikes and validate alerting.
- Build a retrieval index: store metadata in Hot and payloads in Archive.
22. Glossary
- Azure Storage account: The top-level Azure resource that provides access to Blob, File, Queue, and Table services (depending on configuration).
- Blob Storage: Azure’s object storage service for unstructured data.
- Container: A grouping of blobs within Blob Storage (like a top-level folder).
- Blob: An object/file stored in a container (commonly a block blob for files).
- Access tier: A pricing and behavior classification for blobs (Hot, Cool, Archive).
- Archive Storage: Commonly refers to the Archive access tier of Azure Blob Storage.
- Rehydration: The process of changing a blob from Archive to Hot/Cool so it becomes readable again.
- RBAC: Role-Based Access Control using Azure AD identities and role assignments.
- SAS (Shared Access Signature): A token granting time-scoped permissions to storage resources.
- Private endpoint (Private Link): A private IP address in your VNet that connects to an Azure service endpoint.
- Lifecycle management: Rule-based automation that transitions blobs between tiers or deletes them based on conditions.
- CMK (Customer-Managed Keys): Encryption keys you manage (often stored in Key Vault) instead of Microsoft-managed keys.
- Egress: Data transferred out of Azure to the internet or other networks; often billable.
- Immutability / WORM: Controls that prevent modification/deletion for a retention period (feature availability and behavior must be verified for your scenario).
23. Summary
Archive Storage (Azure) is the Archive access tier in Azure Blob Storage, built for deep, long-term, low-cost retention of rarely accessed data. It matters because it can dramatically reduce storage cost for compliance archives, backup exports, and historical datasets—without leaving the Azure Storage ecosystem.
The key tradeoff is operational: Archive is offline, so you must rehydrate before reading, and retrieval introduces time delays and additional costs. Cost success depends on disciplined lifecycle policies, controlled retrieval workflows, and monitoring for unexpected rehydration/egress.
If your data is rarely accessed and your business can tolerate hours to retrieve it, Azure Archive Storage is a strong fit. If you need immediate reads or frequent access, prefer Hot or Cool.
Next learning step: build a small proof-of-concept with lifecycle rules, RBAC, and (optionally) private endpoints—then model costs using the official pricing page and Azure Pricing Calculator before rolling out at scale.