---
title: 'Microsoft Azure: Warden automated incident detection in IcM'
slug: azure-warden
stable_id: 7405cae4f09ed2d4
company: Microsoft Azure
function_code: it_operations
pattern_codes:
  - coordination_compression
  - continuous_decisioning
evidence_strength: verified
publication_tier: showcase
freshness: current
reviewed_at: '2026-08-23'
updated_at: '2026-08-22'
source_quality_summary: 1 peer reviewed, 1 primary; publication outcomes are verified.
caveat_summary: >-
  The public paper does not report an absolute minute delta; time is in normalized time
  units. The 68% figure is among successfully detected incidents, not all incidents.
  Comparison is detection latency versus historical human declaration for the same
  incident, not operating cost or full mitigation time. No public 2026 confirmation that
  Warden remains the production detector under that name.
collections:
  - queue-eliminated
  - embodied-work
bundle_version: 1.0.0
bundle_fingerprint: sha256:c23c6cc2b88153d008ea8fda928f632ce0011fc2d4c5036672a16e5d895bab93
canonical_url: https://brianletort.ai/transformations/azure-warden
---

# Microsoft Azure: Warden automated incident detection in IcM

A cross-service Azure incident is recognized when a human on-call, seeing only a partial service view, pieces together cascading alerts and declares the incident.

Function: IT operations. Patterns: Coordination compression; Continuous decisioning. Evidence: verified.

Freshness: current. Reviewed: 2026-08-23. Updated: 2026-08-22.


Source quality: 1 peer reviewed, 1 primary; publication outcomes are verified.

## Before

1. **Service monitors** — Fire alerts inside individual Azure services as local symptoms appear. (control: Per-service monitors; each team has only a partial view of the platform)
2. **On-call engineers** — Triage service alerts and collaborate to recognize one cross-service incident. (control: Human pattern recognition across teams; motivating example took nearly 50 minutes to declare)
3. **Incident Commander** — Coordinates mitigation after the incident is declared. (control: IcM incident declaration)

## After

1. **Service monitors** — Continue to fire alerts; Warden ingests them globally from IcM. (control: Selected monitors with high weighted mutual information versus historical incidents)
2. **Warden detector** — Every minute, scores recent alerts and flags potential incidents at a greater-than-90% precision operating point. (control: Precision held above 90% (recall ~58%, F1 0.71); late detections relative to humans are not notified)
3. **On-call engineers** — Receive a Recommended Actions notification with grouped incident-indicating alerts and an Emerging Issues dashboard, then prioritize and start cross-team collaboration. (control: Humans still declare and mitigate; Warden does not auto-close or auto-mitigate the incident)

## Decision rights

Warden decides when to notify and which alerts to group. Humans retain incident declaration, severity, and mitigation. Notifications are suppressed when Warden would be slower than the human declaration already in progress.

## Exception path

False positives are constrained by a >90% precision threshold. Incidents Warden misses wait for human declaration. Not all incidents are covered by monitors; the authors flag uncovered incidents as future work.

## Outcomes

- **Share of successfully detected incidents where Warden is faster than human declaration** (verified): On-call incident declaration time for the same incidents → Warden faster in ~68% of successfully detected incidents; median saving 21.8 NTUs, about 15% of time-to-mitigate for those cases. The public paper does not report an absolute minute delta; time is in normalized time units. The 68% figure is among successfully detected incidents, not all incidents. Comparison is detection latency versus historical human declaration for the same incident, not operating cost or full mitigation time. No public 2026 confirmation that Warden remains the production detector under that name.

## Executive lesson

The scarce job in a large cloud is not watching one service's alerts. It is noticing that several services are the same incident. Azure moved that recognition from on-call discussion to a global detector that pages the right people, and was faster than human declaration in about two-thirds of the incidents it caught.

## Anti-pattern

Reporting '68% faster than human' as a minute-scale MTTR cut, or implying Warden closed incidents without an on-call.

## Questions for leaders

- Is the outcome time-to-detect versus the same incident's human declaration, or a modeled counterfactual?
- What precision floor is high enough that on-calls will trust the page?
- If the detector is late, do we suppress the notification, or create a second incident?

## Sources

- [Fighting the Fog of War: Automated Incident Detection for Cloud Systems](https://www.usenix.org/system/files/atc21-li-liqun.pdf) — USENIX ATC 2021
- [Fighting the Fog of War: Automated Incident Detection for Cloud Systems](https://www.microsoft.com/en-us/research/publication/fighting-the-fog-of-war-automated-incident-detection-for-cloud-systems/) — Microsoft Research