MTTR Tracking

Track Mean Time To Resolve as a leading indicator of incident response quality, alongside MTTA and MTTF.

Why

MTTR (Mean Time To Resolve) is the metric that tells you whether your incident response is improving — if it's dropping, your runbooks, tooling, and IC training are working; if it's rising, something is regressing (runbook rot, new complexity, on-call fatigue). But MTTR alone is a lagging composite: MTTA (time to ack) tells you if the paging system and escalation work; MTTF (time to find root cause) tells you if the runbook and observability work; MTTR - MTTF (time to fix) tells you if the rollback/fix tooling works. Tracking all three lets you diagnose where the response is slow: if MTTA is high, fix the paging; if MTTF is high, fix the runbook; if fix-time is high, fix the tooling. A single MTTR number tells you something is wrong; the breakdown tells you what.

When

For every incident. Review the trends (MTTA, MTTF, MTTR) monthly and per severity.

How

1. Instrument the incident timeline
   - T0 = alert fires; T1 = on-call acks; T2 = root cause found; T3 = incident resolved
   - MTTA = T1 - T0; MTTF = T2 - T1; MTTR = T3 - T0; fix-time = T3 - T2
   - Most paging systems record T0 and T1; you must record T2 (IC declares "root cause found")

2. Track per severity and per team
   - SEV1 MTTR target: <30m; SEV2: <2h; SEV3: <1 business day
   - Per team: a team with MTTR 2x the org average has a tooling or runbook gap

3. Review trends monthly, not just averages
   - Plot MTTA, MTTF, fix-time, MTTR over time; look for regressions, not just the mean
   - A rising MTTF across 3 months → runbooks are rotting or complexity is growing

4. Diagnose slow responses using the breakdown
   - High MTTA → paging system, escalation, or on-call schedule problem
   - High MTTF → observability, runbook, or SME-availability problem
   - High fix-time → rollback tooling, deploy pipeline, or permissions problem

5. Set improvement targets per component
   - "Reduce MTTA from 8m to 5m by Q3" → fix escalation policy
   - "Reduce MTTF from 20m to 10m by Q3" → add runbooks for top 5 alert types
   - Don't just target MTTR; target the component that's slowest

Example

# Incident metrics dashboard (last 30 days, SEV1)
incidents: 4
MTTA: 6.2m  (target <5m)  ⚠ slightly high
MTTF: 18m   (target <10m) ⚠ runbook or observability gap
fix-time: 4m (good — rollback tooling works)
MTTR: 28m   (target <30m) ✅ but only because fix-time is fast

# Trend (last 6 months):
#   MTTA:  4m → 5m → 6m → 7m → 6m → 6.2m  (creeping up — escalation policy?)
#   MTTF:  10m → 12m → 15m → 20m → 19m → 18m  (rising — runbooks rotting)
#   fix:   3m → 4m → 4m → 5m → 4m → 4m  (stable)

# Action: MTTF is the regression. Top 5 SEV1 alert types:
#   1. DbPrimaryDown (8 incidents, avg MTTF 22m) → runbook stale, rewrite
#   2. CheckoutSLOBurn (5 incidents, avg MTTF 15m) → runbook good, no action
#   3. ... → focus runbook work on DbPrimaryDown, biggest MTTF contributor

Common Problems

⚠Tracking only MTTR, not the breakdown — you know the response is slow but not whether to fix paging, runbooks, or tooling.
⚠Averages hide regressions — MTTR is 30m average but creeping from 20m to 40m over 6 months; plot the trend, not just the mean.
⚠No T2 (root cause found) timestamp — MTTF can't be computed, so the runbook/observability gap is invisible.
⚠MTTR target without per-component targets — 'reduce MTTR to 20m' is vague; 'reduce MTTF to 10m by rewriting the top 3 runbooks' is actionable.

Best Practices

✓Instrument T0, T1, T2, T3 for every incident and track MTTA, MTTF, fix-time, and MTTR — the breakdown tells you what to fix, not just that something is slow.
✓Plot trends over 6 months, not just the monthly average; a creeping MTTF is an early warning of runbook rot or growing complexity.
✓Set per-component improvement targets (MTTA, MTTF, fix-time) based on the slowest segment, not a single MTTR target.
✓Review per team and per severity; a team with 2x the org MTTR has a specific tooling or runbook gap to invest in.