Cohort Analysis

Grouping users by a shared characteristic (signup date, acquisition source, plan) and tracking their behavior over time —to see retention, engagement, and revenue curves by cohort.

Why

An aggregate metric ('retention is 40%') hides the story. Retention for the August cohort is 52% (great), retention for the September cohort is 28% (terrible), and the aggregate '40%' is an average that describes neither. Cohort analysis is the discipline of splitting users into groups (by signup week, by acquisition source, by plan, by feature usage) and tracking each group's metric over time, so you can see which cohorts are healthy and which are not, and —crucially —when the change happened. A cohort table (rows = signup week, columns = week N after signup, cells = % retained) is the most powerful diagnostic in analytics: it shows that 'retention dropped' is actually 'the Sep 15 cohort retains at 20% vs the Sep 8 cohort at 45%', which points to a change that happened around Sep 15 (a new onboarding flow, a broken feature, a different acquisition source). Without cohorts, you see a number go down; with cohorts, you see which users and when, and that is what makes the drop fixable.

When

When tracking retention, engagement, or revenue over time. When an aggregate metric changes and you need to find which segment drove the change. Always for retention reporting.

How

1. Choose the cohort dimension: signup date (most common), acquisition source, plan, first feature used, or any characteristic shared at a point in time
2. Choose the cohort granularity: weekly (good for B2B, ~50-200 users/week), monthly (good for consumer, ~1000+ users/month), daily (only if > 500 users/day)
3. Choose the metric to track: retention (% active in week N), revenue (cumulative or per-period), engagement (sessions per user), conversion (% completed goal)
4. Build the cohort table: rows = cohorts (e.g., signup week), columns = periods after signup (week 0, week 1, week 2, ...), cells = metric value
5. For retention: cell[cohort=C, period=N] = % of users in cohort C who were active in period N (week 0 = 100% by definition)
6. Read the table diagonally: a diagonal shows users at the same calendar time (e.g., all cohorts in week 5 of their life). A horizontal row shows one cohort's decay curve. A vertical column compares cohorts at the same age
7. Look for: (a) cohort-to-cohort variation (is Sep 15 worse than Sep 8?), (b) the decay shape (linear, exponential, cliff), (c) the plateau (where does retention stabilize?)
8. When you find a bad cohort, investigate what changed at that cohort's signup time: a new feature shipped, a different ad campaign, a pricing change —the cohort's behavior is a signal of what happened
9. Compare cohorts by dimension: organic cohorts vs paid cohorts, free vs pro —the dimension often explains the variation

Example

# Cohort table: weekly retention by signup week
Metric: % active in week N (at least 1 core action)

Signup   | W0   | W1   | W2   | W3   | W4   | W5
week     |      |      |      |      |      |
---------|------|------|------|------|------|-----
Aug 25   | 100% | 62%  | 48%  | 41%  | 38%  | 36%
Sep 01   | 100% | 64%  | 50%  | 43%  | 40%  | 38%
Sep 08   | 100% | 65%  | 51%  | 44%  | 41%  | 39%
Sep 15   | 100% | 41%  | 28%  | 22%  | 19%  | 17%  <- DROP
Sep 22   | 100% | 43%  | 29%  | 23%  | 20%  | 18%  <- still bad
Sep 29   | 100% | 63%  | 49%  | 42%  | 39%  |  --

Reading:
  Horizontal (Sep 08): 100 -> 65 -> 51 -> 44 -> 41 -> 39 (healthy decay)
  Vertical (W1):       62, 64, 65, 41, 43, 63  <- Sep 15 and 22 are low
  Diagonal (this week): 36, 38, 39, 17, 18, --  <- Sep 15/22 cohorts dragging

Diagnosis: Sep 15 cohort W1 retention dropped from 65% to 41% (-24pp).
  What changed around Sep 15?
  -> Sep 14: new onboarding flow shipped (v2.1)
  -> Sep 15: paid ad campaign on a new channel started
  -> Both are candidates; segment Sep 15 cohort by source:
     organic: W1 = 64% (healthy, same as prior cohorts)
     paid:    W1 = 28% (terrible, new ad channel has bad targeting)
  -> Root cause: the new paid channel, not the onboarding change.

Action: pause the new paid channel, re-evaluate targeting. Onboarding
  v2.1 is fine (organic cohort retains normally).

Common Problems

⚠Using monthly cohorts when weekly would reveal the change —a monthly cohort averages 4 weeks and hides a mid-month change; the Sep 15 drop is invisible in the 'September' cohort, which averages the good weeks and the bad.
⚠Comparing cohorts at different ages —'cohort A retains at 40% and cohort B at 35%' is meaningless if A is at week 4 and B is at week 2; always compare at the same period (same column).
⚠Tracking only retention and ignoring revenue/engagement —a cohort with 40% retention but 2x revenue per user is healthier than one with 50% retention and 0.5x revenue, and retention alone misses this.
⚠Attributing a cohort change to the wrong cause —Sep 15 retention dropped and the team blames the Sep 14 onboarding change, but segmenting by source reveals the paid channel is the cause; without the segment, the team reverts the onboarding and keeps the bad ads.

Best Practices

✓Use weekly cohorts for B2B or products with < 10k monthly users —weekly granularity catches mid-month changes that monthly cohorts average away, and the smaller cohort size is still statistically useful at this scale.
✓Always compare cohorts at the same age (same column), never at the same calendar time —comparing at the same age isolates the cohort effect from the time effect, which is the entire point of cohort analysis.
✓Track multiple metrics per cohort (retention, revenue, engagement, conversion) —a cohort that retains less but pays more may be more valuable, and the multi-metric view prevents optimizing one metric at the expense of another.
✓When a cohort deviates, segment by every dimension (source, plan, feature, geography) before attributing the cause —the segment is what separates 'the onboarding change caused it' from 'the paid channel caused it', and acting on the wrong cause is worse than not acting.