Metric Trees: Decomposing 'Why Did X Change'

# Metric Trees: Decomposing "Why Did X Change" Back in Module 4's Case Study A, signups dropped 8% and you were asked to find out why. You worked through it in order: was it noise? which channel? which funnel step? which segment? any external cause? That sequence wasn't arbitrary - it's a general-purpose framework called a **metric tree**, and once you can name it, you can apply it to any "why did X change" question cold, not just the one you already practiced. ## What a metric tree is A metric tree decomposes one output metric into the smaller input metrics that mechanically produce it - written as a formula, then drawn as a hierarchy. The classic example: ``` Revenue = Traffic × Conversion Rate × Average Order Value ``` Each of those three inputs can itself be decomposed further (Traffic = Organic + Paid + Referral; Conversion Rate = Landing→Cart × Cart→Checkout × Checkout→Paid; ...). The tree stops branching once you reach something you can directly act on or directly attribute to a cause. ## Rebuilding Case Study A as a tree Here's Case Study A's investigation, formalized as a metric tree instead of a narrative: ``` Weekly Signups (down 8%) ├── Step 1: Noise check - is 8% outside the trailing std? (No → it's real, keep going) ├── Step 2: Traffic by source │ ├── Paid Social: flat │ ├── Organic: flat │ └── Referral: flat → traffic itself isn't the driver ├── Step 3: Funnel step conversion │ ├── Landed → Started signup: DOWN ← the drop lives here │ └── Started → Completed signup: flat ├── Step 4: Segment cut on the affected step │ └── Landed → Started, by device: mobile down sharply, desktop flat └── Step 5: External cause check └── Mobile app update shipped that week → broke a tracking pixel on mobile landing ``` Every branch is a hypothesis; every leaf is either ruled out (flat) or where the actual story lives (down). This is the same five-step order from "Your Analyst Checklists" - noise → source → funnel step → segment → external cause - just drawn as a tree instead of a checklist. ## The reusable template Next time you get a "why did X change" question - in an interview, in a Slack DM from your CEO - run this: 1. **Noise check.** Is the change outside normal period-to-period variation (trailing std, or a control period)? If not, stop - there's nothing to explain. 2. **Write the formula.** X = A × B × C (or A + B - C, whichever is mechanically true). This is the top level of the tree. 3. **Check each input.** Which of A, B, C actually moved? Rule out the ones that didn't. 4. **Segment the input(s) that moved.** Cut by channel, device, geo, plan tier, cohort - find where inside that input the movement is concentrated. 5. **Check for an external cause** on the segment you've isolated - a ship date, a pricing change, a competitor, a holiday, a tracking regression. ## A second worked example (not Case Study A) Try the template on a new problem: **"Weekly Active Users (WAU) is down 5%."** ``` WAU = Retained Actives (were active last week, still active) + Newly Activated - N/A ``` 1. Noise check: 5% is outside trailing std → real. 2. Formula: WAU this week = Retained + New Activations. Which moved? Retained is flat; New Activations is down 18%. 3. Segment New Activations by acquisition channel: paid channel's activation rate cratered, organic is flat. 4. External cause: paid channel changed its targeting last week, sending lower-intent traffic that signs up but never activates. Same five steps, completely different metric. That's the point of naming the framework - you don't have to reinvent the investigation order every time. ## A simple decomposition helper in Python For metrics related multiplicatively (Revenue = Traffic × CVR × AOV), you can quantify each input's approximate contribution to the overall % change: ```python def decompose_multiplicative_change(baseline: dict, current: dict) -> None: # baseline/current: dicts of the same input metrics, e.g. {"traffic": ..., "cvr": ..., "aov": ...} overall_baseline = 1.0 overall_current = 1.0 for k in baseline: overall_baseline *= baseline[k] overall_current *= current[k] overall_pct_change = (overall_current / overall_baseline) - 1 print(f"Overall change: {overall_pct_change:+.1%}") for k in baseline: pct_change = (current[k] / baseline[k]) - 1 print(f" {k}: {pct_change:+.1%} (baseline {baseline[k]:,.3f} -> current {current[k]:,.3f})") # Revenue = Traffic x Conversion Rate x AOV decompose_multiplicative_change( baseline={"traffic": 100_000, "cvr": 0.030, "aov": 1200}, current={"traffic": 108_000, "cvr": 0.027, "aov": 1215}, ) ``` Traffic grew, CVR fell, AOV barely moved - the printout shows each input's individual swing next to the overall net change, exactly the "which input actually moved" question from step 3 of the template. ## Takeaway Every "why did X change" question has a metric tree hiding inside it, whether or not an