Metric Trees: Decomposing 'Why Did X Change'

Back in Module 4's Case Study A, signups dropped 8% and you were asked to find out why. You worked through it in order: was it noise? which channel? which funnel step? which segment? any external cause? That sequence wasn't arbitrary - it's a general-purpose framework called a metric tree, and once you can name it, you can apply it to any "why did X change" question cold, not just the one you already practiced.

What a metric tree is

A metric tree decomposes one output metric into the smaller input metrics that mechanically produce it - written as a formula, then drawn as a hierarchy. The classic example:

Revenue
  = Traffic × Conversion Rate × Average Order Value

Each of those three inputs can itself be decomposed further (Traffic = Organic + Paid + Referral; Conversion Rate = Landing→Cart × Cart→Checkout × Checkout→Paid; ...). The tree stops branching once you reach something you can directly act on or directly attribute to a cause.

Rebuilding Case Study A as a tree

Here's Case Study A's investigation, formalized as a metric tree instead of a narrative:

Weekly Signups (down 8%)
├── Step 1: Noise check - is 8% outside the trailing std? (No → it's real, keep going)
├── Step 2: Traffic by source
│   ├── Paid Social: flat
│   ├── Organic: flat
│   └── Referral: flat            → traffic itself isn't the driver
├── Step 3: Funnel step conversion
│   ├── Landed → Started signup: DOWN            ← the drop lives here
│   └── Started → Completed signup: flat
├── Step 4: Segment cut on the affected step
│   └── Landed → Started, by device: mobile down sharply, desktop flat
└── Step 5: External cause check
    └── Mobile app update shipped that week → broke a tracking pixel on mobile landing

Every branch is a hypothesis; every leaf is either ruled out (flat) or where the actual story lives (down). This is the same five-step order from "Your Analyst Checklists" - noise → source → funnel step → segment → external cause - just drawn as a tree instead of a checklist.

The reusable template

Next time you get a "why did X change" question - in an interview, in a Slack DM from your CEO - run this:

1. Noise check. Is the change outside normal period-to-period variation (trailing std, or a control period)? If not, stop - there's nothing to explain.
2. Write the formula. X = A × B × C (or A + B - C, whichever is mechanically true). This is the top level of the tree.
3. Check each input. Which of A, B, C actually moved? Rule out the ones that didn't.
4. Segment the input(s) that moved. Cut by channel, device, geo, plan tier, cohort - find where inside that input the movement is concentrated.
5. Check for an external cause on the segment you've isolated - a ship date, a pricing change, a competitor, a holiday, a tracking regression.

A second worked example (not Case Study A)

Try the template on a new problem: "Weekly Active Users (WAU) is down 5%."

WAU = Retained Actives (were active last week, still active) + Newly Activated - N/A

1. Noise check: 5% is outside trailing std → real.
2. Formula: WAU this week = Retained + New Activations. Which moved? Retained is flat; New Activations is down 18%.
3. Segment New Activations by acquisition channel: paid channel's activation rate cratered, organic is flat.
4. External cause: paid channel changed its targeting last week, sending lower-intent traffic that signs up but never activates.

Same five steps, completely different metric. That's the point of naming the framework - you don't have to reinvent the investigation order every time.

A simple decomposition helper in Python

For metrics related multiplicatively (Revenue = Traffic × CVR × AOV), you can quantify each input's approximate contribution to the overall % change:

def decompose_multiplicative_change(baseline: dict, current: dict) -> None:
    # baseline/current: dicts of the same input metrics, e.g. {"traffic": ..., "cvr": ..., "aov": ...}
    overall_baseline = 1.0
    overall_current = 1.0
    for k in baseline:
        overall_baseline *= baseline[k]
        overall_current *= current[k]
    overall_pct_change = (overall_current / overall_baseline) - 1

    print(f"Overall change: {overall_pct_change:+.1%}")
    for k in baseline:
        pct_change = (current[k] / baseline[k]) - 1
        print(f"  {k}: {pct_change:+.1%} (baseline {baseline[k]:,.3f} -> current {current[k]:,.3f})")

# Revenue = Traffic x Conversion Rate x AOV
decompose_multiplicative_change(
    baseline={"traffic": 100_000, "cvr": 0.030, "aov": 1200},
    current={"traffic": 108_000, "cvr": 0.027, "aov": 1215},
)

Traffic grew, CVR fell, AOV barely moved - the printout shows each input's individual swing next to the overall net change, exactly the "which input actually moved" question from step 3 of the template.

Takeaway

Every "why did X change" question has a metric tree hiding inside it, whether or not anyone draws it out. Write the formula, check which input moved, segment that input, and check for an external cause. You already ran this exact playbook once, in Case Study A - now you have the name and the template to run it cold on a metric you've never seen before.