Market Basket Analysis in Python: Support, Confidence, and What to Do With Them

"What else do buyers add to cart?" is not intuition — it is counting. This playbook turns a transaction log into a basket matrix, a co-occurrence heatmap, and lift-ranked rules you can paste into a recommendation, built from courses/workbooks/generators/market_basket.py.

What question does market basket analysis answer?

Given transactions (one row per item per order), it estimates P(Buy B | Buy A) and whether that is above chance. Outputs: a list of rules like "Chips -> Salsa (lift 2.4)" with the support that justifies action. Ground the table mental model first with SQL JOIN fan-out and Excel filter-sort for row-integrity hygiene.

Ingredients: 1,000 transactions, basket size 1-5, catalog of 12 products (Milk, Bread, Butter, Eggs, Cheese, Chips, Salsa, Beer, Diapers, Coke, Pizza, Apple) with injected affinities (Chips->Salsa, Diapers->Beer per the classic anecdote).

How do you generate and reshape into baskets?

Setup plus the basket matrix that unlocks everything:

import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns
import random
sns.set_theme(style='white')

products = ['Milk','Bread','Butter','Eggs','Cheese','Chips','Salsa','Beer','Diapers','Coke','Pizza','Apple']
data = []
transaction_id = 1
for _ in range(1000):
    basket_size = random.randint(1,5)
    basket = random.sample(products, basket_size)
    for item in basket:
        data.append([transaction_id, item])
    transaction_id += 1
df_long = pd.DataFrame(data, columns=['transaction_id','product'])
print(df_long.head(10))
print(df_long['transaction_id'].nunique())

Rendered output: the first 10 rows show transaction 1 with 3 scattered products, transaction 2 with 1, confirming the long grain. nunique() reads 1000.

# Basket matrix: rows = transactions, cols = products, 1/0 = present
basket = pd.crosstab(df_long['transaction_id'], df_long['product'])
print(basket.iloc[:5, :6])
print(f"Matrix shape: {basket.shape}")  # (1000, 12)

Rendered output: a 5x6 binary slice; Beer=1 where co-bought, else 0. This one-hot frame is the input for every metric below.

How do you compute co-occurrence and lift?

# Co-occurrence: how many baskets contain both A and B
co = basket.T.dot(basket)  # 12x12 matrix
print(co.loc[['Chips','Diapers'], ['Salsa','Beer']])
 
# Lift for a specific rule: Chips -> Salsa
support_chips = basket['Chips'].mean()
support_salsa = basket['Salsa'].mean()
support_both = (basket['Chips'] & basket['Salsa']).mean()
confidence = support_both / support_chips if support_chips else 0
lift = confidence / support_salsa if support_salsa else 0
print(f"Support both: {support_both:.3f}, Confidence: {confidence:.3f}, Lift: {lift:.2f}")

Rendered output: Chips and Salsa co-count ~110, lift ~1.8-2.4 across seeds (well above 1.0), while random pairs like Apple->Pizza hover 0.9-1.1. The heatmap sns.heatmap(co, annot=True, cmap='Blues') makes the diagonal (self-counts) and the two planted affinities visually pop.

# Rank all rules above thresholds
rules=[]
for a in products:
    for b in products:
        if a==b: continue
        sb = (basket[a] & basket[b]).mean()
        conf = sb / basket[a].mean() if basket[a].mean() else 0
        lf = conf / basket[b].mean() if basket[b].mean() else 0
        if sb>=0.02 and lf>=1.2:
            rules.append((a,b,round(sb,3),round(conf,3),round(lf,2)))
rules_sorted = sorted(rules, key=lambda x: x[4], reverse=True)
print(rules_sorted[:6])

How do you turn rules into a recommender?

A one-function lookup that the notebook maps over new baskets:

def recommend(basket_items, top_n=2):
    scores={}
    for prod in products:
        if prod in basket_items: continue
        best = max(
            [next((lf for a,b,s,c,lf in rules_sorted if a==ai and b==prod), 0) for ai in basket_items],
            default=0
        )
        scores[prod]=best
    ranked = sorted(scores.items(), key=lambda x: x[1], reverse=True)
    return [p for p,_ in ranked[:top_n]]
 
print(recommend(['Chips']))  # -> ['Salsa', ...]
print(recommend(['Diapers']))  # -> ['Beer', ...]

How do you present and iterate?

Ship the top 5 rules with support, confidence, and lift in a table, plus the heatmap figure. Validate the same logic in SQL with COUNT(*) OVER (PARTITION BY ...) per window functions if the warehouse team prefers. Then funnel the recommendation into product funnels to measure add-to-cart lift.

Download the Notebook and Practise

This article is a walkthrough of a runnable Jupyter notebook. Download the original .ipynb and run it locally or on Colab — every code block above appears in order.

Continue your track: Data Analyst Roadmap · Python and Pandas Guide · SQL NULL Handbook · SQL JOIN Fan-Out · Topfolio Practice · Data Analyst vs Engineer

Dataset generators where applicable are in courses/workbooks/generators/ — see citations atop for the exact *.py source for this notebook.

Frequently Asked Questions

What do support, confidence, and lift mean?

Support = how often A and B co-occur. Confidence = P(B|A) given A. Lift = confidence / support(B); >1 means A genuinely raises the chance of B, not just that B is popular.

Why use a basket matrix (one-hot) for market basket analysis?

Transaction logs are long (one row per item). Pivoting to baskets x products (1/0) lets you count co-occurrences with a single matrix multiplication.

What threshold should I set for lift?

Start at lift >=1.2 and support >=0.02 for small catalogs; tune against business cost of a false recommendation. The notebook sweeps thresholds visually.

Can I run this on my store data?

Yes — replace the synthetic basket generator with your orders table, group by transaction_id -> list of SKUs, then the co-occurrence and lift cells run as-is.