The 6-Step Systematic EDA Blueprint & Outlier Detection

The 6-Step EDA Blueprint: From Messy Table to Trustworthy Insights You open the warehouse extract for the first time — 280 rows, 6 columns, 3 months of orders — and nothing looks clean. Dates are strings, one product category has a typo, two rows are exact duplicates, and a $12,000 "Office Supplies" order is clearly a data-entry error. Do you fix it by gut feel? No. You run 6 checks in order , like a pilot's pre-flight checklist. Skip one and the chart you send to leadership is wrong. Run all six and every chart after it is trustworthy. We will run the whole checklist on a tiny 10-row sample you can trace by hand — then the same code scales to 280 rows and to 280,000.

0. The Messy Sample We Will Clean Together This is what pd.read_sql_query("SELECT * FROM orders", engine) might return before any cleaning — deliberately messy:

id customer_id transaction_date product_category quantity unit_price total_amount 1001 C001 2024-06-22 Electronics 1 899.00 899.00 1002 C002 2024-02-12 Furniture 5 80.00 400.00 1003 C002 2024-02-12 Furniture 5 80.00 400.00 1004 — 2024-03-18 NULL 2 250.00 500.00 1005 C003 2024-04-01 Office Supplies 4 120.00 480.00 1006 C004 2024-06-22 Office Supplies 80 150.00 12000.00 1007 C001 2024-06-28 Apparel 3 45.00 135.00 1008 C005 2024-02-12 Electronics 1 1200.00 1200.00 Spot the issues: row 1002/1003 duplicate, row 1004 missing customer + category (yellow), row 1006 quantity 80 × $150 should be $12,000 but the unit price is clearly off by 10× — that 12000 is an outlier (red). Each step below catches one of them.

Step 1: Structural Audit — What Shape Is the Data? Before any fix, ask: how big, what types, what memory? This is the inventory check. You catch that transaction_date arrived as object (string) not datetime , and that total_amount should equal quantity × unit_price but doesn't for row 1006.

print(df.shape) # (10, 7) — rows, columns print(df.info()) # dtypes + non-null counts — date shows as object! print(df.dtypes) # quick type scan print(df.memory_usage(deep=True).sum() / 1024, "KB") Output tells you: 10 rows × 7 columns, transaction_date object needs parsing, and df['total_amount'] != df