Candidate telemetry diagnostic, error autopsy, and step-by-step query construction walkthrough.
Live aggregated metrics across candidate sandbox attempts
16 solved
First attempt fail
Evaluated submissions
Median time to solve
Unlocked answer
You're the analyst supporting a subscription SaaS company's growth team. They redesigned the signup page and ran a two-week A/B test:
Compute the absolute lift, run a two-sample z-test for proportions, and compute a 95% confidence interval on the difference (treatment − control).
Return a dict result with keys absolute_lift, z_stat, p_value, ci_low, ci_high — all rounded to 4 decimal places.
Operating on a DataFrame slice without `.copy()`, producing `SettingWithCopyWarning`, or using `.apply(axis=1)` with custom Python functions when vectorized NumPy / pandas column operations could run 50x faster with 0 memory overhead.
Interviewers assess whether you write idiomatic, vectorized pandas code instead of slow row-by-row procedural Python loops.
Construct the solution logically from first principles to avoid typical edge case pitfalls.
Use boolean masks with `.loc[row_mask, col_list]` to avoid chained assignment warnings.
subset = df.loc[df['status'] == 'completed'].copy()
Use numpy/pandas native operations like `.groupby()`, `.transform()`, or column arithmetic.
df['rev_share'] = df['revenue'] / df.groupby('country')['revenue'].transform('sum')Sort values and reset index to match the required evaluation output contract.
result = df.sort_values('rev_share', ascending=False).reset_index(drop=True)import math
count_c, nobs_c = 1230, 10000
count_t, nobs_t = 1310, 10000
p_c = count_c / nobs_c
p_t = count_t / nobs_t
absolute_lift = p_t - p_c
# Two-sample z-test for proportions, pooled variance under H0 (matches
# statsmodels.stats.proportion.proportions_ztest's default prop_var=False).
p_pool = (count_t + count_c) / (nobs_t + nobs_c)
se_pool = math.sqrt(p_pool * (1 - p_pool) * (1 / nobs_t + 1 / nobs_c))
z_stat = absolute_lift / se_pool
# Two-sided p-value from the standard normal survival function, via erfc
# (math.erfc is exact stdlib, no scipy needed): P(|Z| > |z|) = erfc(|z|/sqrt(2)).
p_value = math.erfc(abs(z_stat) / math.sqrt(2))
# 95% Wald CI on the difference, *unpooled* SE (matches
# statsmodels.stats.proportion.confint_proportions_2indep(method="wald", compare="diff")).
se_diff = math.sqrt(p_t * (1 - p_t) / nobs_t + p_c * (1 - p_c) / nobs_c)
z_975 = 1.959963984540054 # exact two-sided 95% critical value
ci_low = absolute_lift - z_975 * se_diff
ci_high = absolute_lift + z_975 * se_diff
result = {
"absolute_lift": round(float(absolute_lift), 4),
"z_stat": round(float(z_stat), 4),
"p_value": round(float(p_value), 4),
"ci_low": round(float(ci_low), 4),
"ci_high": round(float(ci_high), 4),
}
Real code patterns candidates submit that fail the grading suite.
results = []
for i, row in df.iterrows():
results.append(row['val'] * 2)Three recurring syntax and semantic traps relevant to this problem domain.
Modifying a filtered slice like df[df.age > 30]['salary'] = 5000 modifies an ephemeral copy instead of the underlying dataframe.
df[df['active'] == True]['status'] = 'verified' # ❌ SettingWithCopyWarning
df.loc[df['active'] == True, 'status'] = 'verified' # ✅ Idiomatic inplace assignment
Calling df.groupby('category').agg(...) places 'category' into the MultiIndex/Index, making subsequent column references fail.
res = df.groupby('dept')['salary'].mean(); print(res['dept']) # ❌ KeyErrorres = df.groupby('dept', as_index=False)['salary'].mean() # ✅ Keeps column intactIterating rows with for index, row in df.iterrows() is an anti-pattern in data science interviews.
for i, r in df.iterrows(): df.at[i, 'total'] = r['price'] * r['qty'] # ❌ Slow
df['total'] = df['price'] * df['qty'] # ✅ C-speed vectorization
Launch our in-browser coding environment. Run queries, view execution plans, and get instant comparative diff grading with no setup.