In data science, analytics engineering, and Python engineering interviews, Pandas is the primary tool used to evaluate your practical data manipulation skills. Hiring panels assess whether you write clean, idiomatic, vectorized code or fall back on sluggish Python for-loops and fragile chained indexing.
In this guide, connecting to our playbook on Python for data analysis and Pandas fundamentals, we break down the top 20 Pandas interview questions, complete with executable code snippets, edge-case demonstrations, and memory optimization tactics.
Question 1: What is the exact difference between loc and iloc?
Feature / Criteria
python
import pandas as pddf = pd.DataFrame( {'sales': [120, 85, 240]}, index=['Store_A', 'Store_B', 'Store_C'])# Label-based slice with loc (inclusive)store_a_and_b = df.loc['Store_A':'Store_B', ['sales']]# Positional slice with iloc (exclusive of stop bound 2)first_two_rows = df.iloc[0:2, 0:1]
Question 2: How do you identify and diagnose SettingWithCopyWarning?
Answer:
This warning occurs when performing chained indexing like df[df['A'] > 2]['B'] = 10. Pandas cannot determine whether the outer slice returned an in-memory view or a copy, leading to unpredictable mutations.
Fix: Use .loc to perform slice assignment in a single, unambiguous step:
Question 3: How do you handle missing values in a production DataFrame?
Answer:
python
# Check for null counts across all columnsmissing_summary = df.isna().sum()# Drop rows where critical identifiers are missingclean_df = df.dropna(subset=['customer_id', 'order_id'])# Impute numerical columns with median and categorical columns with modedf['age'] = df['age'].fillna(df['age'].median())df['country'] = df['country'].fillna(df['country'].mode()[0])# Forward-fill time-series observationsdf['stock_price'] = df['stock_price'].ffill()
Pillar 2: GroupBy, Aggregations & Reshaping
Question 4: How do you perform multiple named aggregations with groupby()?
Answer:
Use named aggregation syntax inside .agg() for clean, readable output column names:
Question 5: What is the difference between pivot_table() and melt()?
Answer:
pivot_table() (Long to Wide): Reshapes data by turning unique row values into column headers and aggregating intersecting cells.
melt() (Wide to Long): Unpivots a wide DataFrame into a long format, consolidating multiple metric columns into key-value pairs.
python
# Unpivot a wide quarterly sales table into a long format:long_df = pd.melt( df, id_vars=['region'], value_vars=['Q1', 'Q2', 'Q3', 'Q4'], var_name='quarter', value_name='revenue')
Question 6: Merge vs Join vs Concat: When to use which?
Feature / Criteria
Pillar 3: Performance & Memory Optimization
Question 7: Why is apply() often considered an anti-pattern, and what should you use instead?
Answer:apply() is essentially an interpreted Python for-loop wrapped inside a Pandas method. It cannot take advantage of SIMD hardware instructions or memory caching.
DataFrame.map() (formerly applymap): Element-wise transformation across every single cell in a DataFrame. (Note: applymap() was renamed to map() in Pandas 2.1+).
Question 10: How do groupby().transform() and groupby().agg() differ in output shape and use cases?
Answer:
groupby().agg() (Aggregation / Reduction): Collapses multiple rows down to a single aggregate value per group. The resulting DataFrame has an index equal to the number of unique groups.
groupby().transform() (Broadcasting / Same-Shape): Evaluates a group-level calculation but returns a Series with the exact same row count and index as the original DataFrame.
Practical Analyst Example: Calculating each transaction's percentage of total regional sales:
python
# transform preserves original row count (no merge needed!)df['regional_total'] = df.groupby('region')['sales'].transform('sum')df['pct_of_region'] = df['sales'] / df['regional_total']
Question 11: How do you pivot and unpivot data using pivot_table() and melt()?
Answer:
pivot_table() (Long to Wide): Reshapes normalized key-value data into wide matrix format with aggregation:
Question 12: How do you compute rolling window and expanding statistics in Pandas?
Answer:
Time-series and financial analytics require smoothing volatility and computing cumulative metrics:
python
# 7-day rolling mean (requires sorting by timestamp first)df = df.sort_values('date')df['rolling_7d_avg'] = df['sales'].rolling(window=7, min_periods=1).mean()# Time-based rolling offset (handles irregular missing dates)df = df.set_index('date')df['rolling_7d_rev'] = df['revenue'].rolling('7D').sum()# Expanding window (cumulative running mean since day 1)df['expanding_mean'] = df['sales'].expanding().mean()
Question 13: How does Pandas handle missing values (NaN, None, <NA>) across data types?
Answer:
Historically, Pandas relied on NumPy's np.nan (a floating-point sentinel value), which forced integer columns with missing values to be cast into float64.
Nullable Data Types (Pandas 1.0+): Introduced capital-letter extension types:
Int64 (nullable integer, uses pd.NA)
boolean (nullable boolean: True, False, <NA>)
string (dedicated StringDtype)
Core Imputation Methods:
python
# Forward-fill previous known value (common in financial ticks)df['price'] = df['price'].ffill()# Interpolate linear values between known coordinatesdf['temp'] = df['temp'].interpolate(method='linear')# Group-specific median imputationdf['salary'] = df.groupby('department')['salary'].transform(lambda g: g.fillna(g.median()))
Question 14: What are the performance and memory differences between query() and boolean masking?
Answer:
Standard boolean masking creates temporary boolean Series in memory:
python
# Standard masking: creates 2 temporary boolean Series in RAMfiltered = df[(df['sales'] > 1000) & (df['region'] == 'East')]
DataFrame.query() utilizes the numexpr C engine under the hood:
python
# numexpr: processes expression chunk-by-chunk in CPU cachefiltered = df.query('sales > 1000 and region == "East"')
For datasets with millions of rows, query() avoids multi-megabyte intermediate array allocations and executes 2x–3x faster while keeping syntax readable.
Question 15: How do you handle MultiIndex (Hierarchical Indexing) in Pandas?
Answer:
MultiIndex enables storing multi-dimensional data in 2D DataFrames:
python
# Creating MultiIndexdf_multi = df.set_index(['department', 'employee_id'])# Slicing with .xs() (Cross-section)engineering_staff = df_multi.xs('Engineering', level='department')# Slicing with IndexSliceidx = pd.IndexSlicesubset = df_multi.loc[idx['Engineering', 101:105], :]# Flattening MultiIndex columns after groupby aggregationsdf_grouped.columns = ['_'.join(col).strip() for col in df_grouped.columns.values]
Question 16: Why is inplace=True strongly discouraged in modern Pandas?
Answer:
Many beginners assume df.drop(columns=['col'], inplace=True) saves memory. In reality:
No Memory Savings: Internally, Pandas almost always creates an entire new copy of the DataFrame under the hood anyway before reassigning the C buffer.
Breaks Method Chaining:inplace=True returns None, making it impossible to write clean, composable data cleaning pipelines:
Deprecation Path: Pandas core maintainers have marked inplace for future deprecation. Modern standard practice is explicit assignment: df = df.drop(...).
Question 17: How do you parse and optimize datetime columns efficiently?
Answer:
Using pd.to_datetime() without specifying the format string forces Pandas to iterate through dozens of regex patterns for every single row, resulting in painful latency:
python
# SLOW (takes 45 seconds for 5 million rows):df['date'] = pd.to_datetime(df['date_str'])# FAST (takes 0.8 seconds with explicit format string):df['date'] = pd.to_datetime(df['date_str'], format='%Y-%m-%d %H:%M:%S')# Extracting temporal components via vectorized .dt accessordf['year'] = df['date'].dt.yeardf['is_weekend'] = df['date'].dt.dayofweek >= 5df['month_period'] = df['date'].dt.to_period('M')
Question 18: How do you perform fast vectorized string operations on text columns?
Answer:
Avoid looping over strings with list comprehensions. Utilize the vectorized .str accessor methods:
python
# Vectorized cleaningdf['cleaned_name'] = df['name'].str.strip().str.title()# Vectorized regex extraction with capture groupsdf[['city', 'state']] = df['location'].str.extract(r'([A-Za-z\s]+),\s*([A-Z]{2})')# Substring containment check without errors on nullsdf['is_corp_email'] = df['email'].str.contains(r'@(company|enterprise)\.com', regex=True, na=False)
Question 19: How do you segment continuous numbers into categorical bins with cut vs qcut?
Answer:
pd.cut() (Equal-Width Bins): Divides the range of the data into equal-interval bins. The number of samples per bin varies depending on data skewness:
python
# Binning customer age into fixed 10-year bracketsdf['age_group'] = pd.cut(df['age'], bins=[0, 18, 35, 50, 100], labels=['Child', 'Young Adult', 'Adult', 'Senior'])
pd.qcut() (Equal-Frequency Bins / Quantiles): Divides data such that each bin contains the exact same number of records (e.g., deciles, quartiles). Essential for RFM customer segmentation and percentile scoring:
python
# Customer spending quartiles (25% in each quartile)df['spend_tier'] = pd.qcut(df['total_spend'], q=4, labels=['Bronze', 'Silver', 'Gold', 'Platinum'])
Question 20: How do you profile execution bottlenecks and leverage the PyArrow backend in Pandas 2.0+?
Answer:
Profiling Execution & Memory:
Run %timeit in Jupyter for line-level timing.
Run df.info(memory_usage='deep') to measure real memory, including variable-length string objects stored outside NumPy blocks.
What are the most common Pandas interview questions?
Common Pandas interview questions test the difference between loc and iloc, groupby aggregations with multiple custom functions, merge vs join vs concat, reshaping data using pivot_table and melt, resolving SettingWithCopyWarning, and vectorization techniques.
What is the difference between loc and iloc in Pandas?
loc is label-based indexing, where you reference rows and columns by their index names/labels (inclusive of end bounds). iloc is purely integer position-based indexing, where you pass 0-based integer positions (exclusive of end bounds, following standard Python slicing).
How do you avoid the SettingWithCopyWarning in Pandas?
The SettingWithCopyWarning happens during chained indexing (e.g., df[df['col'] > 1]['col'] = 5) where Pandas cannot verify whether you are modifying a view or an independent copy. Avoid it by using .loc[mask, 'col'] = 5 or by explicitly creating a copy using df = df.copy().
How does merge differ from concat and join in Pandas?
pd.merge() combines DataFrames horizontally based on shared key columns (similar to SQL JOINs). df.join() combines horizontally based on the DataFrame index. pd.concat() stacks DataFrames vertically (axis=0, like UNION ALL) or binds them horizontally (axis=1).
How do you optimize memory consumption in a large Pandas DataFrame?
Downcast numerical types (e.g., convert float64 to float32, int64 to int16/int8), convert low-cardinality string columns to category data type, load only required columns using usecols in pd.read_csv(), and process huge datasets in chunks using chunksize.
Founder at Topfolio with 6+ years in data & analytics across JPMC, Ultrahuman, and high-growth startups. Sat on hiring panels, reviewed 500+ resumes, and writes practical SQL & data guides.