PandasbeginnerPandas
A practical cheat sheet for loading, cleaning, transforming, analyzing, and exporting tabular data with Pandas, plus how it fits with PyTorch workflows.
Foundations and Mental Model
2 topics
What Pandas is (and how it differs from PyTorch)
Pandas is a data handling library for structured/tabular data. Use it to prepare datasets (CSV, Parquet, SQL tables), engineer features, and summarize data. PyTorch is a machine learning framework used after data prep to train neural networks on tensors.
import pandas as pd
import torch
# Pandas: table-like data
df = pd.DataFrame({"age": [25, 31], "income": [50000, 72000]})
print(df)
# PyTorch: tensor math for learning
x = torch.tensor([[25.0, 50000.0], [31.0, 72000.0]])
print(x.shape)Think: Pandas = data tables, PyTorch = model training.
Most ML pipelines start with Pandas, then convert to NumPy/PyTorch tensors.
Core objects: Series and DataFrame
Series is a single labeled column, DataFrame is a labeled 2D table. Both support vectorized operations for fast transformations.
import pandas as pd
s = pd.Series([10, 20, 30], name="score")
df = pd.DataFrame({
"name": ["Ana", "Ben", "Cara"],
"score": [88, 92, 79]
})
print(s.mean())
print(df["score"].max())Use DataFrame for almost all real-world tasks; Series is useful for column-level work.
Column operations are vectorized—prefer them over Python loops.
Data Loading and Inspection
2 topics
Read common formats (CSV, Parquet, Excel, SQL)
Pandas supports multiple file/database formats. Choose Parquet for analytics pipelines and CSV for interoperability.
import pandas as pd
# CSV
df_csv = pd.read_csv("customers.csv")
# Parquet (fast + compressed)
df_parquet = pd.read_parquet("events.parquet")
# Excel
df_xlsx = pd.read_excel("report.xlsx", sheet_name="Sheet1")
# SQL
# from sqlalchemy import create_engine
# engine = create_engine("postgresql+psycopg2://user:pass@host/db")
# df_sql = pd.read_sql("SELECT * FROM orders", engine)Pass dtype= and parse_dates= in read_csv for consistent schemas.
Use chunksize= for very large files.
Quick dataset profiling
Immediately inspect shape, types, nulls, and distribution to catch data issues before transformations.
import pandas as pd
df = pd.read_csv("customers.csv")
print(df.shape)
print(df.head(3))
print(df.info())
print(df.isna().sum())
print(df.describe(include="all"))Run info() and isna().sum() on every new dataset.
Use include='all' in describe() for categorical overview.
Cleaning and Transformation
3 topics
Handle missing values and duplicates
Clean nulls and duplicate rows/keys before analysis or ML. Strategy depends on business logic: drop, fill, or impute.
import pandas as pd
df = pd.DataFrame({
"city": ["NY", None, "SF", "SF"],
"sales": [100, 200, None, None]
})
# Fill nulls
_df = df.copy()
_df["city"] = _df["city"].fillna("Unknown")
_df["sales"] = _df["sales"].fillna(_df["sales"].median())
# Drop duplicates
_df = _df.drop_duplicates()
print(_df)Use median for skewed numeric columns; mean for symmetric distributions.
If duplicates are expected, dedupe using a subset of key columns.
Filter, select, and create columns
Use boolean masks and assign() to transform data safely and readably.
import pandas as pd
df = pd.DataFrame({
"product": ["A", "B", "C"],
"price": [10, 25, 40],
"qty": [2, 1, 3]
})
high_value = df[df["price"] > 20]
result = df.assign(revenue=df["price"] * df["qty"])
print(high_value)
print(result)Prefer .loc[row_filter, col_list] for explicit indexing.
Use assign() in pipelines for cleaner chained transformations.
GroupBy, aggregation, and pivoting
Summarize business metrics by dimensions (region, category, date) using groupby and pivot tables.
import pandas as pd
df = pd.DataFrame({
"region": ["US", "US", "EU", "EU"],
"category": ["A", "B", "A", "B"],
"sales": [100, 120, 90, 110]
})
agg = df.groupby("region", as_index=False).agg(total_sales=("sales", "sum"))
pivot = df.pivot_table(index="region", columns="category", values="sales", aggfunc="sum")
print(agg)
print(pivot)Use named aggregations for clear output column names.
Set as_index=False when you want SQL-like grouped output.
Joins, Time Series, and Performance
2 topics
Merge and join tables safely
Combine datasets with merge() using one-to-one, one-to-many, or many-to-many relationships; validate joins to avoid silent data blowups.
import pandas as pd
customers = pd.DataFrame({"customer_id": [1, 2], "name": ["Ana", "Ben"]})
orders = pd.DataFrame({"order_id": [101, 102, 103], "customer_id": [1, 1, 2], "amount": [50, 75, 20]})
merged = customers.merge(orders, on="customer_id", how="left", validate="one_to_many")
print(merged)Always check row counts before/after merge.
Use validate= to enforce expected relationship cardinality.
Time series basics
Convert to datetime, index by time, resample, and compute rolling metrics for trends and forecasting features.
import pandas as pd
df = pd.DataFrame({
"date": ["2026-01-01", "2026-01-02", "2026-01-03"],
"sales": [100, 120, 90]
})
df["date"] = pd.to_datetime(df["date"])
df = df.set_index("date")
daily = df.resample("D").sum()
daily["rolling_avg_2d"] = daily["sales"].rolling(2).mean()
print(daily)Use UTC for storage and convert to local timezone only for display.
Sort datetime index before rolling/resampling.
Pandas to PyTorch Pipeline
2 topics
Prepare features in Pandas, then convert to tensors
A common workflow: clean and encode data in Pandas, split features/labels, then convert to torch tensors for training.
import pandas as pd
import torch
# Example tabular dataset
df = pd.DataFrame({
"age": [25, 31, 22, 40],
"income": [50000, 72000, 42000, 99000],
"bought": [0, 1, 0, 1]
})
X = df[["age", "income"]].to_numpy(dtype="float32")
y = df["bought"].to_numpy(dtype="float32")
X_t = torch.tensor(X)
y_t = torch.tensor(y).unsqueeze(1)
print(X_t.shape, y_t.shape)Use float32 for model inputs unless you need higher precision.
Normalize/standardize numeric columns before training.
Tiny PyTorch network after Pandas preprocessing
After Pandas feature prep, define a neural network with torch.nn and train using an optimizer like Adam.
import torch
import torch.nn as nn
import torch.optim as optim
model = nn.Sequential(
nn.Linear(2, 4),
nn.ReLU(),
nn.Linear(4, 1),
nn.Sigmoid()
)
criterion = nn.BCELoss()
optimizer = optim.Adam(model.parameters(), lr=1e-3)
# Assume X_t, y_t already prepared from Pandas
# for _ in range(200):
# preds = model(X_t)
# loss = criterion(preds, y_t)
# optimizer.zero_grad()
# loss.backward() # autograd computes gradients
# optimizer.step()Autograd handles gradient computation after loss.backward().
Start simple; add complexity only when baseline underperforms.
Related Cheat Sheets
More hands-on references connected to this topic.
Pandasshared topics
PythonPython syntax, data structures, comprehensions, OOP, error handling, and standard library essentials.
shared topics, same difficulty
FastAPIFastAPI routes, request models, dependency injection, authentication, and async patterns.
shared topics
PyTorchEssential PyTorch patterns for tensor ops, autograd, model building, training loops, and deployment.
shared topics
Related Articles
Background reading and deeper explanations for this sheet.
Intro to FastAPI: Build a Production-Ready Python API with Real Examples
FastAPI is one of the fastest ways to build modern APIs in Python without sacrificing code quality or developer experience. In this practical, beginner-friendly guide, you’ll learn how to create endpoints, validate data, handle errors, connect a database, secure routes, and prepare your FastAPI app for real-world deployment.
keyword overlap
Python GIL and Async Programming: A Practical Cheat Sheet for Real Apps
Confused about when Python threads help and when they hurt? This beginner-friendly, practical guide explains the GIL and async programming with real examples, so you can choose the right concurrency model for APIs, data pipelines, and background jobs.
keyword overlap
Python Stack vs Heap Memory: A Simple, Practical Guide for Developers
Confused about where Python stores function calls, variables, and real objects? This guide breaks down stack vs heap memory in plain English, with clear examples showing local references on the stack and actual objects on the heap. You’ll learn how this impacts performance, debugging, and writing cleaner Python code.
keyword overlap
System Design Cheat Sheet: Practical Guide for Scalable Architecture
This system design cheat sheet gives you a practical, beginner-friendly framework to design scalable, reliable applications without getting lost in theory. You’ll learn what to evaluate, why each component matters, and how to make trade-offs with real-world examples you can use in interviews and production.
keyword overlap