nonobie — Real dev skills, explained like a friend would

What you'll learn

By the end of this chapter you will:

Load CSV/JSON/Excel into DataFrames with the right dtypes
Select data precisely with loc, iloc, and boolean masks
Inspect data like a pro: head, info, describe, value_counts
Explain the — and when to reset, set, or ignore it

The why

Most product, business, and telemetry data shows up as rows and columns. You can wrangle that with raw Python dictionaries, but it quickly becomes repetitive and error-prone.

Pandas gives a table-native for loading, selecting, aggregating, and inspecting data with less boilerplate.

Mental model: a DataFrame is a typed table with labeled axes. Good pandas work is mostly about managing schema intentionally.

Core concepts: DataFrame fluency

Series and DataFrame are labeled containersOpen section

Let's learn

python

import pandas as pd
 
df = pd.DataFrame(
    {
        "user_id": [101, 102, 103],
        "city": ["Pune", "Delhi", "Pune"],
        "spend": [1200.0, 980.0, 1575.0],
    }
)
print(df.dtypes.to_dict())
# {'user_id': dtype('int64'), 'city': dtype('O'), 'spend': dtype('float64')}

Schema awareness starts with dtypes and column naming consistency.

Load files with explicit dtype controlsOpen section

Let's learn

python

import pandas as pd
 
orders = pd.read_csv(
    "orders.csv",
    dtype={"order_id": "string", "customer_id": "string"},
    parse_dates=["ordered_at"],

loc and iloc solve different selection problemsOpen section

Let's learn

python

import pandas as pd
 
df = pd.DataFrame(
    {"city": ["Pune", "Delhi", "Pune"], "spend": [1200, 980, 1575]},
    index=["

First 60 seconds inspection workflowOpen section

Let's learn

python

print(df.head(2))
print(df.info())
print(df.describe(include="all"))

head, info, and describe quickly expose missing values, cardinality patterns, and suspicious dtypes.

Index design is a modeling choiceOpen section

Let's learn

python

import pandas as pd
 
events = pd.DataFrame(
    {
        "event_id": [1, 2, 3],
        "user_id": [101, 101, 102],
        "event": ["login", "

assign and pipe keep transformations readableOpen section

Let's learn

python

import pandas as pd
 
df = pd.DataFrame({"city": ["Pune", "Delhi"], "spend": [1200, 980]})
 
out = df.assign(spend_bucket=lambda x

Schema-first habit

Treat pandas operations as schema transformations. Log what columns and dtypes changed after each major step.

Quick start checklist for any dataset

Step	Command	Why
Preview rows	`df.head()`	Detect obvious parsing failures quickly
Inspect dtypes	`df.info()`	Catch object-columns pretending to be numeric/date
Distribution scan	`df.describe(include="all")`	Spot skew, nulls, unusual cardinality
Validate key uniqueness	`df[key].is_unique`	Prevent bad joins and duplicates
Save cleaned snapshot	`to_parquet` / `to_csv`	Preserve reproducible pipeline checkpoints

Common mistakes

Mistake	Why it hurts	Better move
`KeyError: 'revenue'`	Column name mismatch, often from whitespace/casing	Normalize headers early (`str.strip`, lower-case policy)
`AttributeError: 'DataFrame' object has no attribute 'totl'`	Typo via dot-access on column names	Prefer bracket access: `df["total"]`
`SettingWithCopyWarning: A value is trying to be set on a copy...`	Chained indexing creates ambiguous mutation target	Use `.loc[row_mask, "col"] = ...`
`TypeError: Invalid comparison between dtype=datetime64[ns] and str`	Date columns compared to raw strings	Parse dates and compare with `pd.Timestamp`
`ValueError: cannot reindex on an axis with duplicate labels`	Duplicate index values during alignment	Validate uniqueness or reset index before reindex

Production notes

Keep a declared schema for important pipelines (expected columns and dtypes).
Separate read, clean, and feature-generation stages for debuggability.
Persist intermediate outputs for long jobs so recovery is cheap.
Use column-level assertions in tests for critical data products.

Practice: Prove your Mastery

beginner Challenge

5 min

Load with schema hints

Read a CSV using explicit dtype mapping and one parsed datetime column.

Input— given data

data

orders.csv with order_id, customer_id, ordered_at

Expected Output— your code should produce this

expected

String ids and datetime64 ordered_at

Write your solutionCtrl+Enter to run

Ctrl+Enter

beginner Challenge

6 min

loc vs iloc drill

Retrieve one row by label and one value by integer position from the same DataFrame.

Input— given data

data

DataFrame with custom index

Expected Output— your code should produce this

expected

Correct label and positional lookups

Write your solutionCtrl+Enter to run

Ctrl+Enter

intermediate Challenge

7 min

Inspection starter block

Create a reusable function that prints head, info summary, and null counts.

Input— given data

data

Any DataFrame

Expected Output— your code should produce this

expected

Compact diagnostic output

Write your solutionCtrl+Enter to run

Ctrl+Enter

intermediate Challenge

8 min

Index lifecycle

Set a business key as index, run one lookup, then reset index for export.

Input— given data

data

DataFrame with id column

Expected Output— your code should produce this

expected

Index-based lookup and plain-column export frame

Write your solutionCtrl+Enter to run

Ctrl+Enter

intermediate Challenge

10 min

Feature column pipeline

Create two derived columns with assign and return only selected columns in final output.

Input— given data

data

spend + order_count data

Expected Output— your code should produce this

expected

DataFrame with engineered features and clean column order

Write your solutionCtrl+Enter to run

Ctrl+Enter

Interview Questions

Beginner:

"When should I use loc instead of iloc?"

Use loc when you want -based selection and iloc when you want position-based selection.
"Why define dtypes during read_csv?"

It prevents incorrect type inference that can silently break downstream calculations and joins.

Senior:

"How do you make pandas pipelines maintainable in production?"

Enforce schema checks, separate stages, persist checkpoints, and test critical transformations with deterministic fixtures.
"How do you decide index strategy for analytics vs operational exports?"

Choose index for frequent internal access patterns, then normalize to plain columns at integration boundaries.

Summary

Pandas DataFrames are typed, labeled tables built for real-world data workflows.
Reliable work starts with explicit schema control at read time.
loc and iloc must be used intentionally to avoid selection bugs.
A small inspection checklist catches most quality issues before analysis drifts.

Quick recall

text

Need controlled ingest -> read_csv with dtype + parse_dates
Need label lookup -> loc
Need positional lookup -> iloc
Need first-pass quality check -> head + info + describe
Need readable transforms -> assign + explicit final column set

Self-Check Quiz

Q1 / 3

MCQ

What is the practical difference between `loc` and `iloc`?

Chapter 3 — pandas DataFrames

The why

Core concepts: DataFrame fluency

Quick start checklist for any dataset

Common mistakes

Production notes

Practice: Prove your Mastery

Load with schema hints

loc vs iloc drill

Inspection starter block

Index lifecycle

Feature column pipeline

Interview Questions

Summary

Quick recall

Self-Check Quiz