nonobie — Real dev skills, explained like a friend would

What you'll learn

By the end of this chapter you will:

Profile before optimizing with cProfile and py-spy flamegraphs
Micro-benchmark honestly with timeit
Apply the optimization ladder: algorithm → data structure → caching → implementation
Use functools.lru_cache and friends without memory leaks

The why

Performance intuition is unreliable. Teams often optimize the wrong function, the wrong layer, or the wrong bottleneck.

The rule is strict: measure first, optimize second.

Core concepts: Evidence-driven optimization

timeit for micro-benchmarksOpen section

Let's learn

python

import timeit
 
stmt = "sum(i*i for i in range(1000))"
result = timeit.timeit(stmt, number=1000)
print(result)

Keep benchmark setup controlled and compare equivalent logic.

cProfile and py-spyOpen section

Let's learn

bash

python -m cProfile -o out.prof app.py

Then visualize with snakeviz or inspect in pstats.

py-spy is useful for sampling live processes with lower runtime overhead.

Optimization ladderOpen section

Let's learn

Order of impact:

Better algorithm
Better data structure
Caching/memoization
Low-level implementation tweaks

Do not start at level 4.

functools cachingOpen section

Let's learn

python

from functools import lru_cache
 
 
@lru_cache(maxsize=512)
def fib(n: int) -> int:
    if n < 2:
        return n
    return fib(n - 1) + fib(n

When Python-level optimization is not enoughOpen section

Let's learn

Escalation options:

vectorized libraries (NumPy/Pandas)
process-level parallelism
C/Cython/Rust extension paths

Only escalate after profiling validates the bottleneck.

Measurement hygiene

Warm up code paths, benchmark multiple runs, and separate setup cost from measured body. Without this, numbers are noise.

Common mistakes

Mistake	Symptom	Fix
Benchmarking different behavior and comparing numbers	False conclusions	Benchmark equivalent logic only
Optimizing cold paths	No user-visible improvement	Profile end-to-end first
Over-caching mutable/non-deterministic operations	Stale or wrong results	Cache only pure deterministic functions
Ignoring memory while optimizing CPU	New OOM incidents	Track CPU and memory together
Premature low-level micro-optimizations	Complex code with tiny gains	Follow optimization ladder

Practice: Prove your Mastery

beginner Challenge

4 min

timeit Comparison

Benchmark list comprehension vs explicit loop for same transformation.

Input— given data

data

Square 1000 integers

Expected Output— your code should produce this

expected

Measured timings for both approaches

Write your solutionCtrl+Enter to run

Ctrl+Enter

beginner Challenge

5 min

Run cProfile

Profile a script and identify top function by cumulative time.

Input— given data

data

python -m cProfile app.py

Expected Output— your code should produce this

expected

Top hotspot function name

Write your solutionCtrl+Enter to run

Ctrl+Enter

intermediate Challenge

6 min

Apply Optimization Ladder

Refactor nested lookup logic from list scans to dictionary indexing.

Input— given data

data

Repeated lookup in list of records

Expected Output— your code should produce this

expected

Asymptotic improvement and faster run time

Write your solutionCtrl+Enter to run

Ctrl+Enter

intermediate Challenge

6 min

Add Safe Memoization

Apply lru_cache to deterministic normalization helper.

Input— given data

data

Function normalizing country codes

Expected Output— your code should produce this

expected

Repeated calls avoid recomputation

Write your solutionCtrl+Enter to run

Ctrl+Enter

intermediate Challenge

7 min

Optimization Postmortem

Write a short postmortem for a failed optimization attempt that increased complexity with no gain.

Input— given data

data

Before/after benchmark data

Expected Output— your code should produce this

expected

Root cause and better next action

Write your solutionCtrl+Enter to run

Ctrl+Enter

Interview Questions

Beginner:

"Why is profiling more important than intuition?"

Because real bottlenecks are often different from what developers expect; profiling provides evidence.
"When should you use lru_cache?"

For deterministic pure functions with repeated inputs where memory tradeoff is acceptable.

Senior:

"How do you design a performance improvement process for a team?"

Define baseline metrics, profile representative workloads, prioritize high-impact hotspots, and enforce benchmark checks in CI for critical paths.
"How do you avoid performance regressions over time?"

Keep benchmark suites, capture key latency/throughput metrics, and alert on regression thresholds.

Summary

Performance work must start with measurement.
Use timeit for micro cases and profilers for system hotspots.
Follow optimization ladder from algorithm to implementation details.
Cache carefully and only where semantics stay correct.

Quick recall

text

Need speed? -> measure first
Need hotspot? -> cProfile/py-spy
Need biggest gains? -> algorithm/data structure first
Need repeated pure result reuse? -> lru_cache

Self-Check Quiz

Q1 / 3

MCQ

What is the first step in a reliable optimization workflow?

Chapter 10 — Performance, Profiling & Optimization

The why

Core concepts: Evidence-driven optimization

Common mistakes

Practice: Prove your Mastery

timeit Comparison

Run cProfile

Apply Optimization Ladder

Add Safe Memoization

Optimization Postmortem

Interview Questions

Summary

Quick recall

Self-Check Quiz