Warming up the neural circuits...
By the end of this chapter you will:
Most async outages are not caused by missing await. They are caused by design mistakes:
This chapter focuses on the patterns that keep async systems stable under real load.
import asyncio
async def worker(name: str, delay: float) -> str:
await asyncio.sleep(delay)
return f"done-{name}"
async def main():
async with asyncio.TaskGroup() as tg:
t1 = tg.create_task(worker("a", 0.2))
t2 = tg.create_task(worker("b", 0.3))
print(t1.result(), t2.result())
asyncio.run(main())TaskGroup gives clear lifecycle boundaries and fail-fast semantics.
import asyncio
async def consume():
try:
while True:
await asyncio.sleep(0.2)
except asyncio.CancelledError:
# cleanup
print("cleanup complete")
raiseCatch CancelledError only to clean up, then re-raise so cancellation semantics stay correct.
import asyncio
async def producer(q: asyncio.Queue[int]):
for i in range(1000):
await q.put(i)
async def consumer(q: asyncio.Queue[
import asyncio
async def with_retry(op, attempts: int = 3, base_delay: float = 0.2):
last = None
for i in range(attempts):
try:
return await asyncio.
Typical isolation moves:
This limits blast radius under partial failure.
Define cancellation, timeout, and retry policy as first-class architecture decisions, not optional helper code.
| Problem | Recommended pattern |
|---|---|
| Related tasks with shared fate | asyncio.TaskGroup |
| Producer overwhelms consumer | Bounded asyncio.Queue |
| Transient network failures | Timeout + bounded retry + backoff |
| Cross-workload interference | Bulkhead isolation |
| Mistake | Why it hurts | Better move |
|---|---|---|
Unbounded create_task loops | Memory and scheduling collapse | Use semaphores or queue worker pools |
Swallowing CancelledError | Tasks never stop correctly | Cleanup then re-raise |
| Retrying all exceptions blindly | Masks permanent failures | Retry only transient classes |
| No queue max size | Backpressure absent, memory spikes | Set queue bounds intentionally |
| Shared semaphore for unrelated systems | Cross-system starvation | Isolate by dependency |
Three async workers with small delaysAll results returned after group completionLong-running consume loopCleanup runs and task cancels properlyQueue(maxsize=50).Fast producer and slower consumerProducer blocks naturally when queue is fullFlaky async operationRetries transient failures, raises final error if exhaustedOne slow dependency and one fast dependencyIndependent semaphores and queues proposedBeginner:
"What is backpressure in async systems?"
Backpressure is controlled slowing of producers when consumers cannot keep up, usually via bounded queues.
"Why re-raise CancelledError after cleanup?"
To preserve cancellation semantics so parent tasks and orchestrators know cancellation succeeded.
Senior:
"How do you prevent cascading async failures across dependencies?"
Use isolation boundaries: separate semaphores, queues, retry policies, and timeouts by dependency.
"What is a safe retry policy for external APIs?"
Explicit transient failure list, bounded attempts, exponential backoff, timeout caps, and failure observability.
Related tasks? -> TaskGroup
Consumer lag? -> bounded queue
Transient fault? -> timeout + bounded retry
Cross-dependency risk? -> bulkhead isolationWhat problem does a bounded asyncio.Queue primarily solve?
Use Queue(maxsize=n) to force producers to wait and prevent unbounded memory growth.
Retry only known transient failure classes when possible.