Warming up the neural circuits...
By the end of this chapter you will:
Not all useful work should block an HTTP response.
If sending an email, indexing a document, or emitting audit logs keeps users waiting, perceived performance drops quickly.
At the same time, realtime features like live notifications or dashboards cannot be modeled as one-shot request-response flows.
This chapter covers two complementary patterns:
from fastapi import BackgroundTasks, FastAPI
app = FastAPI()
def send_welcome_email(email: str) -> None:
# placeholder integration with email provider
print(f"sent welcome email to {email}")
@app.post("/users")
def create_user(email: str, background_tasks: BackgroundTasks):
# create user in database first
background_tasks.add_task(send_welcome_email, email)
return {"status": "created"}BackgroundTasks are simple and require no external broker.
BackgroundTasks limitations:
Queue systems (Celery/RQ with Redis or RabbitMQ) add:
from fastapi import FastAPI, WebSocket, WebSocketDisconnect
app = FastAPI()
@app.websocket("/ws")
async def websocket_endpoint(websocket: WebSocket):
await websocket.accept()
try:
while True:
message
WebSocket connections are long-lived and require explicit lifecycle management.
from fastapi import WebSocket
class ConnectionManager:
def __init__(self) -> None:
self.active: set[WebSocket] = set()
async def connect(self, websocket: WebSocket) ->
Track disconnected sockets and clean them up to avoid leaks.
Operational safeguards:
Realtime systems fail gradually without backpressure controls.
If an event must never be lost, route it through a durable queue first and use WebSockets as a delivery channel, not as the source of truth.
| Requirement | Better default |
|---|---|
| Fast fire-and-forget task, low criticality | BackgroundTasks |
| Needs retries, durability, scheduling | Queue workers (Celery/RQ) |
| Client must receive updates instantly | WebSockets |
| Works fine with periodic fetch | HTTP polling |
| Large fan-out events with reliability requirements | Queue + broadcast layer |
| Mistake | Why it hurts | Better move |
|---|---|---|
| Using BackgroundTasks for critical payment workflow | Lost tasks on process failure | Durable queue with retries |
| No connection cleanup | Memory and descriptor leaks | Explicit disconnect handling |
| Unbounded message fan-out | stalls and latency spikes | Rate limits and backpressure policies |
| Blocking I/O inside websocket loop | Starved realtime connections | Use I/O and offload blocking work |
| Treating WebSocket as durable event log | Missed events on reconnect | Persist state/events in DB or queue |
POST /usersImmediate response plus queued in-process taskText messages over /wsServer echoes each messageMultiple connected clientsAll active clients receive broadcast messageTask failure/retry and scale requirementsClear migration recommendationHigh-frequency event streamDrop/coalesce strategy with monitoring metricsBeginner:
"When are FastAPI BackgroundTasks enough?"
For lightweight, non-critical post-response work where occasional loss is acceptable.
"What is the key difference between HTTP and WebSocket communication?"
HTTP is request-response per call; WebSocket is persistent bidirectional connection.
Senior:
"How do you design reliable event delivery with WebSockets?"
Persist event state in durable storage/queue, use WebSocket for transport, and support replay/resync on reconnect.
"What signals indicate it is time to migrate to a queue-based worker system?"
Increasing task failures on restarts, need for retries/scheduling, growing queue lag, and need for horizontal worker scaling.
BackgroundTasks -> simple in-process defer
Queue workers -> durability + retries + scale
WebSocket -> realtime bidirectional channel
Reliability needs persistence, replay, and monitoringWhen are FastAPI BackgroundTasks usually sufficient?