Warming up the neural circuits...
By the end of this chapter you will:
Your app doesn't run in isolation. It needs a database, a , a message , maybe a search engine. Compose lets you spin up the entire stack with one command — and tear it down just as easily.
An orchestra has violins, cellos, brass, woodwinds, and percussion. Each section can play alone, but the symphony only happens when they play together, in sync, following the same tempo. The conductor doesn't play any instrument — they coordinate: "violins, you enter here; brass, wait for the cellos to finish; percussion, softer in this movement."
Docker Compose is the conductor. Each service (, database, Redis, worker) is a section of the orchestra. Compose coordinates them: which services start first, how they discover each other on the network, where their data lives, and how to check if they're actually ready to play. One docker compose up and the entire symphony starts.
Docker Compose is a tool for defining and running multi-container Docker applications. You describe your entire stack in a single YAML file (docker-compose.yml) and manage it with a single set of commands.
# Start everything
docker compose up -d
# See what's running
docker compose ps
# View logs from all services
docker compose logs -f
# Stop everything
docker compose down
# Stop AND delete volumes (reset database)
docker compose down -vEvery Compose file has at minimum a services section. Networks and volumes are optional but almost always needed.
version: "3.8"
services:
# Each service becomes a container
api:
build: .
ports:
- "3000:3000"
environment:
- DATABASE_URL=postgres://postgres:password@db:5432/myapp
- REDIS_URL=redis://cache:6379
depends_on:
db:
condition: service_healthy
cache
Let's break down every piece of this file.
Each entry under services defines a container. The key (e.g., api, db, cache) becomes the container name and the DNS hostname on the default network.
services:
api: # Other containers reach this at http://api:3000
db: # Other containers reach this at db:5432
cache: # Other containers reach this at cache:6379Using an existing image vs building:
# Use a pre-built image from Docker Hub
db:
image: postgres:16-alpine
# Build from a Dockerfile in the current directory
api:
build: .
# Build from a specific Dockerfile
worker:
build:
context: .
dockerfile: Dockerfile.workerCompose automatically creates a default network for your app. All services join it and can resolve each other by service name. This is Docker's built-in DNS at work.
For more complex topologies, define custom networks:
networks:
frontend:
backend:
services:
api:
networks:
- frontend
- backend # Can talk to both frontend and backend
db:
networks:
- backend # Only on backend network — isolated from frontendThis is a security pattern: the database is on a private network that only the API can reach. The frontend (Nginx, for example) can't directly talk to the database.
Named volumes persist data across docker compose down (unless you use -v). They're listed at the top level and referenced by services:
volumes:
pgdata: # Declare the volume
redisdata:
services:
db:
volumes:
- pgdata:/var/lib/postgresql/data # Mount itdocker compose down stops and removes containers but keeps volumes. docker compose down -v also removes volumes — your database is gone. Useful for development resets, dangerous in production. If you're using Compose in production (not ideal, but common for small projects), never use -v casually.
A container can be "running" but not "ready." PostgreSQL starts instantly, but it takes 5-10 seconds to accept connections. Without healthchecks, your API tries to connect too early and crashes.
healthcheck:
test: ["CMD-SHELL", "pg_isready -U postgres"]
interval: 10s # Check every 10 seconds
timeout: 5s # Fail if check takes > 5 seconds
retries: 5 # Mark unhealthy after 5 consecutive failures
start_period: 15s # Grace period before checks beginCommon healthcheck commands:
# PostgreSQL
healthcheck:
test: ["CMD-SHELL", "pg_isready -U postgres"]
# Redis
healthcheck:
test: ["CMD", "redis-cli", "ping"]
# Custom API endpoint
healthcheck
depends_on controls startup order, but there's a critical nuance:
# Basic: api starts AFTER db starts (but db might not be READY yet)
depends_on:
- db# Long syntax with condition: api starts only after db is HEALTHY
depends_on:
db:
condition: service_healthyWithout condition: service_healthy, depends_on only waits for the container to START — not for the service inside to be ready. PostgreSQL's container starts instantly, but the database isn't accepting connections for 5-10 seconds. Your API will crash with "connection refused" unless you use healthcheck-based dependencies. This is the #1 Docker Compose mistake.
Three ways to pass config to services:
# 1. Inline (fine for non-secret values)
environment:
- NODE_ENV=production
- LOG_LEVEL=info
# 2. From an env file (good for development)
env_file:
- .env
# 3. Explicit file path
env_file:
- ./config/api.envCompose automatically reads variables from a .env file in the same directory. You can use these variables in the Compose file with ${VARIABLE} syntax: POSTGRES_PASSWORD: ${DB_PASSWORD}. Keep .env in .gitignore — never commit secrets.
Not every service needs to run every time. Profiles let you selectively include services:
services:
api:
# No profile — always starts
db:
# No profile — always starts
adminer: # Database GUI — only in development
image: adminer
profiles:
- dev
- debug
ngrok: # Expose local server to internet — only when testing webhooks
image: ngrok/ngrok
# Start only default services (no profiles)
docker compose up -d
# Start with dev profile (includes adminer)
docker compose --profile dev up -d
# Start with both dev and debug profiles
docker compose --profile dev --profile debug up -ddocker-compose.override.yml is automatically merged with docker-compose.yml (no -f flag needed). Use it for development-specific config:
services:
api:
build:
target: development # Use dev stage from multi-stage Dockerfile
volumes:
- .:/app # Bind-mount source for hot reload
- /app/node_modules # Anonymous volume — don't override node_modules
environment:
- NODE_ENV=development
- DEBUG=express:*
command: npm run devProduction overrides go in a separate file you specify explicitly:
docker compose -f docker-compose.yml -f docker-compose.prod.yml up -d# Start in detached mode
docker compose up -d
# Start and rebuild images
docker compose up -d --build
# Start specific services
docker compose up -d api db
# Scale a service (limited — better for dev/testing)
docker compose up -d --scale worker=3
#
A B2B SaaS company with a 12-service architecture (API, 3 workers, PostgreSQL, Redis, Elasticsearch, Nginx, and a few microservices) had a painful onboarding problem. New developers spent their first 2-3 days setting up the local environment:
They created a single docker-compose.yml in the main repo that defined every service. A docker-compose.override.yml added development conveniences (hot reload, debug ports, local volumes).
New developer onboarding became:
git clone repo
docker compose up -d # Everything starts
docker compose run --rm api npm run db:setup # Migrate + seedFive minutes from zero to a fully working stack. The Compose file became the single source of truth for "what does our architecture look like?" — better than any diagram, because it's executable.
When they added Elasticsearch to production, they added 8 lines to docker-compose.yml. Every developer got Elasticsearch the next time they pulled. No "oh, you need to install Elasticsearch separately, here's a wiki page from 2019 that might still work."
| Mistake | Why it's wrong | Fix |
|---|---|---|
depends_on without condition: service_healthy | API starts before DB is ready to accept connections | Add healthchecks and use condition: service_healthy |
Hardcoding credentials in docker-compose.yml | Committed to git — everyone who clones the repo has your DB password | Use .env file (gitignored) with ${VARIABLE} substitution |
| Not using named volumes for databases | Data is lost on docker compose down | Define volumes at top level and mount them in the service |
| Exposing database ports to host in production | ports: - "5432:5432" opens your DB to the world if the host firewall is misconfigured | Use internal Docker networks only; no ports for databases |
| One giant Compose file for everything | Rebuilding the API triggers unnecessary restarts of Redis, DB, etc. | Split into logical Compose files; or use profiles for optional services |
Using :latest for database images | A minor version bump can corrupt data or change behavior silently | Pin exact versions: postgres:16.3-alpine |
restart: unless-stopped on every service. This ensures containers come back after a reboot or Docker daemon restart. For critical services, restart: always is even stronger.image: postgres:16.3-alpine@sha256:abc123... ensures you're running the exact same bits everywhere.docker compose ps in your health-check scripts. If any service shows unhealthy or restarting, alert immediately.deploy.resources to set CPU/memory limits in Compose v3+. deploy: resources: limits: { cpus: '1.0', memory: 512M }. This prevents one container from starving others.:delegated mount option (./src:/app/src:delegated) tells Docker that host-to-container sync can be delayed — improves I/O performance on macOS/Windows for development volumes./docker-entrypoint-initdb.d/) run on first container start only. If you need them to re-run, delete the volume first: docker compose down -v && docker compose up -d.secrets for sensitive config in Docker Swarm mode. For standard Compose, use .env files with restrictive permissions (chmod 600 .env).user: "1000:1000" to service definitions, or handle it in the Dockerfile.Beginner:
docker-compose.yml with two services: a Node.js API and a PostgreSQL database. Use environment variables for the database connection. Start both with docker compose up -d and verify the API can connect to the database.docker compose exec to connect to Redis (redis-cli) and run PING. Then use docker compose logs to view logs from all three services.Intermediate:
depends_on to use condition: service_healthy. Test that the API waits for the database to be truly ready — artificially delay the DB startup (add a slow init script) and observe.docker-compose.override.yml that adds development conveniences: bind-mount source code for hot reload, enable debug ports, and use a dev profile for an Adminer (database GUI) service. Start with and without the dev profile.Advanced:
docker compose down and verify all services stop cleanly.docker compose run --rm to run database migrations, then starts the stack. If migrations fail, the script should NOT start the services. Add this script as a pre-up or document it as the startup procedure.Beginner:
Q: What is Docker Compose and when would you use it?
A: Docker Compose is a tool for defining and running multi-container Docker applications. You describe services, networks, and volumes in a YAML file and manage them with docker compose commands. Use it for local development, CI testing environments, and small production deployments on a single server.
Q: How do containers in a Compose file discover each other?
A: Compose creates a default network. Each service's name becomes a DNS hostname. So api can reach db at db:5432. Docker's built-in DNS resolves service names to container IPs automatically.
Q: What's the difference between docker compose down and docker compose down -v?
A: down stops and removes containers and the default network. down -v also removes named volumes — all persistent data is deleted. Use -v for development resets; avoid it in production unless you're certain.
Senior:
Q: Explain the problem with depends_on without healthchecks, and how you solve it.
A: depends_on without condition: service_healthy only waits for the dependent container to start — not for the service inside to be ready. A PostgreSQL container starts instantly, but the database takes 5-10 seconds to begin accepting connections. If your API starts during that window, it gets "connection refused" and may crash. The fix is: (1) Add a healthcheck to the database service (e.g., pg_isready), (2) use the long syntax depends_on: db: condition: service_healthy. Compose then waits for the healthcheck to pass before starting the API. Additionally, your API should have retry logic with exponential backoff for database connections — healthchecks reduce the problem but robust apps handle transient failures.
Q: How would you structure Compose files for a project with significantly different dev, staging, and production configurations?
A: Use a base docker-compose.yml with common service definitions. Then create override files: docker-compose.dev.yml (bind mounts, hot reload, debug ports, dev profiles), docker-compose.staging.yml (CI-appropriate config, test secrets, no volumes for databases), docker-compose.prod.yml (pinned image digests, resource limits, restart policies, secrets from external sources). Developers just run (override is auto-applied). CI runs . Production runs . Key principle: the base file changes rarely; environment-specific differences are isolated in override files.
Docker Compose turns "I need to start 5 services to work on this feature" into docker compose up -d. You define your entire stack — API, database, cache, worker, reverse proxy — in a single YAML file. Services communicate over a shared network using their names as hostnames. Named volumes persist data across restarts. Healthchecks ensure services are truly ready before dependents start. Profiles let you include optional services (admin tools, debug proxies) without bloating the default configuration. Override files separate development concerns (hot reload, debug ports) from production concerns (resource limits, pinned versions). Compose won't replace Kubernetes for large-scale orchestration, but for local development, CI environments, and single-server deployments, it's the tool that makes multi-service development practical.
docker-compose.yml defines services, networks, and volumes.depends_on with condition: service_healthy is essential — without it, you get race conditions on startup..env file provides variable substitution: ${DB_PASSWORD} keeps secrets out of the Compose file.--profile dev) include optional services without changing the base file.docker-compose.override.yml) auto-merge; use for dev-specific config.docker compose exec <service> <cmd> runs commands in running containers.docker compose run --rm <service> <cmd> runs one-off commands in fresh containers.docker compose down -v nukes volumes — useful for dev resets, dangerous in production.depends_on: - db and depends_on: db: condition: service_healthy? Answer: The first only waits for the container to start (not the service inside to be ready). The second waits for the healthcheck to pass.docker-compose.yml? Answer: In a .env file (gitignored), referenced in the Compose file with ${VARIABLE} syntax.docker compose down vs docker compose down -v? Answer: down stops and removes containers but preserves named volumes (data survives). down -v also deletes volumes (data is gone).profiles: - dev on that service, then start with docker compose --profile dev up -d.docker-compose.override.yml? Forgetting docker compose down -v resets data |
| In dev, you accidentally nuke your local database |
Use docker compose down (no -v) for routine stops; -v only when you want a fresh start |
docker compose -f docker-compose.yml -f docker-compose.staging.yml up -ddocker compose -f docker-compose.yml -f docker-compose.prod.yml up -dQ: How do you handle database migrations in a Docker Compose setup — before the API starts serving traffic?
A: Several approaches: (1) A separate migration service that runs docker compose run --rm api npm run migrate before docker compose up — the migration runs to completion, then services start. (2) An init container pattern: add a service with command: npm run migrate && echo 'done', have the API depends_on this migration service with condition: service_completed_successfully (Compose v2.1+). (3) Run migrations as part of the API's startup script — the API runs migrations on boot before starting the HTTP listener. Approach 1 is simplest and most common. Approach 2 is elegant but requires careful handling of the migration service exiting. Approach 3 is risky in scaled environments (multiple API instances racing to migrate).
docker-compose.yml and typically contains development-specific overrides like bind mounts, hot reload, and debug ports.docker compose run --rm api npm run migrate — creates a new container, runs the command, removes the container when done.