July Retro, Compose in Production-Adjacent Workflows
TL;DR — Migrated three repos to Compose v2.6 patterns. Compose Watch is the dev-loop upgrade of the year. Profiles and healthchecks earn their keep daily. The Mac performance story is real but solvable. Compose stays for daily dev; local k8s when needed.
End of July. End of the Compose theme. Quick retro on what landed and what I’d warn future-me about.
What shipped
Stack-wide Compose Watch adoption. Three internal services moved from “bind-mount + nodemon/air” to Compose Watch. Dev rebuild times dropped 3-4× on Mac. Engineers stopped complaining about Docker being slow.
Profile cleanup across our stacks. All optional services (Kafka, Elasticsearch, monitoring stack) now profile-gated. Default up is 5 services not 11. Cold-start time dropped from 35s to 12s.
Healthchecks + condition: service_healthy everywhere. Eliminated the “first run fails because Postgres wasn’t ready” surprise. Boot is now deterministic.
CI compose stacks isolated per run. COMPOSE_PROJECT_NAME=ci-${RUN_ID} pattern rolled out to all integration test jobs. Parallel runs on the same self-hosted runner stopped colliding.
output: 'standalone'-equivalent images. Migrated services to multi-stage Dockerfiles with dev and prod targets. Same image used in Compose (dev target) and Kubernetes (prod target). Single source of truth.
What worked
Compose Watch alone justified the month. The combination of fast cold-build (BuildKit cache mounts) + Compose Watch sync + dev-server hot reload is the best dev loop I’ve used in containerized Node/Go work. M1 Pro feels native again.
Profiles structurally cleaned up our compose file. Pre-July: 200-line compose file with everything. Post: 80-line core + profiles. New engineers grok the core; opt into profiles as they learn what’s needed.
docker compose ls made forgotten stacks visible. Found three stale compose stacks across my machine that I’d forgotten about. Cleaned up, freed ~6 GB of volumes.
Resource limits caught a leak. One Node service had a slow memory leak in dev that nobody noticed because the container was unlimited and the Mac just got slower. After setting memory: 512M, the OOM-kill surfaced the bug within a day.
What didn’t
secrets: top-level adoption. Wrote about it; haven’t actually adopted it. The friction of “another file to manage” vs .env.local made the team reject it for now. Will revisit if security pushes for it.
Compose Watch on Linux. Honestly, Linux bind mounts are already fast. The Watch payoff is marginal there. Mac is where it shines.
External networks across stacks. Tried the “shared Postgres stack” pattern. Worked technically. Conceptually confusing — “which compose file owns Postgres?” Reverted to per-stack Postgres.
network_mode: host on Mac for a performance test. Doesn’t fully work on Mac (Docker Desktop VM). Wasted half a day before remembering.
What I’d cut
reservations: in dev compose. Useful in Swarm/k8s. Not in single-machine compose. Removed.
Strict CPU limits in dev. Memory limits are essential. CPU limits create more confusion than value in dev. Removed unless specifically needed.
Per-developer compose overrides. docker-compose.override.yml was meant to be per-developer customization. In practice nobody used it consistently; engineers ran the same stack. Removed; document customization via env vars instead.
What’s load-bearing now
- Compose for daily dev across all repos
- CI stacks via project-name isolation
- Healthchecks on every service that has a meaningful “ready” state
- Profiles for anything optional
I’d be sad if any of these reverted. They’re the patterns I’d defend in PR review.
Half-year + 1 wrap
Seven months done. ~91 posts. Rhythm sustains. Some August reader signals:
- A reader asked about July’s Compose Watch post; said it was the missing piece for their Mac dev setup
- The “Compose vs Kubernetes for local dev” post got more shares than I expected — apparently it’s a question many teams revisit yearly
- Search traffic from “docker compose v2 patterns” started showing up
What August looks like
August theme: Industrial IoT (IIoT): Sensor data collection and managing MQTT brokers. Big domain pivot. We’ve been on web-backend topics for most of the year; August goes to the edge-compute / sensor / OT side. Same shape — 13 articles, M/W/F.
Why pivot? Three reasons:
- We’ve started consulting on an IoT project; I’m relearning MQTT and edge patterns
- IIoT is one of those domains web-backend developers find weird; making it accessible is useful
- The patterns translate (event-driven systems, time-series data) and inform back-end thinking too
Versions will be accurate to August 2022: Mosquitto 2.0.15, EMQX 5.0 (just released), TimescaleDB 2.7, OPC UA stacks current to mid-2022. See you Monday.
