Docker Compose Profiles, Opt-In Services Done Right

TL;DR — Profiles let services opt-out of the default up. Tag with profiles: ["events"]; bring up only when wanted via --profile events. Best for heavyweight optional services (Kafka, Elasticsearch, monitoring stack). Watch the depends_on interaction.

After polyglot stacks, profiles are the next quality-of-life feature. Without them, your compose file forces every contributor to run Kafka even if they don’t touch event-driven code. With them, the default up is fast and lean.

The basics

services:
  postgres:
    image: postgres:14-alpine

  redis:
    image: redis:7-alpine

  kafka:
    image: bitnami/kafka:3.2
    profiles: ["events"]

  grafana:
    image: grafana/grafana:9
    profiles: ["observability", "full"]

Behaviour:

• docker compose up — postgres + redis only. Kafka and Grafana skipped.
• docker compose --profile events up — postgres + redis + kafka.
• docker compose --profile events --profile observability up — postgres + redis + kafka + grafana.
• docker compose --profile full up — everything tagged full.

A service in multiple profiles is included if ANY of its profiles is enabled.

depends_on interaction — the gotcha

If api depends_on: kafka and kafka has a profile, what happens when you don’t enable the profile?

Compose handles it: if a service depends on a profiled service, enabling the dependent service implicitly enables the dependency’s profile.

kafka:
  profiles: ["events"]
api:
  depends_on: [kafka]

docker compose up api brings up kafka too, regardless of profile.

But: docker compose up (no specific service) brings up kafka only if you enable the profile or if a non-profiled service depends on it. The default-up flow respects profile gating.

When to use profiles

Three real cases:

Optional heavyweight services. Kafka, Elasticsearch, ClickHouse, Tempo. Heavy to run; not needed for the common case. Tag with a profile.

Workflow-specific tools. A migrator that runs once on demand, a seeder that populates demo data, an admin-ui that’s nice-to-have but not required. Tag.

Observability stacks. Grafana, Loki, Tempo, Promtail — full observability is great for debugging but slow to start. Profile as observability.

When NOT to use profiles

For env-specific differences. “Run differently in CI” → use override files or different compose files, not profiles.

For dev vs prod. Different stacks, not different profiles. Production should not run from compose at all.

For per-developer preferences. If 5 engineers each want a different default profile, that’s a sign your compose file is doing too much.

Common profile groupings

Patterns I’ve seen:

# Per-feature
profiles: ["events"]      # message queue + consumers
profiles: ["payments"]    # Stripe simulator + payment workers
profiles: ["search"]      # Elasticsearch + indexer

# Per-purpose
profiles: ["observability"]  # Grafana, Loki, Prometheus
profiles: ["dev-tools"]      # MailHog, Adminer, MinIO
profiles: ["full"]           # everything

# Multi-profile membership
profiles: ["events", "full"]
profiles: ["observability", "full"]

"full" as a meta-profile is useful for the “I want everything” case without listing each individually.

Combining with COMPOSE_PROFILES env var

Instead of --profile flags every time, set in environment:

export COMPOSE_PROFILES=events,observability
docker compose up

Useful for .envrc or per-developer setup. The env var is comma-separated.

Limitations to know

• Profiles only affect what’s included; they don’t change service config. To change config per profile, use override files.
• A service without profiles is always included.
• Profile names are arbitrary strings — no schema validation. Typos silently exclude services.
• docker compose ps shows only services included by current profile selection. If you “lost” a service, check profile.

A real-world example

For my current stack:

name: dev-stack

services:
  postgres: { image: postgres:14-alpine, ... }
  redis: { image: redis:7-alpine, ... }
  api:
    build: ./services/api
    depends_on:
      postgres: { condition: service_healthy }
      redis: { condition: service_healthy }
  bff:
    build: ./services/bff
    depends_on: [api]

  # Optional services
  kafka:
    image: bitnami/kafka:3.2
    profiles: ["events", "full"]
  kafka-ui:
    image: provectuslabs/kafka-ui:v0.4
    profiles: ["events", "full"]
    depends_on: [kafka]

  mailhog:
    image: mailhog/mailhog
    profiles: ["mail", "full"]
    ports: ["8025:8025"]

  grafana:
    image: grafana/grafana:9
    profiles: ["observability", "full"]

  loki:
    image: grafana/loki:2.6
    profiles: ["observability", "full"]

  promtail:
    image: grafana/promtail:2.6
    profiles: ["observability", "full"]
    depends_on: [loki]

volumes:
  postgres-data:
  redis-data:

Daily work: docker compose up brings up the 4 core services in ~10 seconds.

Working on email flows: docker compose --profile mail up adds MailHog.

Debugging perf: docker compose --profile observability up brings up the Grafana stack.

Everything: docker compose --profile full up.

This is the layout I’d recommend for any team running more than 3-4 services locally.

Common Pitfalls

No profiles at all. Every contributor runs every service. Boot times balloon. Memory pressure.

Too many profiles. 12 profiles, each with 1 service. Decision paralysis. Group sensibly.

Forgetting depends_on from a non-profiled service. If api depends on Kafka but Kafka is profile-gated, default up brings up Kafka anyway (transitive dep). Sometimes desired, sometimes not.

Production-specific services in profiles. Production doesn’t run from compose. Don’t model prod-vs-dev as profiles.

Profiles for env vars. Profiles control which services run; they don’t change env. For env-specific config use .env.<profile> files manually.

Wrapping Up

Profiles keep your compose file lean by default while supporting any expansion. Group by feature or purpose; use full as a meta-profile. Friday: env vars and secrets — the other axis of “how do I configure all this.”