background-shape
Wiring Trivy 0.45 Into a CI Pipeline That Actually Blocks Bad Builds
September 4, 2023 · 6 min read · by Muhammad Amal programming

TL;DR — Pin Trivy to 0.45 and the DB to a hashed offline bundle so CI is deterministic / Gate on --severity HIGH,CRITICAL --ignore-unfixed and keep .trivyignore reviewable in PRs / Cache the vuln DB across runs or you will rate-limit yourself off GHCR by Wednesday.

Most teams I have audited this year run Trivy somewhere. About half of them run it in a way that produces a green checkmark while shipping CVE-2023-something on a base image nobody owns. The scanner is fine. The wiring around it is the problem.

This is the setup I now standardise on for new projects: a single workflow file, deterministic results, severity gating that does not lie, and an ignore policy that survives code review. Trivy 0.45 (released August 31, 2023) is current as I write this and the flags below are stable on that line.

Why a Naive trivy image in CI Is Not Enough

Aqua’s documentation makes it look like a one-liner. It is, until your tenth scan of the day pulls the vulnerability DB from ghcr.io/aquasecurity/trivy-db and you hit the anonymous pull limit. The job goes red, someone reruns it, it goes green, and now your build is non-deterministic. That alone disqualifies the naive setup.

The second problem is severity. A fresh node:20 image right now reports somewhere north of 80 vulnerabilities, the vast majority unfixed or in MEDIUM and LOW. If you fail the build on anything, developers learn to ignore the red X. If you only fail on CRITICAL, you miss high-severity RCE in glibc. The middle path is HIGH,CRITICAL plus --ignore-unfixed, with a tracked exception list.

The third problem is that nobody owns the ignore list. It ends up as a comment in a Jenkinsfile from 2021. .trivyignore solves that if you treat it like code.

A Pipeline That Holds Up

Here is the GitHub Actions job I use. It does four things the default does not: pins the binary, caches the DB, gates correctly, and uploads SARIF for code-scanning view.

name: container-scan
on:
  pull_request:
    paths: ["Dockerfile", "**/Dockerfile", ".github/workflows/container-scan.yml"]
  push:
    branches: [main]

jobs:
  trivy:
    runs-on: ubuntu-22.04
    permissions:
      contents: read
      security-events: write
    steps:
      - uses: actions/checkout@v4

      - name: Build image
        run: docker build -t app:${{ github.sha }} .

      - name: Cache Trivy DB
        uses: actions/cache@v3
        with:
          path: ~/.cache/trivy
          key: trivy-db-${{ runner.os }}-${{ github.run_id }}
          restore-keys: trivy-db-${{ runner.os }}-

      - name: Run Trivy
        uses: aquasecurity/trivy-action@0.12.0
        with:
          image-ref: app:${{ github.sha }}
          format: sarif
          output: trivy.sarif
          severity: HIGH,CRITICAL
          ignore-unfixed: true
          exit-code: 1
          vuln-type: os,library
          trivyignores: .trivyignore

      - name: Upload SARIF
        if: always()
        uses: github/codeql-action/upload-sarif@v2
        with:
          sarif_file: trivy.sarif

The trivy-action@0.12.0 pins to a Trivy 0.45 release. Do not use @master — I have watched a master tag silently bump the binary and break a pipeline on a Friday afternoon.

The Ignore File

.trivyignore is a flat list of CVE IDs. It is reviewable, blameable, and you can grep for staleness. The convention I enforce in PR review is one CVE per line with an inline comment explaining why and an expiry date.

# CVE-2023-29491  ncurses, unreachable in our distroless image, expires 2024-01-15
CVE-2023-29491
# CVE-2023-4863   libwebp, not linked in this binary, expires 2023-12-01
CVE-2023-4863

A quarterly job greps for expired dates and opens an issue. That is the only ignore-list discipline I have seen survive a team change.

Scanning Beyond the Image

The image scan is the headline, but Trivy 0.45 will also scan filesystems, IaC, and SBOMs from the same binary. I run a filesystem scan against the build context before the Docker build to catch node_modules issues that would otherwise be baked in.

trivy fs --severity HIGH,CRITICAL --ignore-unfixed \
  --skip-dirs .git \
  --scanners vuln,secret,config \
  --exit-code 1 .

The secret scanner is the underrated one. It catches AWS keys, GitHub tokens, and Stripe secrets in the repo before they get into the image layer. It misses custom token formats, so it is not a replacement for Gitleaks or a pre-commit hook, but as a backstop it is free and worth the 15 seconds.

For Terraform and Kubernetes manifests, the config scanner runs the same checks as tfsec and kube-bench against your YAML. If you are using Helm, point Trivy at the rendered output of helm template, not the chart source — the values matter.

Common Pitfalls

The base image you do not control. If your FROM line is python:3.11, you inherit Debian’s CVE backlog. Switch to python:3.11-slim or, better, a distroless or Chainguard image. The Trivy report shrinks by an order of magnitude and most of what is left is actually yours.

Trivy DB pull throttling. The default registry is GHCR and the anonymous limit is 60 pulls per hour per IP. On a shared runner you will hit it. Either authenticate with a GitHub token (the action does this automatically when you provide GITHUB_TOKEN), mirror the DB to ECR, or use --skip-db-update after a daily prime job.

--ignore-unfixed hides real risk. It is the right default for a CI gate, because there is nothing the developer can do about an unfixed CVE in a base image at PR time. But you need a separate weekly scan without that flag, feeding into a backlog the platform team owns. Otherwise unfixed CVEs accumulate forever and you find out about them from a customer.

Java fat JARs and the Trivy classifier. Trivy 0.45 improved Java detection but still misses shaded dependencies in some fat JARs. If you ship Spring Boot uber-JARs, cross-check with dependency-check until you have moved to a proper SBOM workflow. I will cover SBOMs in a follow-up.

SBOM-based scanning is faster but not free. You can generate an SBOM at build time with Syft and have Trivy scan that instead of the image. The scan is seconds instead of minutes. The catch is that the SBOM only reflects what the SBOM generator saw — if Syft missed a binary dropped in by a RUN curl ... step, Trivy will too. See the Trivy SBOM docs for the supported formats.

Where the Gate Belongs

I put the blocking scan on pull requests, not on the main-branch push. A failing main build is a fire drill and the wrong place to discover a new CRITICAL in glibc — the change is already merged. PR-time gating means the person who introduced the issue is the one who owns fixing it, while the context is fresh.

For the main branch I run a non-blocking nightly scan against the deployed image (not the just-built one), with a wider severity range and no ignore-unfixed. That output goes to a dashboard, not a build status. The two scans answer different questions: “did this PR make things worse” versus “what is currently exploitable in production”.

Wrapping Up

Trivy is mature enough that the work is no longer in the scanner — it is in the policy around the scanner. Pin the version, cache the DB, gate on HIGH,CRITICAL --ignore-unfixed, and treat the ignore file as code. The next thing to layer on top is signing what you ship and verifying it at admission, which is where Sigstore comes in.