background-shape
gRPC Interceptors in Go, Auth, Logging, and Recovery
March 16, 2023 · 6 min read · by Muhammad Amal programming

TL;DR — Interceptors are gRPC’s middleware mechanism: a chain of functions that wrap every RPC / There are four kinds — unary/stream × server/client — and they don’t compose with each other / Always chain in this order: recovery, logging, metrics, tracing, auth.

If you’ve used net/http middleware, gRPC interceptors will feel familiar but slightly off. The signatures are different for unary vs streaming, server vs client. The composition story is library-dependent. And the order matters more than people realize — putting recovery after logging means a panicking handler never logs, which makes the next 3 AM page much harder.

I’m going to walk through interceptors as I actually use them in services. The previous post on context and deadlines covered request-scoped values; interceptors are where you populate them.

The Four Signatures

Unary server interceptor:

func UnaryServerInterceptor(
    ctx context.Context,
    req any,
    info *grpc.UnaryServerInfo,
    handler grpc.UnaryHandler,
) (any, error) {
    // pre-processing
    resp, err := handler(ctx, req)
    // post-processing
    return resp, err
}

Stream server interceptor:

func StreamServerInterceptor(
    srv any,
    ss grpc.ServerStream,
    info *grpc.StreamServerInfo,
    handler grpc.StreamHandler,
) error {
    // pre-processing
    err := handler(srv, ss)
    // post-processing
    return err
}

Client-side interceptors have similar shapes but receive a grpc.UnaryInvoker or grpc.Streamer. I won’t dwell on client interceptors; the patterns are symmetric and the use cases (retries, client-side metrics, deadline enrichment) are obvious extensions.

The structural inconvenience: a single interceptor function handles only one shape. If you want logging on both unary and streaming RPCs, you write two interceptors that share helpers. The grpc-ecosystem/go-grpc-middleware project does most of this work for you in v2.

Registration and Chaining

gRPC-Go has built-in chaining now:

srv := grpc.NewServer(
    grpc.ChainUnaryInterceptor(
        recovery.UnaryServerInterceptor(),
        logging.UnaryServerInterceptor(logger),
        metrics.UnaryServerInterceptor(),
        auth.UnaryServerInterceptor(verifier),
    ),
    grpc.ChainStreamInterceptor(
        recovery.StreamServerInterceptor(),
        logging.StreamServerInterceptor(logger),
        metrics.StreamServerInterceptor(),
        auth.StreamServerInterceptor(verifier),
    ),
)

The order is outermost-first. Recovery wraps everything else. Logging wraps the rest. Auth is innermost — by the time auth runs, logging has already started a span, metrics have started a timer, and recovery is ready to catch a panic.

If you put auth first and it rejects a request, that’s fine — fast path, no overhead. But your metrics for “request count” exclude rejected ones, which is misleading. Putting auth last means every request gets counted before being accepted or rejected. Pick the order that matches what you want your dashboards to mean.

Recovery: Panic Insurance

A handler that panics takes down the goroutine. In gRPC-Go without recovery, that means a cryptic INTERNAL error and a goroutine crash that may bring useful state down with it.

func UnaryRecoveryInterceptor(logger *slog.Logger) grpc.UnaryServerInterceptor {
    return func(
        ctx context.Context,
        req any,
        info *grpc.UnaryServerInfo,
        handler grpc.UnaryHandler,
    ) (resp any, err error) {
        defer func() {
            if r := recover(); r != nil {
                stack := debug.Stack()
                logger.ErrorContext(ctx, "handler panic",
                    "method", info.FullMethod,
                    "panic", fmt.Sprintf("%v", r),
                    "stack", string(stack),
                )
                err = status.Error(codes.Internal, "internal error")
            }
        }()
        return handler(ctx, req)
    }
}

Two notes:

  • Don’t return the panic message to the client. That can leak internals. Use a generic codes.Internal and log the detail server-side.
  • Capture the stack. runtime/debug.Stack() gives you the goroutine’s stack at the recovery point, which is what you actually want.

Use named returns so the deferred function can set err. Without named returns, the recover can’t mutate the return value.

Logging: Structured, Correlated, Useful

The Go 1.21 slog package isn’t out yet as of March 2023, but the design is in flight. For now, I use zap or zerolog. The interface is what matters.

func UnaryLoggingInterceptor(logger *zap.Logger) grpc.UnaryServerInterceptor {
    return func(
        ctx context.Context,
        req any,
        info *grpc.UnaryServerInfo,
        handler grpc.UnaryHandler,
    ) (any, error) {
        start := time.Now()
        reqID := getOrCreateRequestID(ctx)
        ctx = context.WithValue(ctx, requestIDKey, reqID)

        l := logger.With(
            zap.String("rpc.method", info.FullMethod),
            zap.String("request_id", reqID),
        )

        resp, err := handler(ctx, req)
        duration := time.Since(start)
        code := status.Code(err)

        if err != nil {
            l.Error("rpc failed",
                zap.Duration("duration", duration),
                zap.String("code", code.String()),
                zap.Error(err),
            )
        } else {
            l.Info("rpc completed",
                zap.Duration("duration", duration),
                zap.String("code", code.String()),
            )
        }
        return resp, err
    }
}

The structured fields are non-negotiable. If your logs are unstructured strings, you can’t filter by method or alert on error rate per RPC.

Request IDs come from incoming metadata or get generated here. Either way, they go into the context for the handler and downstream calls to pick up.

Don’t log the request and response payloads at INFO. PII, credentials, large bodies — all reasons to keep payload logging at DEBUG or off entirely.

Authentication: JWT Validation

Auth interceptors live closest to the handler because they’re the cheapest to skip on the happy path: if everything before them is logged, traced, and metered, then the auth check is just one more step.

func UnaryAuthInterceptor(verifier *jwt.Verifier, skip map[string]bool) grpc.UnaryServerInterceptor {
    return func(
        ctx context.Context,
        req any,
        info *grpc.UnaryServerInfo,
        handler grpc.UnaryHandler,
    ) (any, error) {
        if skip[info.FullMethod] {
            return handler(ctx, req)
        }
        md, ok := metadata.FromIncomingContext(ctx)
        if !ok {
            return nil, status.Error(codes.Unauthenticated, "missing metadata")
        }
        auth := md.Get("authorization")
        if len(auth) == 0 {
            return nil, status.Error(codes.Unauthenticated, "missing authorization")
        }
        token := strings.TrimPrefix(auth[0], "Bearer ")
        claims, err := verifier.Verify(ctx, token)
        if err != nil {
            return nil, status.Errorf(codes.Unauthenticated, "invalid token: %v", err)
        }
        ctx = WithUserID(ctx, claims.Subject)
        ctx = WithRoles(ctx, claims.Roles)
        return handler(ctx, req)
    }
}

The skip map handles health checks and reflection. Don’t try to put if method == "Health" { skip } logic inline — make it data-driven and configurable.

JWT verification should not call out to an authority on every request. Cache the JWKS. Verify locally with the cached keys. Refresh on key rotation. This is where most JWT performance disasters originate.

Stream Interceptors: The Wrapped ServerStream

Stream interceptors are trickier because the stream’s context isn’t directly settable. You wrap the stream:

type wrappedServerStream struct {
    grpc.ServerStream
    ctx context.Context
}

func (w *wrappedServerStream) Context() context.Context {
    return w.ctx
}

func StreamAuthInterceptor(verifier *jwt.Verifier, skip map[string]bool) grpc.StreamServerInterceptor {
    return func(
        srv any,
        ss grpc.ServerStream,
        info *grpc.StreamServerInfo,
        handler grpc.StreamHandler,
    ) error {
        if skip[info.FullMethod] {
            return handler(srv, ss)
        }
        claims, err := authenticate(ss.Context())
        if err != nil {
            return err
        }
        wrapped := &wrappedServerStream{
            ServerStream: ss,
            ctx:          WithUserID(ss.Context(), claims.Subject),
        }
        return handler(srv, wrapped)
    }
}

The pattern is standard enough that go-grpc-middleware ships a WrappedServerStream type. Use it if you bring in the dependency.

Common Pitfalls

The familiar shapes:

  • Interceptor order accidents. Recovery must be outermost. Auth changes the context, so anything that reads the context (logging the user ID, tracing the user) must be inside auth. Draw the chain out before committing.
  • Forgetting stream interceptors entirely. Teams configure unary interceptors and wonder why streaming endpoints have no logs. Both need explicit registration.
  • Modifying the context for unary but not stream. If your unary interceptor adds the user ID to the context but the stream version doesn’t, your streaming handlers can’t see the user. Symmetry matters.
  • Calling expensive operations in interceptors without caching. JWT key fetches, database lookups, downstream gRPC — these all bloat every request’s latency. Cache or move them out.
  • Returning errors that leak internals. codes.Internal with a generic message; the detail lives in logs. Don’t echo database errors to clients.
  • Mutating req in a server interceptor. Type assertions on the request to modify it are brittle. If you need to enrich the request, do it in the handler with first-class types.
  • Not passing the new context into handler. Trivially overlooked. The whole point of mutating the context is to pass the mutated version down.

Wrapping Up

Interceptors are where gRPC services get their non-functional behavior. Get the chain right and observability, security, and resilience are uniform across every endpoint. Get it wrong and you’ll have bespoke handling per handler, which is exhausting to maintain. The next post moves to a related concern: connection pooling and resource lifecycle, particularly for services that fan out to many downstream gRPC servers.