When Zero Plus Zero Equals Two: Fixing Trace Fragmentation in Istio and Kiali

You install Istio, point Prometheus at the mesh, and get request rate, latency, and error rate across your service graph. You instrument your application with OpenTelemetry and get traces with spans. Then you open your tracing UI and find that the Istio traces and the application traces are separate records for the same request — disconnected, uncorrelated, showing different transaction halves. Zero plus zero equals two instead of one.

The Framework

The problem is context propagation incompatibility. By default, Envoy’s OpenTelemetry tracer ignores incoming W3C traceparent headers and creates a new root span for each request it intercepts. The application’s OTel instrumentation creates its own root span. Each believes it is starting a trace, and both are correct from their own perspective — which produces the fragmentation.

Kiali provides the visualization layer showing service-to-service traffic topology, but it can’t resolve underlying trace fragmentation. The fix requires two configuration changes: switch Envoy’s tracer and enable B3 propagation in the application.

Fix 1 — Switch Envoy to the Zipkin Tracer

The Zipkin tracer (as opposed to the native OpenTelemetry provider) extracts B3 headers instead of W3C traceparent headers. Configure Istio’s meshConfig to use the Zipkin tracer pointed at your collector’s port 9411:

apiVersion: install.istio.io/v1alpha1
kind: IstioOperator
spec:
  meshConfig:
    defaultConfig:
      tracing:
        zipkin:
          address: <otel-collector-host>:9411

This change affects all Envoy sidecars in the mesh — no per-service configuration needed.

Fix 2 — Enable B3 Propagation in Applications

Add B3 multi-format support to the application’s OpenTelemetry propagators alongside the standard W3C headers:

OTEL_PROPAGATORS="tracecontext,baggage,b3multi"

This allows the application’s OTel SDK to both read and write B3 headers, aligning with what the Zipkin-configured Envoy sidecars now emit. The application will propagate the trace context it receives from Envoy rather than starting a new root span.

The Result

After both changes, Envoy and application traces share the same trace ID and parent-child span relationship. Where the tracing UI previously showed 14 or 2 separate span clusters for a transaction, the unified view shows “51 to 75 spans together” — the full end-to-end request path visible in a single trace, including network hops at the mesh layer and code-level spans from the application.

Where It Doesn’t Apply

This fix assumes the OpenTelemetry Collector is receiving trace data from both Envoy (via Zipkin protocol on port 9411) and applications (via OTLP). If the collector configuration routes these to different backends or applies transformations that strip trace ID continuity, the fix addresses the propagation but not the backend correlation.

The Zipkin tracer workaround also means Envoy is emitting B3 rather than W3C traceparent format. If any downstream system expects W3C headers from the mesh layer, that expectation breaks. Most CNCF-ecosystem collectors and backends accept both formats, but verify before deploying across all namespaces. Kubernetes infrastructure observability patterns at the policy and GitOps layer remain separately governed.

Source: CNCF Blog

Content created with AI assistance and reviewed for accuracy.