Distributed TracingSupported

Supported. createRouter and createApp accept an Observability configuration. The router creates request, route, validation and handler spans; tracedDriver creates database spans; and HTTP and message carriers propagate W3C traceparent plus optional tracestate.

The span hierarchy, every attribute name, and propagation in both directions are frozen in packages/web/src/observability/SPEC.md against semantic conventions v1.30.0. Its #647 ownership amendment assigns the generic ports, propagation and database instrumentation to @zmdb/app/observability; HTTP spans remain web-owned. zmdb ships ports and a separately installed OpenTelemetry adapter, not an SDK, exporter, collector configuration or global auto-instrumentation.

Install the adapter's package and its sole external peer with yarn add @zmdb/app@1.0.0-beta.2 @opentelemetry/api@^1.9.1, then import @zmdb/app/otel. Neither is part of the @zmdb/core default install. The application selects and owns any SDK, provider, processor, exporter, sampler, collector connection, global registration, flush, and shutdown.

Configure the framework#

The app observability entry point declares narrow Tracer, Span and Meter ports and has no third-party runtime dependency. The separately installed @zmdb/app/otel package is the only current surface that imports @opentelemetry/api, its sole required peer:

import { metrics, trace } from '@opentelemetry/api';
import { fromOpenTelemetry } from '@zmdb/app/otel';
import { createApp } from '@zmdb/web';

const observability = fromOpenTelemetry({
  tracer: trace.getTracer('checkout'),
  meter: metrics.getMeter('checkout'),
});

await using app = createApp(AppModule, { observability });

If you construct the router directly, pass the same object to createRouter(observability). The app forwards it to the HTTP router and every extension context; transportExtension supplies that same object to its message dispatcher.

OpenTelemetry's Node auto-instrumentation remains an alternative for patching node:http, database clients and fetch. Enabling its HTTP or database instrumentation alongside zmdb's corresponding spans can produce two spans for one operation, so choose deliberately.

The framework span tree#

For a matched route with validation and one query:

POST /posts
├── zmdb.route
├── zmdb.validate
└── zmdb.handler
    └── INSERT posts

http.route is the low-cardinality name. Without it, a trace backend shows one operation per id and aggregate latency is meaningless. http.response.status_code, not http.status_code — the v1.23.0 HTTP stabilisation renamed it.

http.route is not derivable from anything a handler or an adapter sees. Ctx carries params, body, query, headers, method, path and an optional span; path is the concrete /posts/1. Only the matched route knows /posts/:id, so the router creates the server span. A request that matches nothing has no http.route; its span is named only for the method, with the raw path kept as an attribute.

There is deliberately no interceptor span. The router runs its effective guards, validation and handler, but runChain remains an explicit handler-level call.

Server-span attributes#

These are the complete server-span attributes emitted today under the pinned OpenTelemetry semantic conventions v1.30.0:

A normal 4xx records the response status but is not marked as a server-span error. An unmatched request has no http.route; its concrete path remains in url.path.

Query spans with useful attributes#

tracedDriver instruments the execute boundary. Parenting is explicit: pass the handler's ctx.span when the query should appear beneath that handler.

import { tracedDriver } from '@zmdb/app/observability';

async function list(ctx: Ctx) {
  const driver = tracedDriver(baseDriver, observability, ctx.span);
  const users = defineRepository(UserSchema, driver);
  return users.findAll();
}

There is no ambient current span and the OpenTelemetry adapter does not consult ambient context. Omitting the third argument therefore creates root database spans; passing ctx.span is what establishes the handler → query edge.

The database-span table is likewise the complete emitted set:

Also avoid putting request bodies, headers or full URLs on spans, for the same reason.

db.statement, db.system, db.operation and db.sql.table are the pre-v1.30.0 spellings. They are why the frozen spec pins a convention version and treats a rename as an edit to the file: nothing fails to compile when an attribute is renamed, the dashboard just goes flat.

zmdb.db.parameter_count is namespaced outside db. on purpose. Recent conventions use db.operation.parameter.<key> for parameter _values_, which is exactly what the warning above forbids, and a neighbouring key would invite the confusion.

The whole compile-time half of that set — system, operation and table — is attached to the compiled query rather than re-derived, which matters more than it sounds; see the note on statement parsing under Observability.

Propagation#

HTTP headers and message envelopes are carriers for traceparent and optional tracestate. The router extracts valid inbound context before creating the server span. A malformed traceparent is ignored and starts a new trace; it never fails the request. Invalid tracestate is dropped while a valid traceparent is retained.

zmdb does not patch fetch. Use your SDK's propagation API, or write the framework span into an outbound carrier:

import { toTraceHeaders } from '@zmdb/app/observability';

const headers = ctx.span === undefined ? {} : toTraceHeaders(ctx.span);
await fetch(url, { headers });

The validation is exact and the frozen spec spells it out, including the case an implementation is most likely to get wrong: a version above 00 is accepted by reading the first four fields and ignoring the rest, because that is the forward-compatibility rule W3C requires. Rejecting it is how a service stops accepting traces the day the spec gains a field.

The message client and event publisher accept an explicit span and put its carrier on TransportRequest / the emitted envelope. A custom TransportStrategy must preserve both carrier fields. A request/reply consumer is a child of the supplied span; a queued consumer is linked to it and starts its own trace. Linking avoids making queue delay look like handler duration.

Connecting traces to SQL#

pg_stat_activity shows a slow query but not which request caused it. The sqlcommenter decorator can append the query span's traceparent plus selected route metadata. It remains off unless observability.comments is present, so configuring tracing alone does not change SQL text. See SQL Comments for the wiring, escaping and statement-cache trade-offs.

Sampling#

Trace everything in development, sample in production — a busy service produces more span volume than logs, and the cost is real:

import { NodeSDK } from '@opentelemetry/sdk-node';
import { TraceIdRatioBasedSampler, ParentBasedSampler } from '@opentelemetry/sdk-trace-base';

new NodeSDK({ sampler: new ParentBasedSampler({ root: new TraceIdRatioBasedSampler(0.05) }) });

ParentBasedSampler keeps a trace whole: if the caller sampled it, you sample it. Independent sampling per service produces fragments, which are worse than no trace.

Deliberate boundaries#

OpenTelemetry is not a dependency of the app or HTTP core. @zmdb/app/observability declares the narrow port, and the separately installed @zmdb/app/otel package adapts its sole required peer, @opentelemetry/api.

The port is a port rather than a claim of structural compatibility, which is a deliberately modest position. @opentelemetry/api's Tracer.startActiveSpan has four overloads and its Span has around ten methods, and the dependency-free core entry points cannot compile an assertion against that API. A claim that cannot be checked rots in silence. @zmdb/app/otel carries the API as both its required peer and development evidence, so the compatibility claim is typechecked at the integration boundary.

• No configured tracer or meter: the original router path runs after one branch; no no-op span is installed.
• No ambient context: the current span rides explicitly on Ctx.
• No exporter, SDK or collector configuration: application-owned.
• No global fetch patch and no sampling policy: SDK responsibilities.
• No interceptor span: runChain is still explicit rather than router-owned.

---

See also: Observability · SQL Comments · Logging