Observability — request IDs + distributed tracing

CloakAPI emits a unique identifier on every request, plus a W3C-compliant traceparent header that lets you correlate gateway logs, sidecar logs, and upstream-provider calls into a single distributed trace.

Why this matters

When a customer reports “request from 14:32 returned an error”, support needs to join logs across:

• the gateway (your HTTP call landed here),
• the upstream-proxy sidecar (we held an HTTP/2 connection to OpenAI / Anthropic / Grok),
• the cloak-receipt sidecar (we signed the receipt),
• and the provider itself (Anthropic, OpenAI, etc.).

Without a stable correlator the join is guesswork. With request IDs and traceparent it’s a grep.

X-Request-Id

Every response from api.cloakapi.io includes:

X-Request-Id: 0d7b8f31-2e9a-4f57-b631-7a8f9d3c1e4d
X-Response-Time: 184ms

• Server-generated UUIDv4 by default.
• You can pre-set one by sending X-Request-Id: <uuid> on the inbound request — the server will accept and echo any UUIDv4 ≤ 128 characters, otherwise it generates one.
• The same ID is recorded on usage_records.request_id, the receipt envelope extensions.request_id, and every structured log line for that request.

Include this ID in any support ticket and we can trace your call through every system in seconds.

W3C traceparent

CloakAPI implements W3C Trace Context. Send a standard traceparent header and the same trace ID propagates across:

1. Gateway (Laravel) — root span, attributes: http.method, http.route, http.status_code, duration_ms.
2. cloak-upstream-proxy (Go) — child span, attributes: http.host, elapsed_ms.
3. cloak-receipt (Rust) — child span, attributes: op (sign/verify), elapsed_ms.
4. Upstream provider — traceparent is forwarded verbatim to OpenAI/Anthropic/Grok.

Sending a traceparent

TRACE_ID=$(openssl rand -hex 16)
SPAN_ID=$(openssl rand -hex 8)
curl -X POST https://api.cloakapi.io/api/v1/chat/completions \
  -H "Authorization: Bearer $CLOAKAPI_KEY" \
  -H "Content-Type: application/json" \
  -H "traceparent: 00-${TRACE_ID}-${SPAN_ID}-01" \
  -d '{"model":"claude-haiku-4-5","messages":[{"role":"user","content":"hello"}]}'

The 01 flag at the end is the sampled flag — set to 01 to force-sample, 00 to leave sampling to the server’s default 1% ratio. Errors (5xx) are always sampled at 100%.

Forcing a trace (admin only)

For debugging, you can force a request to be sampled even if it would otherwise be dropped at the 1% ratio:

X-CloakAPI-Force-Trace: 1

This header is honoured on all /api/v1/* endpoints. Use it sparingly; it bypasses sampling and emits a span to our OTLP collector for every request.

Idempotency-Key

In addition to tracing, every mutating endpoint accepts an Idempotency-Key header. This is a separate concern (safe retries) but is part of the same observability story: each retry carries the same key, the server returns the cached response, and the response carries X-CloakAPI-Idempotent-Replay: true.

Endpoints that honour Idempotency-Key today:

Endpoint	Method	Notes
/api/v1/chat/completions	POST	Streamed + non-streamed
/api/v1/messages	POST	Anthropic-format
/api/keys	POST	Create API key
/api/keys/{id}	PUT, PATCH	Update API key
/api/keys/{id}/rotate	POST	Rotate API key
/api/billing/topup, /api/v1/billing/topup	POST	Stripe checkout intent
/api/billing/checkout/topup	POST	Stripe checkout intent
/api/billing/redeem-license	POST	License-code credit

The same key with a different request body returns 409 Conflict with error.code = idempotency_key_conflict (per IETF httpapi-idempotency-key-header draft).

Keys are scoped per user — two different users sending the same UUID will never collide.

Putting it together: a support ticket worth answering

Bad:

“I got an error around 2pm yesterday.”

Good:

X-Request-Id: 0d7b8f31-2e9a-4f57-b631-7a8f9d3c1e4d
traceparent: 00-4bf92f3577b34da6a3ce929d0e0e4736-00f067aa0ba902b7-01
Idempotency-Key: 1d9c0f24-2b7c-4d68-9b4f-bb88fa9d4d5a
Error response body: {"error":{"code":"upstream_timeout", ...}}

With those four lines we can pull the exact gateway log line, the matching sidecar forward, the provider’s response timing, and confirm whether your retry hit the idempotency cache.

Where the IDs show up

Surface	Field
HTTP response	X-Request-Id, X-Response-Time, traceparent
usage_records table	request_id
Receipt envelope	extensions.request_id
Structured logs (JSON)	trace_id, span_id, request_id
OTLP/Tempo trace	traceId, spanId
Upstream-proxy logs	traceparent=... request_id=...
Receipt-sidecar logs	traceparent=... request_id=...