CLIProxy observability

Private model-call capture for stock CLIProxyAPI 7.3.5. A separate native plugin and durable exporter feed this Convex component; the component stores bounded summaries and immutable private-content references. TypeScript owns client-protocol normalization.

This package is a release candidate. Native delivery and receiver normalization are implemented; publication, Meshix adoption and DEV activation require their separate release/deployment gates.

| Layer | Coverage | CI | | ----------- | -------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------- | | Unit | Protocol framing, recorded replay, inherited Responses regressions | check | | Integration | Authenticated receiver, Convex receipts/projection, private blob ownership | check | | e2e api | Isolated nginx + official stock binary + simulated provider; six real success fixtures and one real cancellation recording | native-capture (offline playback only) | | e2e web | None; host Ops integration is pending | No |

No deployed receiver or browser proof is claimed by these tests.

Native receipt storage and capacity

Native 0.2.2 removes the backlog scan from each destination-budget admission. SQLite triggers maintain pending bytes per destination across insert, delete, payload/state/destination changes and ACK rollback. Startup recounts every pending destination and refuses a mismatch without repairing or discarding data. The wire schema, queue sizes, byte budgets, retries and indefinite retention stay unchanged. A content-free localAdmission field in private boot health reports Unix admission attempts, exact ACKs, 507/other failures, total/max round-trip latency and sampled per-destination queue peaks. Receiver 0.3.3 accepts that optional field but does not project it; inspect the local persisted health row. These metrics describe local admission, not remote delivery or lossless capture.

The offline admission regression seeds 25,000 pending rows (about 148 MB), enables real delivery and destination budgets, and runs six concurrent Sol-like streams at 100 chunks/sec each. Its receiver stand-in drains at 50 events/sec. It verifies complete local capture under that bounded burst; it does not prove that a sustained campaign fits remote delivery throughput or finite storage. Migration and rollback against current production data, exact Linux artifacts and a separate coordinator window remain required before live replacement.

Native 0.2.1 stores immutable ACK receipts separately from pending payloads. The events read view reconstructs every historical field, including exact identity, digest, scope, timestamp and batch ID. Pending payloads move to the receipt ledger only in the same durable transaction as an exact ACK. Startup migration and reopening preserve unacknowledged records, including retired destinations.

GET /status reports current allocation, reusable pages, payload bytes, receipt count and admission headroom. Optional boot-health capacity carries the same information. Exhaustion estimates extrapolate measured net growth between samples; they disappear when there is no positive measured growth. They do not guarantee a failure time or bounded indefinite retention. Older health senders and receivers remain compatible.

Stop the exporter before migration or rollback. Before using a candidate, qualify the exact binaries, logical-row preservation, disk reserve, replay/conflict rules, outage/restart paths and the isolated stock/nginx fixture. Keep rollback binaries. For rollback, use the candidate exporter with --restore-legacy-outbox --socket /absolute/private/capture.sock --db /absolute/private/events.db and the installed budget/reserve settings. It acquires the service's existing socket lock and transactionally restores current logical rows for the prior exporter; an old database snapshot would discard observations captured after migration. Restoration requires enough space within the unchanged database ceiling and filesystem reserve. Recheck complete fingerprints and integrity before starting the prior exporter.

The 1.6-million-receipt benchmarks are explicit, offline measurements: go test ./internal/capture -run '^$' -bench 'BenchmarkReceipt(Ledger|AcknowledgementBurst)1600000' -benchtime=1x. They compare equal ingress queues and isolate acknowledgement selection; they do not prove lossless production capture or live provider acceptance.

Host integration

Mount @shpitdev/convex-cliproxy-observability/convex.config with app.use. Construct CliproxyObservability(components.cliproxyObservability) in the host backend. Expose a dedicated HTTP action through handleCliproxyCaptureRequest; use private receiver credentials, exact destination/instance enrollment and private blob storage. Never pass receiver credentials to inference sandboxes.

The host supplies PrivateCaptureStorage.put/get and a durable scheduleProjection callback. Projection actions call projectPendingSegments, rescheduling when more is true. Scheduler failure returns 503 after the raw receipt, making exporter retries safe. The host must monitor and recover failed projection actions; persisted raw content and receipts remain independently readable. No TTL or cleanup cron removes calls, receipts or content by default.

The receiver accepts CaptureSegmentV1 from the /capture export: immutable NDJSON batches of up to 256 contiguous observations. A segment is at most 1408 KiB; the outer JSON envelope is at most 2 MiB. Each observation payload is at most 1 MiB; native 0.2.5 splits larger bodies into ordered bodyPart records (up to 256) that readers reassemble. The exporter must persist batch boundaries before retrying. Identical identities and digests deduplicate; changed digests or overlapping ranges conflict.

Reads and evidence

• pageRecentSummaries and pageCorrelationSummaries return at most three summaries under a 32 KiB response budget, using indexed keyset cursors.
• getCall reads one bounded record; pageEventSegments pages private references. resolveCallBlob requires membership in the host-authorized call manifest and checks size/hash while streaming.
• Raw admission and projection have separate sequence watermarks. Missing segments stop projection; invalid protocol data and missing usage remain explicit. Boot health retains loss counters and labels incomplete prior-boot calls unknown.
• ModelCallV1 is the common read contract. fromOpenRouterSpan is a structural adapter with no runtime dependency on the OpenRouter component. fromLegacyCliproxySpan labels historical rows distinctly.
• Gateway identity is known. Provider identity is derived with explicit provenance (see below), attempt detail is unavailable, and cost is never estimated. Pre-hook capture is unavailable; previews are not enrolled.

For capturePolicy: hook-body-v1, decode each observation body from base64 and verify contentBytes/contentSha256 before presenting it. Keep callbacks separate for raw evidence; their bytes may be partial JSON or UTF-8. request_after_auth is executed-request evidence, not the original client request. executionModel, executionProtocol, selectedAuthId and selectedAuthIndex are exact stock fields, not inferred provider or attempt identities. Header collections and opaque auth metadata are excluded. Model payload content is unredacted and must remain private.

projectCapturedPayloads reconstructs selected-call content from bounded pages. For hook-content-block-v1, verify the whole record body/digest, then use the shared /protocols export expandStockHookBlock(body, stockHookChunkLengths) before replay. Never treat concatenated bare JSON objects as one frame. Deploy receiver 0.3.7 and compatible readers before the producer. Native 0.2.5 records with bodyPart are exact 1 MiB slices of one larger body: feed every validated observation, in sequence order, to /protocols' BodyPartAssembler and use the body it returns at the final part (it verifies order, offsets, length and SHA-256); stockHookPieces(observation, body) returns block callbacks or the single split callback. Bound BodyPartAssembler(maxBytes) to the read budget; in a 64 MiB Convex action, 8 MiB (MAX_PROJECTED_SPLIT_BYTES, which package projection uses) leaves room to decode the result. Keep gaps and incomplete capture explicit; an incomplete terminal may have no known provider outcome. Pass stockHookChunks: true only for raw stock hook pieces from either hook policy; historical observations already contain canonical framing. Its 16 MiB replay limit is a read-operation bound, not a retention limit. Larger calls remain available through raw event pages. Summary checkpoints retain scalar state and private references for large unfinished frames; immutable segments retain text, reasoning, tools and native usage snapshots.

Provider, tokens and cost

providerIdentity derives providerName and providerProvenance from one call's recorded facts, never from time or call order. The first rule that applies wins:

1. observed: the recording itself names the provider. Only the OpenRouter adapter sets this today, and reads keep any provider name an older summary stored. For CLIProxy calls, the only exact source would be the native plugin recording the selected auth's provider on request.intercept_after; that is a follow-up.
2. derived_from_wire_format: executionProtocol, the upstream wire format CLIProxy recorded after auth (its ToFormat), is in PROVIDER_BY_WIRE_FORMAT. claude maps to anthropic; openai, openai-response and codex to openai; gemini, gemini-cli and antigravity to google. This names the API family, not the vendor account: claude also reaches other Anthropic-compatible hosts, and CLIProxy uses codex for xAI and Meta as well.
3. derived_from_model: PROVIDER_BY_MODEL_PREFIX matches the requested model, including when the wire format is absent or unmapped (such as interactions). claude- maps to anthropic; gpt-, o plus a digit, and codex to openai; gemini- to google. CLIProxy can route any alias to any upstream, so this is only a guess.
4. unavailable when no rule applies.

The recordings in fixtures/real have no after-auth frames, so they derive openai from gpt-5.6-luna although their recorded upstream was a gateway.

Token fields use the OpenRouter names and unit (tokens per call): inputTokens, outputTokens, totalTokens, reasoningTokens, cachedInputTokens (cache reads) and cacheCreationInputTokens (cache writes). They come only from terminal client-protocol usage and are set only when that usage is final. The rule is one table in src/protocols/usage.ts:

• Input includes cache reads and writes. Anthropic reports cache_read_input_tokens and cache_creation_input_tokens apart from input_tokens, so its input is their sum. Anthropic and CLIProxy's translators omit a cache count when no cache is used, so an absent one counts as 0; input_tokens itself is required.
• totalTokens is the reported total_tokens. Anthropic Messages reports none, and this package never sums one.
• Cache reads: Anthropic cache_read_input_tokens, Responses input_tokens_details.cached_tokens, Chat prompt_tokens_details.cached_tokens.
• Cache writes: Anthropic cache_creation_input_tokens (its cache_creation.ephemeral_* split stays raw), Responses input_tokens_details.cache_write_tokens or cache_creation_tokens, Chat prompt_tokens_details.cache_write_tokens, cached_creation_tokens or cache_creation_tokens. The first reported alias wins.
• Reasoning: Anthropic output_tokens_details.thinking_tokens, Responses output_tokens_details.reasoning_tokens, Chat completion_tokens_details.reasoning_tokens.

Every reported native count also stays in usage as a UsageMeasurement with its nativeField, finality and semanticsVersion: "client_protocol_usage_v2".

Cost is never estimated. Native calls keep cost: { kind: "unknown" } with costProvenance: "unknown", because CLIProxy's client protocols report no price. fromOpenRouterSpan keeps OpenRouter's charge as a proxy_reported USD cost. On a BYOK call that charge is only OpenRouter's fee, so the adapter adds byokInferenceUsageCost, the inference billed to the caller's own provider key; a BYOK span without it has an unknown cost.

providerProvenance and costProvenance are optional in ModelCallV1. Summaries projected by 0.3.0 or later store them. For older summaries, getCall and pageRecentSummaries derive them from the facts those summaries recorded. The same reads apply the current token rule to an older summary's final native counts, which adds cacheCreationInputTokens and counts absent Messages cache counts as 0. Stored rows are not rewritten.

Projected summaries of the real recordings measure 1,783 to 2,934 bytes. A streamed Opus call with an after-auth frame, all ten correlations and every Anthropic usage count measures 3,771 bytes, under the 8 KiB summary cap. With every bounded string at its validator maximum a summary still reaches 9,583 bytes (10,253 before 0.3.0), so the cap is not a guarantee for adversarial metadata.

Differences from the OpenRouter component

• The OpenRouter component stores costs as bare numbers with no currency; fromOpenRouterSpan labels them USD. CLIProxy records no cost.
• Provenance exists only in ModelCallV1. The OpenRouter component's own summaries carry neither provider nor cost provenance.
• OpenRouter's providerName is the upstream it routed to, as OpenRouter names it (for example OpenAI). CLIProxy's is a lowercase API family or model-name guess. It is not authoritative until the native plugin records the selected auth's provider.
• OpenRouter component summaries from 0.4.2 carry cacheCreationInputTokens, isByok, and byokInferenceUsageCost; the structural OpenRouterSpanInput accepts all three. Older summaries lack them, so their cache writes stay unset and a BYOK call's cost is its fee alone.
• OpenRouter summaries expose input, output and total tokens; its cached and reasoning counts are only on full spans (exportFullSpan).

Parser provenance

The client-protocol assemblers and Responses regressions were adapted from Meshix commit 737ea929919f82921accaa4171d6abed5c85a1cb (the reviewed #1371 stack). Synthetic log fixtures are extracted only in tests; this package does not ingest legacy text logs. Sanitized real recordings and their provenance live in fixtures/real.

Run pnpm --filter @shpitdev/convex-cliproxy-observability test, typecheck, and pack:check. The latter installs the tarball in an isolated consumer with the minimum supported Convex version and checks exported APIs plus executable component test modules.

Copyright 2026 Anand Pant. Apache-2.0; the native ABI adapter includes its upstream MIT notice.