@aidex/observability

Installation

pnpm add @aidex/observability

npm install @aidex/observability

Reusable observability utilities for Aidex: measuring execution duration, collecting an ordered event timeline, estimating token cost, and logging a strategy's start/finish safely. This package is not a Plugin — it has no dependency on Plugin, Lifecycle, or Aidex at all. It is meant to be consumed by a future observability Plugin (in @aidex/plugins), which would compose these primitives around the beforeExecute/afterExecute hooks.

Contents

• metrics/ExecutionMetrics — pure duration calculation: recordStart(), recordEnd(), getDuration(). No logging, no persistence, no timestamp generated unless the caller omits one.
• timeline/Timeline — an ordered, caller-driven event collector. Never generates its own timestamp; only preserves insertion order for whatever ObservabilityEvents the caller records (e.g. started, provider-called, provider-returned, strategy-finished).
• cost/CostEstimator (estimateCost) — pure math over { inputTokens, outputTokens, inputPricePerMillion, outputPricePerMillion }. No hardcoded vendor pricing table of any kind — the caller supplies prices.
• logger/ExecutionLogger — a thin, safe ILogger wrapper for a strategy's start/finish. Never throws, even if the supplied logger itself throws or is undefined.
• types/ObservabilityEvent — the shared event shape ({ event, metadata? }) Timeline records. Deliberately generic — no provider, vendor, or application field of any kind.
• bus/ObservabilityBus — the unified event system: subscribe(handler) (returns an unsubscribe function) / emit(event), backed by a Timeline for ordered history (getTimeline()). Eight named track*() convenience methods cover every signal this package tracks — trackTokens, trackCost, trackDuration, trackProvider, trackEngine, trackWorkflow, trackError, trackRetry — each just emit()ing a well-known event name (ObservabilityEventName) with caller-supplied metadata. trackCostFromEstimate()/trackDurationFromMetrics() tie the bus to the existing estimateCost()/ExecutionMetrics rather than duplicating their math. Three more — trackRoutingDecision(), trackAttempt(), trackGuardrailDenial() — track Gateway routing/retry/fallback/guardrail-denial correlation; see "Routing and guardrail observability" below.

Routing and guardrail observability (Gateway 1.1)

Three more ObservabilityBus methods, added for @aidex/gateway's routing/retry/fallback loop and @aidex/guardrails integration — this package still knows nothing about Gateway or guardrails themselves; it only defines the metadata shapes below and emits them under three more well-known event names (ObservabilityEventName.ROUTING/ATTEMPT/ GUARDRAIL).

• trackRoutingDecision(metadata: RoutingDecisionMetadata) — the ranked candidate set considered for one execution, and (once execution concludes) which one was actually selected.

  interface RoutingDecisionMetadata {
    executionId: string;
    candidatesConsidered: readonly RoutingCandidateInfo[];
    selected?: RoutingCandidateInfo; // absent if every candidate was exhausted without success
  }
  interface RoutingCandidateInfo {
    providerType: string;
    modelId: string;
  }

• trackAttempt(metadata: AttemptMetadata) — one candidate's outcome for one dispatch — an individual attempt, a retry, and a fallback are all the same event, distinguished by outcome.

  type AttemptOutcome = 'success' | 'retry' | 'fallback' | 'exhausted';
  interface AttemptMetadata {
    executionId: string;
    candidate: RoutingCandidateInfo;
    outcome: AttemptOutcome;
    errorType?: string; // a class name only (e.g. "ProviderRateLimitError") — never the raw error/message
    retryable?: boolean;
    durationMs: number;
    startedAt: string; // ISO timestamp
    attempt?: number;  // which attempt against this candidate (1 after the first try)
  }

• trackGuardrailDenial(metadata: GuardrailDenialMetadata) — a guardrail denied the request, at the input or output stage.

  type GuardrailDenialStage = 'input' | 'output';
  interface GuardrailDenialMetadata {
    executionId: string;
    stage: GuardrailDenialStage;
    code?: string; // a short classification token (e.g. "content_too_large") — see below
  }

executionId is mandatory on all three — unlike every other track*() method's free-form metadata bag, none of these events can be tied back to the request they belong to without it.

Never the raw denial reason. The GuardrailDeniedError a denying guardrail causes @aidex/gateway to throw back to the caller carries a full, human-readable reason string. GuardrailDenialMetadata deliberately does not carry it — only code, a short, structural classification token every built-in guardrail supplies (e.g. "content_too_large"), because reason is free-form and an application-supplied guardrail could put anything into it. The same "structural signal only, never a raw message" discipline applies to AttemptMetadata.errorType (a class name, never error.message).

Read these back like any other event, via subscribe() or getTimeline():

const bus = new ObservabilityBus();
bus.subscribe((event) => {
  if (event.event === 'guardrail') {
    console.log(event.metadata); // { executionId, stage, code } — never `reason`
  }
});

@aidex/gateway's GatewayExecutor is the real producer of all three events; @aidex/admin's Routing and Guardrails sections are the intended consumer. See those packages' own READMEs for the full integration.

Independence

This package knows nothing about Gemini, OpenAI, Claude, Ollama, Design Platform, Print Platform, or any other application or vendor. It only observes execution in the abstract — durations, ordered events, token-based cost math, and log lines — and every one of those inputs is supplied by the caller.

Architecture rules this package follows

• Composition only — no inheritance, no abstract base class, no class hierarchy anywhere in this package.
• No singletons, no global mutable state — every class is instantiated by its caller and holds only its own instance state.
• Pure functions where state isn't needed (estimateCost); small focused classes where it is (ExecutionMetrics, Timeline, ExecutionLogger).
• src/index.ts exports the public utilities and their I/O types only.

Dependency direction

@aidex/observability depends on @aidex/core only (for ILogger). It does not depend on, and must never depend on, @aidex/providers, @aidex/strategies, @aidex/plugins, or any future workflow/memory/sdk package.