TelemetryProof

OpenTelemetry dashboard, alert, and SLO compatibility testing in CI.

TelemetryProof catches the consumer-side failures that schema validation can miss: a metric still exists but a dashboard selects zero series, an attribute rename breaks grouping, a unit change corrupts a threshold, or cardinality grows beyond budget.

Why this exists

OpenTelemetry Weaver validates semantic conventions and registry evolution. TelemetryProof complements it by executing explicit contracts for the downstream consumers—dashboards, alerts, and SLOs—against recorded OTLP data. Its narrow originality wedge is consumer semantics, not another telemetry schema linter.

Quick start

npm install -g https://github.com/virajsabhaya23/telemetryproof/releases/download/v0.1.0/telemetryproof-0.1.0.tgz
telemetryproof verify --contract examples/contract.json --telemetry examples/telemetry.json

No account, backend, network call, or paid service is required. Node.js 20+ is the only runtime dependency.

Contract checks

• exists: selector must match at least N points
• attribute: every selected point must preserve an attribute used by a query
• unit: metric unit must remain stable
• cardinality: bound distinct values for a label
• selectivity: compare match count with --baseline
• threshold: verify an aggregate value with >=, >, <=, <, or ==

Selectors currently support an exact metric name plus exact attribute filters. Input can be OTLP JSON or the compact fixture format shown in examples/.

CI

- run: npx telemetryproof verify --contract observability.contract.json --telemetry testdata/otlp.json --format sarif --out telemetryproof.sarif

The CLI returns 0 for pass, 1 for a contract violation, and 2 for invalid input. JSON reports include a SHA-256 evidence hash; SARIF integrates with code scanning.

Adoption path

Start with one critical alert or SLO, record representative OTLP fixtures before and after an instrumentation upgrade, and encode the exact metric/attribute/unit/cardinality assumptions. Publish minimized regressions upstream and preserve report hashes, versions, external integrations, and confirmed fixes as contribution evidence.

Scope and security

TelemetryProof reads local JSON and never sends telemetry anywhere. Treat fixtures as sensitive: sanitize customer identifiers before committing them. See SECURITY.md.

Research basis

• OpenTelemetry Weaver
• OpenTelemetry telemetry schemas
• Semantic-convention versioning and stability

Apache-2.0 licensed. Contributions are welcome.