@veridot/kafka

Kafka-backed metadata broker for Veridot.

KafkaMetadataBroker distributes Veridot's public-key metadata across all your services through a Kafka topic, with a local LMDB cache for sub-millisecond reads. It's the recommended broker for high-throughput, multi-region deployments.

Installation

pnpm add @veridot/core @veridot/kafka

Quick start (with the Veridot facade)

import { Veridot } from '@veridot/core';
import { KafkaMetadataBroker } from '@veridot/kafka';

const broker = await KafkaMetadataBroker.of({
  clientId: 'billing-api',
  brokers: process.env.KAFKA_BROKERS!.split(','),  // ['kafka1:9092', 'kafka2:9092']
  topic: 'veridot_keys',                            // optional, defaults to 'veridot_keys'
  dbPath: './data/veridot-keys',                    // optional, LMDB cache path
});

const veridot = await Veridot.create({
  metadataBroker: broker,
  salt: process.env.VDOT_SALT!,
  hashPepper: process.env.VDOT_PEPPER!,
  expectedIssuer: 'https://auth.example.com',
  expectedAudience: 'billing-api',
});

const { accessToken, refreshToken } = await veridot.signWithRefreshToken(
  { sub: 'user-123', role: 'admin' },
  { subject: 'user-123' }
);

Options

| Option | Type | Default | Description | | ---------- | -------------------------- | --------------------- | ------------------------------------------------------------------------- | | clientId | string | — | Kafka client ID — should be unique per service instance. | | brokers | string \| string[] | — | Bootstrap brokers ('kafka:9092' or ['kafka1:9092', 'kafka2:9092']). | | topic | string | 'veridot_keys' | Kafka topic carrying the public-key metadata. | | dbPath | string | './veridot-keys' | LMDB cache directory (one per service instance, must be writable). | | logger | Logger (@veridot/core) | ConsoleLogger | Pluggable structured logger (pino / winston / NestJS / …). |

Architecture

┌──────────────┐  publish   ┌──────────────┐  consume  ┌──────────────┐
│   Issuer     │──────────▶│              │──────────▶│   Verifier   │
│   service    │            │   Kafka      │            │   service    │
│   (RSA priv) │            │   topic      │            │              │
└──────┬───────┘            └──────────────┘            └──────┬───────┘
       │                                                        │
       │  cache  ┌──────────────┐                cache  ┌──────▼───────┐
       └────────▶│   LMDB       │◀───────────────────────│   LMDB       │
                 │   (local)    │                         │   (local)    │
                 └──────────────┘                         └──────────────┘

• Producers publish key metadata to a Kafka topic.
• Every consumer keeps a local LMDB mirror so verify() is a sub-millisecond, zero-network operation in the steady state.
• LMDB survives restarts; on cold start, the broker replays from the topic.

Production tips

Kafka topic configuration

Create the topic with a long retention (or compaction) — clients that lag need to be able to fetch keys signed days ago:

kafka-topics.sh --create --topic veridot_keys \
  --partitions 3 \
  --replication-factor 3 \
  --config cleanup.policy=compact \
  --config min.compaction.lag.ms=86400000

cleanup.policy=compact keeps the last value for each key (= each keyId) forever, perfect for long-lived public keys.

Logger injection

import pino from 'pino';
const log = pino();

const broker = await KafkaMetadataBroker.of({
  clientId: 'billing-api',
  brokers: ['kafka:9092'],
  logger: {
    debug: (m, ctx) => log.debug(ctx, m),
    info:  (m, ctx) => log.info(ctx, m),
    warn:  (m, ctx) => log.warn(ctx, m),
    error: (m, ctx) => log.error(ctx, m),
  },
});

Graceful shutdown

process.on('SIGTERM', async () => {
  await veridot.shutdown();   // disconnects the broker as well
  process.exit(0);
});

Observability hooks

The broker also publishes keys:rotated events on the Veridot event bus:

veridot.events().on('keys:rotated', (e) => {
  metrics.gauge('veridot.active_key', 1, { kid: e.nextKeyId });
});

Environment variables (legacy fallbacks)

If you don't pass options, Config reads from the environment:

| Variable | Maps to | Default | | --------------------------------- | -------------------- | -------------------- | | VDOT_KAFKA_BOOTSTRAP_SERVERS | brokers | localhost:9092 | | VDOT_TOKEN_VERIFIER_TOPIC | topic | veridot_keys | | VDOT_EMBEDDED_DATABASE_PATH | dbPath | ./veridot-keys |

Explicit options always take precedence.

Lower-level usage (no facade)

If you need to drive the broker yourself:

import { GenericSignerVerifier, BasicConfigurer, TokenMode } from '@veridot/core';
import { KafkaMetadataBroker } from '@veridot/kafka';

const broker = await KafkaMetadataBroker.of({ clientId: 'svc', brokers: ['kafka:9092'] });
const signer = new GenericSignerVerifier(broker, process.env.VDOT_SALT!, {
  allowedAlgorithms: ['RS256'],
});

const config = new BasicConfigurer().mode(TokenMode.JWT).validity(15, 'minutes');
const token  = await signer.sign({ userId: 1 }, config);
const data   = await signer.verify(token, JSON.parse);

await signer.shutdown();
await broker.disconnect();

Related packages

• @veridot/core — Core facade & interfaces
• @veridot/databases — SQL alternative + persistent stores
• @veridot/redis — Redis-backed refresh-token / revocation stores
• @veridot/nestjs — NestJS module

License

MIT — see LICENSE.