@m-software-engineering/heat-collector

Express collector service for heat-tracker. It provides ingestion and query APIs, supports SQL/MongoDB backends, and can authenticate by project key, JWT, or both.

Install

npm install @m-software-engineering/heat-collector
# or
pnpm add @m-software-engineering/heat-collector
# or
yarn add @m-software-engineering/heat-collector

Database drivers (peer dependencies)

• Postgres: pg
• MySQL: mysql2
• SQLite: better-sqlite3
• MongoDB: mongodb

Quick start (Express)

import express from "express";
import { createCollector } from "@m-software-engineering/heat-collector";

const collector = await createCollector({
  db: { dialect: "sqlite", file: "./heat-tracker.db" },
  auth: { mode: "projectKey" },
  autoMigrate: true
});

const app = express();
app.use(collector.router);

app.listen(3000, () => {
  console.log("Collector listening on http://localhost:3000");
});

Quick start (NestJS)

Register collector.router in main.ts after creating the Nest application:

import { NestFactory } from "@nestjs/core";
import { AppModule } from "./app.module";
import type { NestExpressApplication } from "@nestjs/platform-express";
import { createCollector } from "@m-software-engineering/heat-collector";

async function bootstrap() {
  const app = await NestFactory.create<NestExpressApplication>(AppModule);

  app.enableCors();

  const collector = await createCollector({
    db: { dialect: "sqlite", file: "./heat.db" },
    auth: { mode: "projectKey" },
    autoMigrate: true
  });

  app.use(collector.router);

  await app.listen(3000, "0.0.0.0");
}

bootstrap();

How the collector works

1. POST /ingest receives event batches from the SDK.
2. The request is authenticated via x-project-key, JWT, or both.
3. The collector upserts project/user/session data and stores events.
4. Query APIs return aggregated points and session timelines for dashboards.

Endpoints

Ingestion

POST /ingest

Accepted headers:

• x-project-key: required for projectKey and both modes.
• authorization: Bearer <jwt>: required for jwt, optional for both.

Query APIs

• GET /api/projects/:projectId/heatmap
• GET /api/projects/:projectId/events
• GET /api/projects/:projectId/sessions
• GET /api/sessions/:sessionId
• GET /api/metrics

Response headers and error payloads

All endpoints now include:

• X-Request-Id: request correlation id for logs/troubleshooting.
• X-Heat-Collector: static collector identifier.

Query endpoints also send:

• Cache-Control: no-store
• X-Total-Count or endpoint-specific count headers.

Error payload format is consistent across routes:

{
  "error": "invalid query",
  "code": "invalid_query",
  "message": "Invalid heatmap query parameters.",
  "requestId": "..."
}

Validation failures include a details object.

Heatmap query behavior notes

• type now accepts: all, click, move, scroll, pageview, custom, input, keyboard.
• If type is omitted, the collector first tries click (backward compatible) and automatically falls back to all when click has no matches.
• Heatmap points are only plotted from click, move, and scroll; other event types still appear in heatmap metadata (total, typeBreakdown, warning) for better debugging.

Configuration

type CollectorConfig = {
  db:
    | { dialect: "pg" | "mysql" | "sqlite"; connectionString?: string; file?: string }
    | { dialect: "mongodb"; connectionString?: string; database?: string };
  auth:
    | { mode: "projectKey" }
    | { mode: "jwt"; jwksUrl: string; issuer: string; audience: string }
    | { mode: "both"; jwksUrl: string; issuer: string; audience: string };
  autoMigrate?: boolean;
  ingestion?: {
    maxBodyBytes?: number;
    rateLimit?: { windowMs: number; max: number };
  };
  hooks?: {
    onBeforeInsert?: (payload: any) => any | Promise<any>;
  };
  logging?: { level: "debug" | "info" | "warn" | "error" };
};

MongoDB permissions note

When autoMigrate is enabled, the collector attempts to create indexes. If the MongoDB user does not have index-management permissions, the collector now continues without failing startup. For controlled production environments, you can still set autoMigrate: false and manage indexes externally.

Troubleshooting not authorized on <db> to execute command { find: "projects" }

This error usually means the collector is querying a different database than the one your MongoDB user can access.

Use one of these options:

1. Put the database directly in the connection string path:

db: {
  dialect: "mongodb",
  connectionString: "mongodb://user:password@host:27017/heat_tracker?authSource=admin"
}

2. Or explicitly set database in the collector config:

db: {
  dialect: "mongodb",
  connectionString: "mongodb://user:password@host:27017/?authSource=admin",
  database: "heat_tracker"
}

The collector now prefers db.database when provided, otherwise it uses the database from the MongoDB connection string.

Production migration command

HEAT_DIALECT=pg DATABASE_URL=postgres://... heat-collector-migrate