Mapbox Bus

Teams in Mapbox often share data using scheduled jobs and one‑off links between services. This causes delays (not real‑time), many moving parts, and different approaches across teams. Mapbox Bus is a shared, event‑driven way to publish and subscribe to signals so info moves in near real time with less custom work. See the original SDP for background and trade‑offs: https://docs.google.com/document/d/1eThlNwdrFgPR0ehtFv02YrepiDz72EoOlLAsPjjGRIs/edit

This repo contains:

• A small TypeScript library for creating, parsing, and validating bus events using JSON Schema.
• An AWS CDK stack that provisions the central EventBridge event bus, archive, rules/targets, and DLQs.

Event bus mapbox-bus (AWS EventBridge), lives in data-tooling-shared-services-prod / data-tooling-shared-services-stg AWS accounts.

Message in #ask-data-tooling Slack channel for support.

Adding a new event schema

1. Create the JSON Schema in schema/events/ named exactly as the detailType (e.g., edit-suggestion-created-v1.json). The schema must include:

• detailType (const string), source (string), detail.metadata (ref to shared metadata), and detail.data (your payload).

2. Update the schema index at schema/index.json to include a $ref to the new schema.

3. Generate TypeScript types:

npm run type:gen

This updates src/types/index.ts and exposes your new event union type.

4. Add/adjust routing rules and IAM roles in the CDK stack if your event needs consumers. Each rule needs a dedicated IAM role with sqs:SendMessage permission on the target queue, passed as roleArn on the rule target.

5. Add tests in test/events/* that create and parse the event to guard the schema.

6. Update package version in package.json (e.g., 1.0.1 → 1.0.2).

7. Publish the new version of the package after merging the PR:

mbx npm publish

Consuming events

Event routing is configured with EventBridge rules in the central CDK stack. Targets can be Lambda, SQS, SNS, etc. We usually use SQS with a DLQ per target.

Filtering examples (EventBridge event pattern):

{ "detail-type": ["edit-suggestion-created-v1"] }

How to subscribe

• If your service needs an event, add a rule + target in cdk/lib/event-bridge-stack.ts via PR, with a DLQ for the target. Make the pattern as specific as needed (by source, detail-type, and optionally detail.data.*).
• For cross‑account delivery or publishing, include the AWS account IDs so we can update the bus policy.

Rule examples you can mirror:

• edit-suggestion-created__export-to-to-fix
• edit-suggestion-created__send-submission-email
• to-fix-item-updated__update-feedback-status

Make sure to add the Bus role to your target's IAM resource policy so EventBridge can deliver to it.

Example for SQS:

targetQueue.addToResourcePolicy(
  new iam.PolicyStatement({
    effect: iam.Effect.ALLOW,
    principals: [new iam.ArnPrincipal('ROLE ARN')], // IAM role created for your rule in event-bridge-stack.ts
    actions: ['sqs:SendMessage'],
    resources: [targetQueue.queueArn],
  }),
);

Using the library

Install (published as @mapbox/bus-events):

npm install @mapbox/bus-events

Create a new event object (adds metadata and validates against the JSON Schema):

import { createEvent } from '@mapbox/bus-events';

const event = await createEvent({
  detailType: 'edit-suggestion-created-v1',
  source: 'contribute',
  detail: {
    data: {
      editSuggestionId: 'es_123',
      type: 'poi',
    },
  },
});

Validate an event you already have:

import { validateAgainstSchema } from '@mapbox/bus-events';

await validateAgainstSchema(event, event.detailType);

Parse events from EventBridge (hyphenated/camel case handled):

import { parseEvent } from '@mapbox/bus-events';

// data can be a JSON string or object; detail-type or detailType are both accepted
const parsed = await parseEvent(incoming, /* optional */ 'edit-suggestion-created-v1');

Publish to EventBridge (example with AWS SDK v3):

import { EventBridgeClient, PutEventsCommand } from '@aws-sdk/client-eventbridge';
import { createEvent } from '@mapbox/bus-events';

const client = new EventBridgeClient({ region: process.env.AWS_REGION });

const event = await createEvent({
  detailType: 'edit-suggestion-created-v1',
  source: 'contribute',
  detail: { data: { editSuggestionId: 'es_123', type: 'poi' } },
});

const isProduction = process.env.NODE_ENV === 'production';

// requires events:PutEvents permission
await client.send(
  new PutEventsCommand({
    Entries: [
      {
        EventBusName: isProduction
          ? 'arn:aws:events:us-east-1:859060840721:event-bus/mapbox-bus'
          : 'arn:aws:events:us-east-1:571600836192:event-bus/mapbox-bus',
        Source: event.source,
        DetailType: event.detailType,
        Detail: JSON.stringify(event.detail),
      },
    ],
  }),
);

Goals and non‑goals

Goals

• One simple way to publish signals and events.
• Let teams subscribe only to what they need using filters.
• Cut delays by moving from batches to events.
• Reduce custom integrations and duplicate work.

Non‑goals

• Replace every current method (batches/APIs still fine when they fit).
• Be a data lake or long‑term store (archive/replay is for operations, not analytics).
• Control all event design (we give patterns, not strict rules).
• Provide strict transactions (we aim for reliable, at‑least‑once delivery).

Architecture overview

We use AWS EventBridge as the core event router.

• Event bus: mapbox-bus with a resource policy allowing events:PutEvents from the default Mapbox account; additional publishers can be explicitly allowed via policy.
• Archive: AllEvents with 35‑day retention for replay.
• Rules and targets: Defined centrally in this repo’s CDK stack. Targets are primarily SQS queues with per‑target DLQs. Example rules in cdk/lib/event-bridge-stack.ts route:
  • edit-suggestion-created-v1 (source contribute) to several SQS queues (export to To‑Fix, create feedback, send submission email).
  • to-fix-item-updated-v1 and to-fix-item-created-v1 (source to-fix) to update feedback status.
  • correction-processed-v1 (source osm-correction-engine) to update feedback.

Why EventBridge

• Flexible filtering and routing, multiple targets, managed scaling, archive/replay.
• Alternatives we looked at:
  • Kinesis: strong ordering and streaming, but more ops and higher cost here.
  • SNS + SQS: cheap fan‑out, but weaker filtering and no native replay.

Event format and versioning

An event has AWS fields plus our detail payload. In code we use detailType (camelCase). In EventBridge it appears as detail-type (hyphenated). The library accepts both when parsing.

Top‑level fields

• AWS (may appear depending on context): version, id, time, account, region, resources.
• Required by us:
  • detailType: string, event name with version (e.g., edit-suggestion-created-v1).
  • source: string, service/domain (e.g., contribute, to-fix).
  • detail: object with:
    • metadata (added by the library):
      • eventId (uuid)
      • timestamp (epoch ms)
      • traceparent (OpenTelemetry)
      • tracestate (OpenTelemetry)
    • data: your payload.

Versioning

• Any breaking change creates a new version (e.g., *-v2).
• Each version is a separate event so old consumers keep working.
• Teams subscribe to specific versions and upgrade when ready.

Example (as seen inside AWS EventBridge)

{
  "source": "contribute",
  "detail-type": "edit-suggestion-created-v1",
  "detail": {
    "metadata": {
      "eventId": "6b6f7c9e-...",
      "timestamp": 1739999999999,
      "traceparent": "00-...",
      "tracestate": "vendor=..."
    },
    "data": {
      "editSuggestionId": "es_123",
      "type": "poi"
    }
  }
}

Security and access control

• Publishing: The bus policy explicitly lists which AWS accounts can call events:PutEvents. By default we allow the Mapbox default account; request additions via PR to the CDK stack.
• Subscribing: Rules and targets are defined centrally; IAM roles attached to targets are scoped to only required actions (e.g., sqs:SendMessage). Each target gets a DLQ.

Scaling and cost

Scale

• EventBridge scales automatically and supports high throughput (~10k PutEvents/sec per bus by default; can be increased with AWS Support). We can also shard by adding additional buses if ever needed.

Cost (order‑of‑magnitude for 10M events @ 64KB)

• Event publishing: ~$10/month.
• Cross‑account delivery: ~$10/month.
• Archive + 35‑day storage + replay: ~$85/month.

Release

To release a new version of the package use:

mbx npm publish

Docs: https://github.com/mapbox/mbxcli/blob/master/docs/commands/npm.md

Development

Build

npm run build

Generate types from JSON Schemas

npm run type:gen

Run tests

npm run test

Lint and format

npm run lint
npm run format