watchtower-collector

Central metrics collector for watchtower-agent. Ingests Prometheus text format, stores in a ring buffer, evaluates alert rules, and fires webhooks to Slack, Discord, and email.

Install

npm install

Setup

1. Copy .env.example to .env and fill in your credentials:

cp .env.example .env

SLACK_WEBHOOK_URL=https://hooks.slack.com/services/...
DISCORD_WEBHOOK_URL=https://discord.com/api/webhooks/...
RESEND_API_KEY=re_xxxxxxxxxxxx
[email protected]
[email protected]

2. Start the collector:

node src/collector.js
# [watchtower-collector] running on http://localhost:9090

Alert rules

Edit alerts.json to configure rules:

[
  {
    "name": "high-error-rate",
    "service": "*",
    "metric": "http_requests_total",
    "condition": "error_rate > 0.05",
    "window": 4,
    "channels": ["slack", "discord", "email"]
  },
  {
    "name": "high-p95-latency",
    "service": "*",
    "metric": "http_request_duration_ms",
    "condition": "p95 > 400",
    "window": 4,
    "channels": ["slack", "discord", "email"]
  }
]

| Field | Description | |---|---| | service | Service name or "*" for all | | condition | error_rate > 0.05 or p95 > 400 (ms) | | window | Number of ring buffer ticks to evaluate over | | channels | slack, discord, email |

Alerts deduplicate — the same rule won't refire within 5 minutes.

API

| Endpoint | Description | |---|---| | POST /ingest | Receive Prometheus text (set X-Service-Name header) | | GET /metrics/:service | Latest snapshot for a service | | GET /services | List all registered services |

Getting webhook URLs

Slack: Workspace Settings → Manage Apps → Incoming Webhooks → Add to Slack

Discord: Channel Settings → Integrations → Webhooks → New Webhook → Copy URL

Resend (email): resend.com → API Keys → Create. Free tier: 100 emails/day.

License

MIT