faultline-cli

Command-line interface for Faultline — push internal events, report errors, and manage service maintenance windows directly from your servers and CI/CD pipelines.

npm install -g faultline-cli

Why

Faultline's health checker monitors your services from the outside — HTTP endpoints, Docker containers, ECS tasks. But some failures are invisible from the outside:

• A background job that silently fails
• A queue processor that crashes and stops consuming
• A cron task that doesn't run
• An internal service that returns 200 but produces corrupted output

The CLI bridges that gap. Instrument your scripts, jobs, and application code to push events into Faultline when something goes wrong internally, even when your HTTP endpoint still looks healthy.

Installation

npm install -g faultline-cli

Requires Node.js 18 or later (uses the built-in fetch API).

Authentication

All CLI commands authenticate using a Faultline API key. Keys are generated in the Faultline dashboard under Settings → API Keys.

Two ways to provide your key:

Environment variables (recommended for servers and CI):

export FAULTLINE_API_KEY=flt_your_key_here
export FAULTLINE_API_URL=https://api.fltln.io

Config file (recommended for local use):

faultline configure

Saves credentials to ~/.faultline/config.json. Environment variables always take precedence over the config file.

Key scopes

When generating an API key in the dashboard, you can scope it to a specific service or leave it tenant-wide:

| Scope | Use case | |---|---| | Service-scoped | One key per server/service — the key is permanently tied to that service, no need to pass --service | | Tenant-wide | One key for all services — pass --service <id> per command to associate events with the right service |

For most setups, service-scoped keys are simpler and safer (a compromised key can only affect one service).

Commands

faultline configure

Interactive setup. Saves your API URL and key to ~/.faultline/config.json.

faultline configure
# API URL [https://api.fltln.io]:
# API Key (flt_...): flt_your_key_here
# Configuration saved to ~/.faultline/config.json

faultline ingest <message>

Push an internal event to Faultline. If an open incident already exists for the service, the event is appended to its timeline as a note. If no open incident exists, a new one is created and your notification channels are alerted.

faultline ingest "Payment processor job failed — 0 records processed"

Options:

| Flag | Default | Description | |---|---|---| | --severity <level> | MINOR | CRITICAL, MAJOR, or MINOR | | --service <serviceId> | (from key) | Associate with a specific service (only needed for tenant-wide keys) |

Examples:

# Report a failed cron job
faultline ingest "nightly-export cron failed: database timeout" --severity MAJOR

# Report a critical failure tied to a specific service
faultline ingest "payment queue consumer stopped" \
  --severity CRITICAL \
  --service clx8m2k3p0000abc123def456

# Append context to an already-open incident
# (if an incident is open for this service, this adds a timeline note instead of creating a new incident)
faultline ingest "retry attempt 3/3 also failed — giving up"

Output:

Incident created: clx9n3l4q0001xyz789ghi012
# or, if an incident was already open:
Appended to incident: clx9n3l4q0001xyz789ghi012

faultline ask <question>

Ask the AI ops assistant about your infrastructure in plain English. It runs a conversational agent against Faultline's /agent API — listing services, inspecting incidents, diagnosing what's wrong, and recommending a fix.

faultline ask "why is checkout-api degraded?"
faultline ask "what's down right now and who's on call?"
faultline ask "diagnose incident clx9n3l4q0001 and fix it if there's a runbook"

Actions that change things — running a runbook or resolving an incident — pause for an explicit confirmation before they execute:

  → diagnose_incident…
⚠️  Approve run_runbook({"incidentId":"clx9…","runbookId":"rb_…"})? [y/N]

Requires ANTHROPIC_API_KEY in your environment (the model runs locally; your Faultline API key authorizes the data access). Read-only questions never prompt.

faultline check create <name>

Create a check. Two types are supported:

• --type heartbeat (default) — a dead-man's-switch for a cron job, queue worker, or backup script. Faultline expects a ping every --interval seconds; if none arrives within interval + grace, it opens an incident. The command prints a ping URL to wire into your job.
• --type http — Faultline polls a --url every --interval seconds and checks the response.

# Heartbeat — for a cron or worker
faultline check create nightly-backup --interval 86400 --grace 600
# Created heartbeat check "nightly-backup" (service clx8m2k3p0000abc123def456).
#
# Ping URL — call this at the end of your cron or worker run:
#   curl -fsS https://api.fltln.io/hb/hb_AbC123...
#
# If no ping arrives within 87000s, Faultline opens an incident.

# HTTP — poll an endpoint
faultline check create api-health --type http --url https://example.com/health --interval 30
# Created HTTP check "api-health" (service clx8m2k3p0000abc123def456).
# Faultline will GET https://example.com/health every 30s.

Options:

• -t, --type <type> — heartbeat or http (default heartbeat)
• -i, --interval <seconds> — seconds between pings (heartbeat) or polls (http) (default 60)
• -g, --grace <seconds> — heartbeat only: extra slack before a missed ping counts as down (default 60)
• --url <url> — http only: URL to monitor (required for --type http)
• -m, --method <method> — http only: request method (default GET)
• -d, --description <text> — optional description for the check

For a heartbeat, wire the printed URL into the job — the token in the URL is the credential, so no API key is needed at ping time:

# crontab: ping Faultline only if the backup succeeds
0 3 * * * /usr/local/bin/backup.sh && curl -fsS https://api.fltln.io/hb/hb_AbC123...

faultline maintenance start <serviceId>

Pause health checks for a service. Use this before a deploy to prevent Faultline from creating false DOWN incidents while your service is intentionally offline or restarting.

faultline maintenance start clx8m2k3p0000abc123def456
# Service clx8m2k3p0000abc123def456 is now PAUSED — health checks suspended.

While paused, the service shows as PAUSED in the dashboard and the health checker skips it entirely.

faultline maintenance end <serviceId>

Re-enable health checks after a deploy. The service status returns to PENDING and the scheduler picks it up on its next cycle.

faultline maintenance end clx8m2k3p0000abc123def456
# Service clx8m2k3p0000abc123def456 is now PENDING — health checks will resume shortly.

Integration patterns

CI/CD deploy pipeline

Wrap your deploy step with maintenance start/end to avoid false alerts:

#!/bin/bash
set -e

SERVICE_ID="clx8m2k3p0000abc123def456"

# Pause health checks before deploy
faultline maintenance start $SERVICE_ID

# Your deploy step
./scripts/deploy.sh

# Resume health checks after deploy
faultline maintenance end $SERVICE_ID

If the deploy script fails, the service stays paused. Add a trap to always resume:

trap "faultline maintenance end $SERVICE_ID" EXIT
faultline maintenance start $SERVICE_ID
./scripts/deploy.sh

Cron job monitoring

Report failures from cron jobs or scheduled tasks. Combine with the ingest command to surface silent failures:

#!/bin/bash
# nightly-export.sh

python3 export_job.py
if [ $? -ne 0 ]; then
  faultline ingest "nightly-export failed — exit code $?" --severity MAJOR
  exit 1
fi

Or wrap the whole script to catch any error:

#!/bin/bash
run_job() {
  python3 export_job.py
}

if ! run_job; then
  faultline ingest "nightly-export cron failed on $(hostname)" --severity MAJOR
fi

Application-level error reporting

Call the CLI from application code for failures that are invisible to the HTTP health checker:

Node.js:

import { execSync } from "node:child_process";

function reportToFaultline(message, severity = "MINOR") {
  try {
    execSync(`faultline ingest "${message}" --severity ${severity}`, {
      env: {
        ...process.env,
        FAULTLINE_API_KEY: process.env.FAULTLINE_API_KEY,
        FAULTLINE_API_URL: process.env.FAULTLINE_API_URL,
      },
    });
  } catch {
    // Never let monitoring break the application
  }
}

// In your background job:
try {
  await processPaymentQueue();
} catch (err) {
  reportToFaultline(`Payment queue processor crashed: ${err.message}`, "CRITICAL");
  throw err;
}

Python:

import subprocess

def report_to_faultline(message: str, severity: str = "MINOR") -> None:
    try:
        subprocess.run(
            ["faultline", "ingest", message, "--severity", severity],
            timeout=10,
            check=False,
        )
    except Exception:
        pass  # Never let monitoring break the application

# In your worker:
try:
    process_queue()
except Exception as e:
    report_to_faultline(f"Queue worker crashed: {e}", "CRITICAL")
    raise

Shell script with env check:

#!/bin/bash
# Check that required env vars are set before the job runs
if [ -z "$DATABASE_URL" ]; then
  faultline ingest "DATABASE_URL not set — job aborted" --severity CRITICAL
  exit 1
fi

GitHub Actions

jobs:
  deploy:
    runs-on: ubuntu-latest
    env:
      FAULTLINE_API_KEY: ${{ secrets.FAULTLINE_API_KEY }}
      FAULTLINE_API_URL: https://api.fltln.io
    steps:
      - uses: actions/checkout@v4

      - name: Install faultline-cli
        run: npm install -g faultline-cli

      - name: Pause health checks
        run: faultline maintenance start ${{ vars.SERVICE_ID }}

      - name: Deploy
        run: ./scripts/deploy.sh

      - name: Resume health checks
        if: always()   # runs even if deploy fails
        run: faultline maintenance end ${{ vars.SERVICE_ID }}

      - name: Report deploy failure
        if: failure()
        run: |
          faultline ingest "Deploy failed on branch ${{ github.ref_name }}" \
            --severity MAJOR \
            --service ${{ vars.SERVICE_ID }}

Environment variables

| Variable | Description | |---|---| | FAULTLINE_API_KEY | Your API key (flt_...). Overrides the config file. | | FAULTLINE_API_URL | API base URL. Defaults to https://api.fltln.io if not set and no config file exists. | | ANTHROPIC_API_KEY | Required only for faultline ask — the model runs locally with your own key. |

Exit codes

| Code | Meaning | |---|---| | 0 | Success | | 1 | API error (4xx/5xx), invalid config, or missing credentials |

All error messages are written to stderr.

Config file

~/.faultline/config.json — created by faultline configure:

{
  "apiKey": "flt_your_key_here",
  "apiUrl": "https://api.fltln.io"
}

Environment variables take precedence over this file. The file is never read if both FAULTLINE_API_KEY and FAULTLINE_API_URL are set.

License

MIT