faultline-remediation-agent

Let Faultline runbooks restart a systemd service or Docker container on a host Faultline can't reach directly — your own VPS, an on-prem box, anything behind a firewall or NAT.

npm install -g faultline-remediation-agent

Why

Faultline's restart_ecs_service / scale_ecs_service runbook steps call the AWS ECS API directly, which only works for AWS-hosted services. For everything else — a systemd unit on a VPS, a Docker container on your own box — Faultline has no way to reach in. It doesn't hold SSH keys to your infrastructure and never opens a connection to it.

This agent closes that gap from the other direction: it runs on your host and polls out to Faultline for work.

runbook step fires  →  Faultline queues a job  ←  this agent polls, claims it,
                                                     runs it locally, reports back

No inbound port is ever opened on your host. If the agent process is down, nothing about it is reachable from the outside — the runbook step just times out after 5 minutes and fails, same as any other failed step.

Installation

npm install -g faultline-remediation-agent

Requires Node.js 18 or later. systemctl and/or docker must be on PATH for whichever action types you use.

Setup

1. Register the agent — Faultline dashboard → Settings → API Keys → New Agent. Name it (e.g. the hostname it'll run on) and copy the token shown once (fla_...) — it isn't shown again.

2. Configure it:

   faultline-remediation-agent init

   Prompts for the token and the rest of the settings below, then writes /etc/faultline-remediation-agent/agent.env (or a path you choose). If it can't write there (e.g. no sudo), it prints the file instead so you can save it yourself. Prefer to do it by hand? Copy agent.env.example to that path instead.

3. Run it as a service — copy faultline-remediation-agent.service to /etc/systemd/system/, then:

   systemctl enable --now faultline-remediation-agent

   Or run it directly for testing:

   FAULTLINE_AGENT_TOKEN=fla_... faultline-remediation-agent

4. Use it in a runbook — in a runbook's step editor, choose Restart Service (via Agent) or Restart Docker Container (via Agent), pick this agent from the dropdown, and name the unit or container.

Configuration

All configuration is environment variables (see agent.env.example):

| Variable | Required | Default | Description | |---|---|---|---| | FAULTLINE_AGENT_TOKEN | Yes | — | This agent's token (fla_...), from Settings → API Keys → New Agent | | FAULTLINE_API_URL | No | https://api.fltln.io | Faultline API origin to poll | | POLL_INTERVAL_SECONDS | No | 5 | How often to poll when idle. Ignored while jobs are queued — it drains all of them back-to-back before waiting again | | FAULTLINE_AGENT_ALLOWED_UNITS | No | (any) | Comma-separated systemd unit names this agent will restart. Unset = allow any name a runbook step specifies | | FAULTLINE_AGENT_ALLOWED_CONTAINERS | No | (any) | Comma-separated container names this agent will restart. Unset = allow any |

The allowlists are a host-local second guard, independent of whatever the runbook step is configured to restart — useful if you want this specific host to only ever be able to touch specific things, regardless of what's authored on the Faultline side.

Security

• No inbound access. The agent only makes outbound HTTPS requests (long-poll for a job, POST the result). Nothing listens on this host.
• No stored SSH keys, no root requirement. Run it as an unprivileged user with narrowly-scoped passwordless sudo for exactly the systemctl/docker restart commands it needs — never as root.
• No shell. Commands run via execFile, never a shell, and the target name is defensively rejected if it starts with - (so it can never be parsed as a flag) — even though a runbook step's target name is already set by an admin on your Faultline tenant, not arbitrary user input.
• Scoped token. An agent token (fla_...) only unlocks this agent's polling endpoints — it cannot read incidents, services, or anything else in your Faultline account.
• Revoke anytime. Settings → API Keys → Remediation Agents → revoke. Already-running jobs in flight will just time out and fail; nothing else changes.

How it works

The agent is a simple loop, not a long-held connection:

1. GET /remediation/jobs/next with its token — claims the oldest pending job for this agent, if any.
2. Runs it: systemctl restart -- <unit> or docker restart -- <container>.
3. POST /remediation/jobs/:id/complete with the outcome.
4. If a job was handled, poll again immediately (there may be more queued); otherwise wait POLL_INTERVAL_SECONDS before polling again.

SIGTERM/SIGINT stop the loop after the current poll finishes — no job is left half-reported.

Uninstalling

systemctl disable --now faultline-remediation-agent
npm uninstall -g faultline-remediation-agent

Then revoke the agent in the Faultline dashboard so its token stops being valid.

License

MIT