wellspring (ws)

A macOS network diagnostic that tells you which question to ask next.

The problem it exists for is not "I cannot find the answer." It is "I do not know what to look at." So ws does not print a dashboard — it prints a ladder, and a failing rung explains everything above it.

link → address → gateway → overlay → resolver → resolution
                                   → transit → captive → app → fleet

Design and rationale: openspec/changes/wellspring-network-doctor/.

Install

cd apps/wellspring
bun install
bun run build            # -> dist/ws and dist/ws-helper

System-wide (both binaries; ws-helper must sit beside ws for install-helper to find it):

sudo install -o root -g wheel -m 0755 dist/ws /usr/local/bin/ws
sudo install -o root -g wheel -m 0755 dist/ws-helper /usr/local/bin/ws-helper

Afterwards, one command rebuilds and replaces all of it — both binaries and the SwiftBar plugin, if SwiftBar knows where its plugin folder is:

bun run reinstall                    # from apps/wellspring
bun run wellspring:reinstall         # from the repo root

Either asks for sudo once.

Then, only if you want the privileged checks and remediations:

sudo ws install-helper --commit

It prints the full plan, the exact sudoers line and the residual risk, and does nothing without --commit. See The privileged helper below before you run it.

Requires macOS. wg/wg-quick come from Homebrew (brew install wireguard-tools) and are optional — their absence is a reported blind spot, not an error. Everything else is in the base OS.

Use

ws                # live TUI: ↑↓ move · space evidence · enter remediate · r re-probe
ws report         # the static report: what to fix first, and what it explains
ws --json         # complete structured findings
ws --quick        # local reads only: no probe leaves the host, ~0.2s
ws --menubar      # SwiftBar plugin format (see Menu bar, below)
ws undo           # reverse the most recent action
ws remediate KEY  # run one finding's remediation, recording its inverse first

ws on its own opens the TUI, because that is what gets run many times a day. ws watch still works and means the same thing. Anything that selects a non-interactive tier — report, --json, --quick, --menubar — gives the static output instead, and so does any run without a terminal. That last part is what makes the default safe for scripts and agents: Ink cannot open against a pipe, so a piped ws prints the report rather than failing.

Exit code is 1 when any finding is a fault, 0 otherwise.

ws watch splits its two keys the way lazygit does: space inspects — drill into a finding's evidence and back out — and enter acts, arming the selected remediation so a second enter confirms it. Nothing mutates until enter has been pressed twice, and the screen in between names the inverse that will be recorded first. esc cancels, r re-probes, q quits.

State lives in ~/.local/state/wellspring/ (0600): first/last sighting per finding, the ack/silence/resolve store, the fleet cache, and the undo stack. Deleting it is safe. WELLSPRING_STATE_DIR moves all of it somewhere else — which is how the test suite keeps out of yours.

Every one of those files is replaced by write-to-temp-then-rename, because the menu bar's per-minute tick and an interactive run write them with no coordination. Last-write-wins is fine; a torn file is not, and it fails in the worst direction — an unparseable store reads as an empty one, silently restarting every fault's grace window.

The one thing to understand: ping is the least trustworthy signal

It is untrustworthy in both directions. It succeeds for hosts whose service is dead, and it fails for hosts that are perfectly healthy but drop ICMP. So ws never asks "is it up?" — it probes at three depths and reads the disagreement between them as the diagnosis.

| L3 ICMP | L4 connect | L7 protocol | verdict | |---|---|---|---| | ✅ | ✅ | ✅ | healthy | | ✅ | ✅ | ❌ | service misconfigured — reachable, not working | | ✅ | ❌ | — | port filtered (local pf / app firewall / remote policy) | | ❌ | ✅ | ✅ | ICMP filtered. Healthy. Not a fault. | | ❌ | ❌ | — | routing failure or host down | | ✅ | ✅ | ⚠ wrong | intercepted — everything "succeeds", nothing works |

Row 2 is why this exists: an overlay-supplied resolver that replied to ICMP, accepted TCP on port 53, and returned an empty answer for the very domain it was scoped to. Two green rungs above a dead service.

No rung may conclude reachability from the L3 result alone, enforced by src/gates.test.ts rather than by convention. That gate does not look one layer up, and it showed: the transit rung once stopped at L4 and reported "off-net transit fails everywhere" on a hotspot that merely blocks port 443 to public resolvers — while the captive and app rungs were fetching over that same path in the same run. Every reachability rung now goes to L7.

Suppression

A fault on a lower rung suppresses faults above it. The suppressor named is the lowest failing rung, not the nearest — the nearest is a symptom. Suppressed findings stay listed and name their suppressor, so the blast radius is visible; hiding them makes a diagnosis harder to trust, not easier to read.

A healthy finding above a failure is never suppressed. "Transit works while the gateway is down" is surprising and worth seeing.

posture is deliberately off-chain: the local firewall explains a filtered port but is not a step traffic passes through, and as an ancestor "the firewall is enabled" — the normal state — would suppress everything.

A fault is not actionable until it has persisted for the grace window (30s), so a Wi-Fi roam or DHCP renew is not reported as a fault.

Remediations

| glyph | meaning | |---|---| | ↯ | reversible — the way back is recorded before it runs. enter twice in ws watch, a click in the menu bar | | ✎ | a command to run yourself — no recorded way back, so neither surface will fire it |

Two glyphs and not one, because that distinction is the whole point. Every mutation records its inverse first, and if the undo record cannot be persisted the action does not run — a mutation whose way back was silently lost is worse than not acting, because you would believe there was one.

Menu bar (SwiftBar)

brew install --cask swiftbar          # once; it asks for a plugin folder
cp swiftbar/wellspring.1m.sh "$SWIFTBAR_PLUGIN_FOLDER/"
chmod +x "$SWIFTBAR_PLUGIN_FOLDER/wellspring.1m.sh"

The plugin finds an installed ws — on $PATH, or in /usr/local/bin, /opt/homebrew/bin, ~/.bun/bin — so install ws first. It never assumes a checkout, and if it cannot find one it says so in the menu rather than failing silently.

The menu bar never probes. The filename's 1m sets the scheduled interval and refreshOnOpen re-runs it before the menu is presented; every one of those runs is ws --menubar --quick, ~0.2s and no traffic.

Opening the menu used to run the full ladder. That is 5s on a healthy network and up to the whole 30s budget on a broken one — every probe runs to its timeout — and a broken network is precisely when you click the icon. So the expensive rungs are opt-in: Run the full ladder in the menu, which refreshes the menu when it finishes and keeps the icon at ● for ten minutes.

The cheap tier catches every fault this tool has actually found, because those were all configuration disagreements — a forgotten resolver, an overlay outranking it, a tunnel silently down — and its remediations fire from the menu in about a third of a second.

What the icon means

| | | |---|---| | ✗ | an unsuppressed fault | | ! | a warning, or something that could not be established | | ● | everything evaluated passed, and the full ladder ran recently | | ◐ | everything evaluated passed, but only the cheap tier has run |

● and ◐ differ in shape, not colour, because a menu bar renders light, dark or monochrome by theme. The distinction is the point: a quick tick has not looked at transit, captive, app, resolution or fleet, and an icon that implied otherwise would be asserting five rungs it never ran.

Clicking a ↯ finding runs its remediation — the inverse is recorded first, and Undo last action is in the menu. ✎ findings are not clickable; their command is shown to run yourself.

Configuration

Optional. ~/.config/wellspring/config.json; every field falls back to a default, and a bad value is reported on stderr rather than silently ignored.

{
  "celiloEndpoint": "celilo-api@celilo-mgr",
  "probeName": "example.com",
  "transitTargets": [
    { "host": "1.1.1.1", "port": 443, "url": "https://1.1.1.1/", "label": "a public resolver" },
    { "host": "example.com", "port": 443, "url": "https://example.com/", "label": "by name" }
  ],
  "captiveUrl": "http://captive.apple.com/hotspot-detect.html",
  "captiveExpect": "Success",
  "egressUrl": "https://api.ipify.org",
  "tlsDestinations": ["https://example.com"],
  "runBudgetMs": 30000,
  "graceWindowMs": 30000,
  "staleHandshakeSec": 300
}

celiloEndpoint is the only value that is genuinely about your site — the rest are opinionated defaults you may want to change, not facts about your network. Keep at least one transit target addressed by IP and one by name: an IP-only set cannot distinguish "DNS works but routing does not" from a network that simply blocks the public resolvers.

Not configurable, deliberately: the helper's binary paths. A config-controlled path in a program that runs as root would let anyone who can write the config choose what root executes. wg/wg-quick are resolved from a fixed two-entry allowlist by existence, never from input or $PATH.

The fleet rung

Reads alerts list --json over celilo's existing remote API and caches it, so a broken network still reads. It answers the discriminator nothing else can: is it this machine, the path, or the fleet?

celilo api grant wellspring --key <pubkey> --can alerts:list   # on celilo-mgr

The flow is one-directional by requirement: wellspring never writes to celilo, gated four ways in src/rungs/fleet.test.ts. An unreachable or slow celilo reports the rung as unknown with the cache age — never as a failure of the run, and never as green.

The privileged helper

Some remediations need root. ws-helper is a separate root-owned binary, because ws lives in a user-writable path and naming it in a NOPASSWD rule would hand root to anything able to write that file.

/usr/local/libexec/ws-helper   root:wheel 0755   ← not user-writable
/etc/sudoers.d/wellspring      root:wheel 0440
    <user> ALL=(root) NOPASSWD: /usr/local/libexec/ws-helper

| verb | mutates | | |---|---|---| | dns-set <service> <addr…\|Empty> | ✎ | set or clear a service's resolvers | | vpn-up <tunnel> | ✎ | wg-quick up | | vpn-down <tunnel> | ✎ | wg-quick down | | pf-show | read | packet-filter status and rules | | wg-show | read | handshake age and byte counters |

• The sudoers entry names no arguments and no wildcards — argument wildcards in sudoers are routinely bypassable, so all validation lives in the helper.
• It never invokes a shell. Verbs dispatch through execve with argument arrays; there is no string to escape and therefore no escaping bug.
• Every argument is validated against a strict pattern and against live system state — the service must appear in the current service list, the tunnel must have a config file. A pattern alone accepts a service that does not exist.
• pf is read-only. pfctl -f loads an arbitrary ruleset, which is passwordless root by another name.
• ws re-verifies the helper's ownership and mode before every privileged call. A helper that has become user-writable is a live vulnerability, not a warning, and is refused.

Residual risk, stated rather than buried: this is a permanent passwordless-root path on the machine, and its safety rests entirely on the helper's argument validation and file ownership. That is why the verb set is closed and small, the helper is shell-free, and the negative tests in src/helper/helper.test.ts are the largest test block in the project.

wg show needs root even for wg-quick's own tunnels (its control socket is root:daemon 0600), which is why wg-show exists as a read verb. Without the helper installed, handshake age is invisible and reported as a blind spot.

WireGuard: migrating from the GUI app to wg-quick

wg show returns empty with exit 0 for tunnels created by the WireGuard macOS app — they are com.wireguard.macos Network Extension configurations with no /etc/wireguard presence. Latest-handshake age is the single highest-value tunnel health signal and the one the GUI withholds: a stale-handshake tunnel presents as fully healthy — interface up, routes installed, scutil reporting Connected — while passing no traffic.

1. Export each tunnel from the app (File → Export Tunnels to Zip…). These files contain private keys.
2. Place each .conf in /etc/wireguard, owner-only. The filename is the interface name and is capped at 15 characters.

   sudo install -m 600 -o root <name>.conf /etc/wireguard/<name>.conf

3. Bring one up and confirm the signals appear, before removing anything:

   sudo wg-quick up <name>
   sudo wg show      # must show peers, `latest handshake` and `transfer`

4. Only then remove that tunnel's app profile. Repeat per tunnel.

Consequences, accepted deliberately:

• Toggling needs privilege. scutil --nc start|stop drives app-managed tunnels without sudo; wg-quick does not. That is what vpn-up/vpn-down are for.
• On-demand rules and reboot persistence are lost. The app reconnects on its own; wg-quick does not.
• You cannot change DNS by hand while a tunnel is up. wg-quick backgrounds a route monitor that re-applies the config's DNS = to every network service every ~2s, so a manual networksetup -setdnsservers reverts within seconds and appears to do nothing. Bring the tunnel down first.
• Never run two tunnels with overlapping AllowedIPs. The second gets route: File exists and installs almost none of its routes, leaving you on a tunnel that cannot carry what it claims.
• A literal Endpoint address rots. When your ISP address changes the tunnel keeps dialling the old one and dies silently — interface up, routes installed, zero traffic. ws catches it as a stale handshake. See openspec/changes/wellspring-network-doctor/WIREGUARD_ROLLOUT.md.

The macOS commands this wraps

All in the base install except wg. Documented so the knowledge survives independently of the code.

| command | what it gives | notes | |---|---|---| | scutil --dns | the effective resolver stack, with order and if_index | parsing precedence correctly is the whole game | | scutil --nc list | VPN profiles, connection state, managing extension | sees GUI tunnels wg show cannot | | netstat -rn -f inet | the route table | what a tunnel claims | | route -n get default | the default gateway | | | arp -n <gateway> | the gateway's MAC | layer 2, distinct from an IP fault | | ifconfig <iface> | carrier (status:), addresses, MTU | the flags word says UP on a dead link — read status: | | ipconfig getsummary <iface> | SSID, BSSID, DHCP-offered DNS | no sudo needed | | networksetup -listnetworkserviceorder | service name → BSD device | | | networksetup -getdnsservers <service> | the configured resolvers | the diff against the DHCP offer is the best stateless check here | | dig | rcode, latency, +tcp/+notcp, truncation | the L7 DNS probe | | nc -z | transport connect; refused vs silent | -n only for literals — it disables name resolution | | curl | HTTP status, body, TLS verification, --interface | the L7 HTTP and egress probes | | ping | ICMP echo | one input to a verdict, never a verdict | | wg show | handshake age, byte counters, peers | empty for GUI tunnels; needs root otherwise | | socketfilterfw --getglobalstate | application firewall | works unprivileged | | pfctl -si -sr | packet filter | needs root; unknown, never "no rules" |

Not present on macOS — do not depend on them: socat, nmap, mtr. airport was removed in Sonoma; wdutil info needs sudo. Use ipconfig getsummary.

Constraints

• No runtime dependency on @celilo/*. A diagnostic must not depend on the thing it diagnoses, and a broken network is exactly when bun install cannot run. Enforced by src/gates.test.ts.
• Offline is the design centre. Every rung runs and a report is produced with no network and no reachable celilo.
• Every probe is bounded, and the whole run shares a budget. A probe that does not answer is a result, never a hang.
• No literal address is canonical. Roles — gateway, configured resolver, overlay-supplied resolver — are the durable names. Fixtures use RFC 5737 documentation ranges; see test-fixtures/README.md.
• The finding shape and TUI patterns are copied from celilo, never imported.

Tests

bun test src/
bunx tsc --noEmit
bunx biome check ./src
bun run build && ./dist/ws --json > /dev/null   # the binary is the deliverable

That last line is a gate, not a formality: bun build --compile once broke while tsc, biome and every test stayed green.