open-sites-runner

The container runner for an open-sites hub. It turns a site version that ships a Dockerfile into a running, sandboxed container and proxies the hub's requests to it — so a hub can host apps in any language or framework (Python, Go, Rails, a full Next.js app, …), not just static files and JS server functions.

The runner is a small Node service that talks to a Docker daemon through its socket (or DOCKER_HOST) and exposes an HTTP API protected by a shared token. The hub never talks to Docker itself; every request for a container app goes hub → runner → container. That works next to a Docker-hosted hub (compose) or on a separate VPS while the hub stays on Vercel.

What it does

• Builds images from the uploaded tar context with the daemon's classic builder (POST /build), streams the build log back to the hub as events, tags the result os-site/<siteId>:v<n> and labels it (open-sites.site, open-sites.version, open-sites.kind=image).
• Runs one container per app key (os-<key>-<generation>) on its own bridge network os-app-<key>, with memory / CPU / pids limits, all capabilities dropped (except CHOWN SETUID SETGID NET_BIND_SERVICE), no-new-privileges, a 64 MB tmpfs /tmp, log rotation and no restart policy. The app listens on $PORT (always 8080).
• Swaps containers without downtime when the spec (image id, env, limits, idle policy) changes: start the new one, wait until its port answers, switch the target, then stop and remove the old one. If the new container fails to start, the old one keeps serving and the runner reports start_failed with the log tail.
• Proxies ANY /v1/apps/:key/http/* to the container, streaming both directions (SSE, uploads), stripping hop-by-hop headers and forwarding everything else (x-opensites-*, x-forwarded-*, host).
• Cold-starts stopped apps on the next request and stops idle ones (idleStopS), enforces RUNNER_MAX_APPS, watches container exits (exited / OOM-killed) and forwards stdout/stderr lines (rate-limited) to the hub.
• Adopts containers it started before a restart (state is rebuilt from labels), so restarting or upgrading the runner is invisible to visitors.

Run

The npm package and the ghcr.io/jans25/open-sites-runner image are published by the release workflow from 0.7.0 on; to build the image from this repository, see below.

Docker image, next to a Docker-hosted hub (compose):

services:
  runner:
    image: ghcr.io/jans25/open-sites-runner:latest
    restart: unless-stopped
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
    environment:
      RUNNER_TOKEN: ${RUNNER_TOKEN}          # same value as the hub's RUNNER_TOKEN
      HUB_URL: http://web:3000               # where build/app/log events are pushed
      RUNNER_SHARED_NETWORK: ${COMPOSE_PROJECT_NAME}_default   # optional: lets apps resolve `db`
    # no `ports:` needed — the hub reaches it on the compose network as http://runner:7070

Build the image yourself from this repository:

pnpm --filter open-sites-runner build && docker build -t open-sites-runner packages/runner

Bare process on a VPS (hub on Vercel, runner on a box you own with Docker installed):

RUNNER_TOKEN=$(openssl rand -hex 32) HUB_URL=https://my-hub.vercel.app npx open-sites-runner

Put it behind TLS (Caddy, nginx, a Cloudflare tunnel, …) and point the hub's RUNNER_URL at it. The runner listens on 0.0.0.0:7070 by default; when a reverse proxy sits in front, bind to 127.0.0.1 with RUNNER_BIND.

open-sites-runner --version prints the version; --help the environment summary. SIGTERM/SIGINT stop the runner cleanly — app containers keep running and are adopted by the next start.

Environment

| Variable | Default | Meaning | |---|---|---| | RUNNER_TOKEN | — (required, ≥ 16 chars) | Shared bearer secret. Every API call must carry Authorization: Bearer <token>; the runner sends the same header when pushing events to the hub. | | HUB_URL | — | Hub base URL. Events are POSTed to <HUB_URL>/api/v1/internal/runner/events. Without it events stay in memory (a warning is logged once) — fine for local experiments, useless for a real hub. | | RUNNER_PORT | 7070 | Listen port. | | RUNNER_BIND | 0.0.0.0 | Listen address. | | DOCKER_SOCKET | /var/run/docker.sock | Docker socket path. | | DOCKER_HOST | — | tcp://host:port or unix:///path instead of DOCKER_SOCKET. | | RUNNER_NETWORK_MODE | auto | attach / publish / auto (see below). | | RUNNER_INTERNAL_NETWORKS | 0 | 1 = create app networks with --internal (apps get no egress). | | RUNNER_SHARED_NETWORK | — | Extra network every app joins (e.g. compose's default network so db resolves). | | RUNNER_BUILD_CONCURRENCY | 1 | Parallel image builds. | | RUNNER_MAX_APPS | 20 | Maximum running containers; further PUT /v1/apps/:key → 503 too_many_apps. | | RUNNER_LABEL_PREFIX | open-sites | Prefix of the labels on images, containers and networks. | | RUNNER_DEFAULT_START_TIMEOUT_S | 60 | Used when a spec omits startTimeoutS. | | RUNNER_DEFAULT_IDLE_STOP_S | 0 | Used when a spec omits idleStopS (0 = never stop idle apps). |

Network modes

• attach — the runner itself runs in Docker. On boot it finds its own container (/.dockerenv + HOSTNAME), joins each app's network and proxies to the container's IP on port 8080. Nothing is published on the host.
• publish — the runner is a bare process (npx open-sites-runner, CI). Each app publishes port 8080 to 127.0.0.1:<ephemeral>; the runner reads the port back from the container and proxies to it. Because docker-proxy accepts TCP connections before the app listens, readiness in this mode means "the connection stays open for 400 ms".
• auto (default) — attach when self-detection succeeds, otherwise publish.

Security notes

• The Docker socket is root on the host. Whoever controls the runner (or the socket) controls the machine. Run it on a host you own, never expose the socket to anything else, and keep the API off the public internet unless it sits behind TLS with the token — RUNNER_TOKEN is the only thing between the internet and docker run.
• The runner process runs as root inside its image for the same reason (the socket is root/docker-group only); the image contains nothing but the bundled runner.
• App containers are constrained: memory (MemorySwap = Memory, no swap), CPU, pids, CapDrop ALL + a minimal CapAdd, no-new-privileges, private per-app network (optionally --internal), rotating json-file logs (5 MB × 2), no restart policy, host.docker.internal resolves to the host gateway. They are not a hard security boundary against a hostile image — the daemon's own isolation applies; treat the runner as multi-tenant only among people you trust with docker run.
• Secrets reach an app only as environment variables in the container spec; they are not written to labels (only the env names are, so adoption can tell app env from image env).
• Builds use the daemon's classic builder (/build without a BuildKit session): RUN --mount=…, heredocs and other BuildKit-only Dockerfile features are unsupported; multi-stage builds work.

API

Bearer auth on every route except GET /v1/health. Errors are JSON {error, code} and also carry the code in an x-runner-code response header (plus x-runner-state: starting for the cold-start case), so a hub can tell runner answers from app answers on the proxy path without reading the body. App keys match /^[a-z0-9_-]{1,80}$/ (the hub uses <siteId> for the deployed app and <siteId>--v<n> for previews).

| Route | Purpose | |---|---| | GET /v1/health | {ok, version, docker:{version, apiVersion}, networkMode, apps:{running,total,max}, builds:{queued,running}} — 503 when Docker is unreachable | | POST /v1/builds/:siteId/:version | body application/x-tar (≤ 32 MiB, must contain a root Dockerfile), headers x-os-timeout-s, x-os-memory-mb → 202 {state, seq}; 400 bad_context | | GET /v1/builds/:siteId/:version | {state: queued\|building\|ready\|failed\|unknown, seq, logTail, error?, imageId?, imageBytes?, startedAt?, finishedAt?} | | DELETE /v1/builds/:siteId/:version | cancels an in-flight build and force-removes the image | | GET /v1/images | [{siteId, version, imageId, bytes}] for every labelled image | | PUT /v1/apps/:key | JSON spec {siteId, versionNumber, image, imageId, env, limits:{memoryMb,cpus,pids}, idleStopS, startTimeoutS} → 200 {state:"running", containerId, specHash, changed} (changed: false = spec hash matched a running app, nothing done); 409 no_image, 503 too_many_apps, 502 start_failed {error, logTail}, 400 invalid_spec | | GET /v1/apps/:key | {key, siteId, version, state: running\|starting\|stopped\|failed, containerId, startedAt, generation, lastHit, specHash, exitCode?, oomKilled?, error?, swapping}; 404 unknown_app | | POST /v1/apps/:key/restart | rolling restart with the stored spec | | POST /v1/apps/:key/stop | stop + remove the container, keep the spec (next request cold-starts) | | DELETE /v1/apps/:key | container + network | | DELETE /v1/sites/:siteId | every app, network and image of a site → {ok, apps, images} | | GET /v1/apps | list of app statuses | | ANY /v1/apps/:key/http/* | proxy; header x-os-timeout-ms (default 25000) bounds the cold start and the upstream headers/body timeouts; 404 unknown_app, 502 start_failed, 503 starting + x-runner-state: starting, 502 upstream_error, 504 timeout |

Events pushed to the hub (POST <HUB_URL>/api/v1/internal/runner/events, JSON {events: [...]}, batches of ≤ 200 or every second, retried with backoff, log lines dropped first when the queue overflows):

{type:"build", siteId, version, seq, state, error?, imageBytes?, imageId?, ts}
{type:"log",   siteId, version, appKey, seq, stream: stdout|stderr|build, message, ts}   // appKey = "build:<siteId>:v<n>" for build output
{type:"app",   key, siteId, version, seq, state: running|exited|failed|stopped, exitCode?, oomKilled?, error?, ts}

seq is monotonic per source (per build, per app, per log stream) so the hub can ignore replays.

Development

pnpm --filter open-sites-runner typecheck
pnpm --filter open-sites-runner build          # dist/open-sites-runner.cjs
pnpm --filter open-sites-runner test           # unit tests; the Docker suite runs when /var/run/docker.sock exists (or TEST_DOCKER=1)

The Docker suite builds a tiny node:22-alpine echo app, exercises build → ensure → proxy (headers, POST body, SSE) → idle stop / cold start → swap → failed swap → restart → exit watcher → adoption → removal, and cleans up after itself.