@telorun/k8s-runner
v0.16.0
Published
Telo Kubernetes Runner - HTTP service that runs Telo Applications as sandboxed Kubernetes Pods. A backend over @telorun/runner-core, sibling to docker-runner.
Readme
@telorun/k8s-runner
HTTP service that runs Telo Applications as sandboxed Kubernetes Pods. A
backend over @telorun/runner-core, sibling to
docker-runner — it presents the identical /v1 session
contract (/v1/health, /v1/capabilities, /v1/probe, /v1/sessions,
/v1/sessions/:id/events SSE, /v1/sessions/:id/io WS) but spawns a Pod per
session instead of a container. On /v1/capabilities it advertises image /
pullPolicy as readOnly (server-enforced — the runner serves untrusted
code under a hard-ceiling policy), so the editor lets the user edit only the
runner URL.
⚠️ Security posture
The runner is a dumb executor: no auth, no per-IP concurrency, no rate
limiting — only a global RUNNER_MAX_SESSIONS backstop. Do not expose it
directly to anonymous internet traffic. Front it with the Telo Cloud control
plane (or an equivalent abuse-control proxy), which resolves identity → tier →
limits and forwards to the runner. Standalone, it is safe only on a
trusted/internal network.
Resource limits are hard ceilings: a request may ask for less than the
configured cap but never more (min(requested, ceiling)). For a bare runner
serving an anonymous tier, the ceiling is the policy.
How it works
Per session the runner resolves the image, creates a Pod (telo run), watches it
for status, attaches a PTY over the Pod attach subresource for the interactive
/io channel, and — when a routing base domain is configured — creates a
per-session Service plus routing objects (an Ingress, or one HTTPRoute per port)
at <port>-<sessionId>.<base-domain>, garbage-collected via an ownerReference to
the Pod.
Every session runs the plain kernel image. A body-fetch initContainer untars
the staged bundle into a writable /app emptyDir and the session container runs
telo run /app/<entry>, resolving its own module closure into a /telo-cache
emptyDir on the way up. readOnlyRootFilesystem stays on — every write lands on
a mounted emptyDir. The runner needs no image registry of its own: it builds
nothing, pushes nothing, and creates no Jobs.
The runner used to prebuild a self-contained per-app image with an on-cluster Kaniko Job, so that a slow package registry could not stall the start path. That is gone, and the cost is where it now shows: a run session downloads its closure on every start, since the cache lives and dies with the pod. A watch session pays it once and keeps the cache for the pod's life, which is the shape to reach for when start latency matters.
Coming-up progress is reported over the /v1 SSE stream as progress events
(provision → boot) while the session is still starting. The session is
created with a fast 201 carrying the streamUrl before the pod is created,
so the client connects immediately and sees provisioning live; the backend then
runs in the background and a start failure surfaces as a terminal failed status
on the stream. The session flips to running when the Pod reaches Running.
Sandbox hardening is always on (non-root, read-only rootfs, drop-all caps, no
service-account token, seccomp RuntimeDefault); a sandbox RuntimeClass
(gVisor/Kata) is layered on when configured. A controller whose install needs to
run scripts or build native code runs them inside the session pod under that
hardening — there is no trusted build step to run them in.
A Kubernetes API rejection reaches the client as its status and reason, never
as the exception. The API's error carries the full HTTP dump — the raw Status
body naming the runner's ServiceAccount, the audit id, every response header —
and a start failure's message travels verbatim to the client as the session's
terminal status. So the client is told the operation, the HTTP code and the
one-word reason (Forbidden, NotFound), plus who can fix a 403; the raw
exception rides along as the error's cause and lands in the runner's log.
Watch sessions
Everything above describes a run session: one pod per Run click, terminal when
the workload exits. POST /v1/sessions with mode: "watch" asks for something
different — a workspace that runs continuously. One pod holds a shared
/workspace volume, a workspace container serving the editor's file routes, one
app-<name> container per application running telo run --watch, and optionally
a co-resident agent drawn from the RUNNER_APPS catalog. An edit then costs a
kernel reload instead of a pod: no schedule, no pull, no module re-resolve.
Off unless RUNNER_WATCH_SESSIONS is set. Run sessions are entirely unaffected.
The pod
| Container | Image | Present | Writes | Credential |
| --- | --- | --- | --- | --- |
| workspace | The kernel image, over a runner-supplied manifest | Always | /workspace | None |
| agent | Operator catalog image | When agent is requested | /workspace | Operator env |
| app-<name> | The kernel image | One per app | /workspace, /telo-cache/<name>, scratch | Session-declared env only |
The env split is the credential boundary. It used to be structural (two pods)
and is now a code invariant: the operator env goes on agent alone, every
app-<name> gets the session's declared env and nothing else, and workspace
gets neither — it serves files and holds no secrets.
A shared fsGroup is required, and its absence is the kind of thing that
fails as a confusing "file not found" one reload after a write: every container
reads and writes /workspace, so they must share a GID or the agent writes files
the app cannot read.
The agent is routed like an app port, and reported like one. Its catalog
entry declares the port its image listens on, and the session refuses an agent
without one (400 agent_port_undeclared) rather than starting a container
nothing can reach. The pod's containers share one network namespace, so two of
them cannot bind the same port at all — but where an application declares the
agent's port the manifest wins: the session starts without the agent and says
so on its stream, rather than refusing to run the user's app over a container
they never asked for and cannot decline. Nothing else is arranged: the pod's containers
share one network namespace, so the port simply joins the session's own Service
and routing objects and answers at <agentPort>-<sessionId>.<base-domain>. The running
status carries it as an agent endpoint beside endpoints — separate, because
endpoints are the ports the user's applications declared, and an
operator-launched container is not one of them. It is reachable without auth for
as long as the session lives, which is the exposure an app session of the same
image already has.
Nothing verifies that the agent listens where port says. port tells the
runner where to route; what the image actually binds is configured separately
(the authoring agent reads PORT, defaulting to 8080). If the two disagree the
runner publishes a port and routes a host with nothing behind it, and the agent
appears to start while every request to it fails — neither backend watches the
agent's port the way both watch an application's. Keep port and any PORT in
the entry's env in agreement.
The workspace surface is runner infrastructure, not agent functionality. It is
part of the /v1 session contract, so the runner owns its manifest and its
routes; the agent is one more writer on the volume beside the app containers,
using its own filesystem tools. It runs the plain kernel image over a manifest the
runner reconciles into a content-addressed ConfigMap — there is no third image to
build, and a runner upgrade that changes the manifest leaves running sessions
mounting the one they booted with.
A watch session resolves its closure once. It lands in the workspace volume, which lives as long as the pod, so the download happens once per session and every later reload resolves from local disk — where a run session, whose cache dies with its pod, pays for it on every start.
One cache for the whole session. The runner seeds telo-workspace.yaml at
the workspace root when the session starts, and the kernel anchors its .telo
cache at the directory holding that marker — so two apps importing the same
module resolve it once between them, and an app in a subdirectory does not get a
cache of its own. The application containers therefore carry no TELO_CACHE_DIR:
that variable OUTRANKS the marker, so setting it per app is exactly what would
undo this. Only the workspace container keeps an explicit root, because its own
manifest lives outside the workspace and the walk-up would never reach the
marker. A workspace that brings its own marker keeps it — overwriting one with a
real modules: list would change what telo release discovers.
Egress lives in the session namespace. Every session resolves its own module closure, so the session namespace's NetworkPolicy has to reach the module registries as well as the model provider. Core NetworkPolicy is CIDR-only and registries sit behind rotating-IP CDNs, so a locked-down operator needs a CNI with FQDN policy or an egress proxy here.
Two nouns on one stream
Session status (status events) is starting / running / suspended /
stopped / failed. Run outcome (run events) is one per app per reload
generation:
{ "type": "run", "app": "web", "generation": 3, "phase": "started", "trigger": "watch" }
{ "type": "run", "app": "web", "generation": 3, "phase": "completed", "code": 0, "durationMs": 412 }
{ "type": "run", "app": "worker", "generation": 4, "phase": "failed", "reason": "ERR_MANIFEST_VALIDATION_FAILED" }A one-shot Runnable finishing emits run with phase: "completed" and leaves
status: running — the session is alive and the next edit starts that app's next
generation. generation is monotonic per app and starts at 1.
Those events are projected from the kernel debug stream, not parsed out of the
terminal: a watch session always runs with --inspect on, and Kernel.Starting /
Kernel.Stopped bracket each generation on one debug connection that survives
reloads. The one thing that stream did not carry is a manifest that fails to load
at all, so the CLI now emits Kernel.RunFailed ({ phase: "load" | "start", code?,
message }) — a dotted event name inside an existing frame kind, so it obliges no
other runtime.
Routes
| Route | Purpose |
| --- | --- |
| GET /v1/sessions/:id/workspace | Content-hash tree the editor diffs against its own files |
| POST /v1/sessions/:id/workspace | Apply a { write: [{path, content, encoding?}], delete: [path] } change set |
| GET /v1/sessions/:id/workspace/file?path= | One file's contents |
| POST /v1/sessions/:id/reload?app=<name> | Re-run one app with no file change (omit app for all) |
| PUT /v1/sessions/:id/apps | Change the running app set (checkpoint + pod recreate) |
| POST /v1/sessions/:id/resume | Bring a suspended session back under the same id |
A port set that changes on reload
Adding a ports: entry is as ordinary an edit as adding an import, and a
container may bind any port regardless of what the pod spec declares — so without
handling, the app listens and is simply unreachable: no ingress, no error, no
event.
The kernel re-resolves its ports: block on every load and says so on the stream
the runner is already reading (Kernel.PortsResolved), so nothing re-parses a
manifest on the reload path. The runner patches the Service and the Ingress live
and emits an endpoints event; a pod's containerPort list is documentation, so
this costs no pod recreate. A port another app in the session already declares
cannot be routed — session hosts carry no app name — and comes back on that same
event as rejected rather than being dropped.
It routes the DECLARED set, not what happened to bind. A port the manifest never
declared is not exposed, and "did anything actually bind it" is already answered
per port by the reachability watcher (checking → reachable / unreachable).
Keying on a listening event instead would rest on a per-module convention: a
transport whose kind does not emit one would silently get no routing.
Reload and the app set
reload exists because --watch reloads on change, and pressing Run again after a
one-shot app completed is not a change. It touches the named app's entry manifest
through the same path everything else uses, so it needs no signalling into the
container, no shared PID namespace and no exec — RBAC gains only configmaps:
get, create and update on services and routing objects, the latter so a reload
that changes an app's declared port set can re-patch its routing. Without that,
adding a ports: entry leaves the app bound to a port with no route, no error and
no event.
Changing the app set costs a pod recreate because a pod's container list is fixed at creation. That is the only editing action in the design that costs a pod, and it reuses suspend/resume rather than adding a second path.
Suspend and resume
A session is no longer a pod. With no SSE/WS subscriber for
RUNNER_WATCH_IDLE_SECONDS, the runner snapshots the workspace, deletes the pod
and keeps the session record — status: suspended, which is deliberately not
terminal. POST /v1/sessions/:id/resume creates a fresh pod seeded from that
checkpoint under the same session id. Aggressive reaping is what makes per-visitor
watch sessions affordable, and the checkpoint is what makes aggressive reaping
safe; they only work as a pair.
The editor holds the authoritative workspace; the checkpoint is a cache. The
runner is a single replica with an in-memory registry, so a redeploy, crash or
node move drops every suspended session and resume answers 404. That is by
design and it buys less than it looks: a durable suspended workspace is only
meaningful when there is an identity to reattach it to, and accounts are an
explicit non-goal. A watch session exists because an editor is driving it, that
editor already holds every file and already diffs its own copy against
GET /workspace, so a 404 costs one change set. Two consequences, stated rather
than discovered: a suspended session is best-effort, and a watch session with no
editor attached and unsaved agent writes is the one place work can be lost —
bounded by RUNNER_WORKSPACE_CHECKPOINT_SECONDS.
Capacity changes shape. Concurrency becomes bounded by simultaneous editors rather than simultaneous runs; the run-session ceilings were sized for the opposite assumption, which is why watch has its own.
io — terminal or separated streams
Each app declares io: "tty" (default) or io: "streams". The difference is
observable to the application, not just to the client: isatty() drives
colour, line-versus-block buffering, progress bars and prompts, so a loop that is
always a PTY systematically hides how the app behaves in production.
| | tty | streams |
| --- | --- | --- |
| Output | One merged stream, as a terminal produces | Separated at the source |
| /io resize | Yes | Rejected — meaningless without a PTY |
| CLICOLOR_FORCE | Injected | Not injected |
Nothing is invented at the transport layer: the Pod attach subresource without a
TTY already gives separate stdout and stderr channels. The TTY is what collapses
it. streams forces nothing off either — with no terminal the colour precedence
already resolves to no colour, and an explicit NO_COLOR would sit above an app's
own color: always and suppress a decision worth observing.
GET /v1/sessions/:id/io?app=<name> attaches to one app's terminal; ?app= is
required whenever the session runs more than one, since there is no defensible
default among several. Every binary frame is [seq:4 BE][stream:1][payload].
Ports are unique across the whole session
Session hosts are <port>-<sessionId>.<base-domain>, a single label, so two apps
both listening on 3000 would collide with nothing to distinguish them. That is a
400 port_conflict at session create rather than an app name added to the host
scheme: the user controls both manifests, and a rejected request is a better
outcome than a URL that silently reaches the wrong app.
Configuration (env)
| Env | Default | Purpose |
| --- | --- | --- |
| RUNNER_SELF_URL | (required) | Runner's in-cluster base URL (bundle fetch) |
| PORT | 8062 | HTTP listen port |
| RUNNER_DISPLAY_NAME | Telo Runner | Display name advertised on /v1/capabilities (the editor's runner label) |
| RUNNER_DESCRIPTION | Runs the Telo application in a cloud environment | Description advertised on /v1/capabilities |
| RUNNER_APPS | (unset → no apps) | JSON map of operator-predefined apps launchable by name (chart: inline apps.catalog, or apps.catalogSecret referencing a Secret holding the JSON — use the Secret whenever entries embed secrets in env); see the docker-runner README for the entry shape, including the port an entry must declare to be usable as a session's co-resident agent. App sessions run the catalog image directly as a pod |
| RUNNER_APP_MAX_CPU | 500m | CPU ceiling for predefined-app pods (separate from the anonymous-session ceiling) |
| RUNNER_APP_MAX_MEMORY | 512Mi | Memory ceiling for predefined-app pods |
| RUNNER_APP_MAX_TTL_SECONDS | 21600 | Wall-clock TTL for predefined-app pods (agent sessions are long-lived) |
| RUNNER_APP_MAX_EPHEMERAL_STORAGE | 1Gi | Ephemeral-storage ceiling for predefined-app pods |
| RUNNER_SESSION_NAMESPACE | telo-sessions | Namespace for session objects |
| RUNNER_IMAGE | (baked at build: the CLI version for a released runner, telorun/node:latest-slim for a dev build) | Default base image; always offered in the picker and the fallback when the catalog is unreachable. Leave the chart's session.image empty to keep the session kernel in lockstep with the runner |
| RUNNER_INIT_IMAGE | busybox:stable | Bundle-fetch initContainer image (wget + tar) |
| RUNNER_IMAGE_PULL_SECRET | (unset) | dockerconfig Secret (in telo-sessions) the kubelet pulls session images with — needed only for a kernel or catalog image in a private registry |
| RUNNER_RUNTIME_CLASS | (unset → runc) | Sandbox RuntimeClass (gvisor/kata) |
| SESSION_ROUTING_MODE | auto | auto | ingress | gateway | none. See Routing layer |
| SESSION_ROUTING_BASE_DOMAIN | (unset → logs-only) | Wildcard base for per-session hosts |
| SESSION_ROUTE_READY_TIMEOUT_SECONDS | 60 | How long a published route may go unclaimed before the session reports it unprogrammed |
| SESSION_INGRESS_CLASS | (unset → cluster default) | IngressClass for session Ingresses (ingress mode) |
| SESSION_INGRESS_TLS_SECRET | (unset → no TLS block) | kubernetes.io/tls Secret (in telo-sessions) the session Ingress presents; must cover *.<base-domain>. Set for Cloudflare Full (Strict) / any origin-cert upstream. Ingress mode only — refused at boot in gateway mode, where the cert belongs to the Gateway listener |
| SESSION_GATEWAY_NAME | (unset) | Gateway each session's HTTPRoute attaches to. Required in gateway mode |
| SESSION_GATEWAY_NAMESPACE | (= session namespace) | Namespace of that Gateway |
| SESSION_GATEWAY_SECTION_NAME | (unset → any listener) | Listener name, when only one should carry session traffic |
| RUNNER_MAX_CPU | 50m | CPU ceiling |
| RUNNER_MAX_MEMORY | 100Mi | Memory ceiling |
| RUNNER_MAX_TTL_SECONDS | 3600 | Wall-clock TTL (Pod activeDeadlineSeconds) |
| RUNNER_MAX_EPHEMERAL_STORAGE | 512Mi | Per-Pod ephemeral-storage ceiling |
| RUNNER_MAX_SESSIONS | 32 | Global session backstop; at capacity the oldest exited session is evicted before a new run is rejected |
| RUNNER_WATCH_SESSIONS | false | Server-side gate. Watch sessions are never client-requestable when off |
| RUNNER_WATCH_IDLE_SECONDS | 300 | No SSE/WS subscriber for this long → suspend |
| RUNNER_WATCH_MAX_TTL_SECONDS | 21600 | Pod deadline for a watch session. One deadline covers the agent and the app containers, so it takes the longer ceiling and lets idleness do the real work |
| RUNNER_WATCH_MAX_SESSIONS | 8 | Concurrency ceiling for watch sessions, separate from RUNNER_MAX_SESSIONS |
| RUNNER_WATCH_RELOAD_LIMIT | 30 | Per-session reloads per minute |
| RUNNER_WATCH_SUSPENDED_TTL_SECONDS | 86400 | How long a suspended session record is retained before eviction. Deliberately not the pod deadline: that bounds a pod, so on its own nothing would ever evict a suspended record |
| RUNNER_WORKSPACE_CHECKPOINT_SECONDS | 30 | How often the runner pulls a whole-tree workspace snapshot |
| RUNNER_EXIT_TTL_MS | 14400000 | How long exited sessions stay in the registry (so the editor can re-attach and replay their history after a reload) before eviction |
| RUNNER_TERMS_FILE | (unset) | Path to the agreement file (plain text / markdown), read at startup — mount it from a ConfigMap (e.g. /etc/telo/terms.md). Setting this (or RUNNER_TERMS_BODY) enables terms: the runner advertises them on /v1/capabilities and rejects POST /v1/sessions with 428 unless the client sends x-telo-accepted-terms matching the version. An unreadable path fails startup. The public cloud should set this |
| RUNNER_TERMS_BODY | (unset) | Inline agreement text, for short notes; ignored when RUNNER_TERMS_FILE is set |
| RUNNER_TERMS_TITLE | Usage agreement | Heading shown above the agreement |
| RUNNER_TERMS_VERSION | (hash of body) | Acceptance version; defaults to a content hash so any edit to the body automatically re-prompts every client. Set explicitly only to control material-change vs typo |
Base-image picker
The runner advertises a menu of base images on /v1/capabilities, resolved from
a Docker Hub repo's tags (filtered) and cached. The editor renders it as an
editable image dropdown; the chosen image is re-validated server-side
against the same list, so a client that skips the editor can't widen the set.
RUNNER_IMAGE is always offered and is the fallback when Docker Hub is
unreachable. Disable the catalog to lock image to RUNNER_IMAGE.
Pinned tags (e.g. 0.30.1-slim) are immutable. A picked moving tag like
latest-slim only refreshes when the session's pullPolicy is always, which
is the Pod's own imagePullPolicy: the kubelet re-pulls the tag on every
session. missing and never reuse whatever that node already cached.
| Env | Default | Purpose |
| --- | --- | --- |
| RUNNER_BASE_IMAGE_CATALOG_ENABLED | true | Resolve + advertise the picker; false locks image to RUNNER_IMAGE |
| RUNNER_BASE_IMAGE_REPO | telorun/node | namespace/repository queried on Docker Hub |
| RUNNER_BASE_IMAGE_PINNED_ONLY | true | Keep only pinned MAJOR.MINOR.PATCH[-variant] tags (drops latest, 0, 0.30) |
| RUNNER_BASE_IMAGE_EXCLUDE_SHA | true | Drop commit-hash tags |
| RUNNER_BASE_IMAGE_EXCLUDE_PRERELEASE | true | Drop semver prereleases (-rc.1, -alpha); -slim / -rust-* variants are kept |
| RUNNER_BASE_IMAGE_INCLUDE | (unset) | Regex a tag must match (escape hatch) |
| RUNNER_BASE_IMAGE_EXCLUDE | (unset) | Regex that drops a matching tag (escape hatch) |
| RUNNER_BASE_IMAGE_LIMIT | 20 | Cap on advertised tags (newest first) |
| RUNNER_BASE_IMAGE_REFRESH_SECONDS | 3600 | Catalog re-fetch cadence |
Deploy (Helm)
The chart is published as an OCI artifact at
oci://ghcr.io/telorun/charts/k8s-runner:
helm install telo-runner oci://ghcr.io/telorun/charts/k8s-runner \
--set watch.enabled=true \
--set session.runtimeClass=gvisorHelm resolves the newest release when no --version is given; pass one to pin.
From a checkout, ./chart works the same way and is what you want when changing
the chart itself.
The chart's version IS the runner's. version and appVersion both carry
@telorun/k8s-runner's package version, stamped in the release PR and checked on
every PR — so chart 0.13.0 installs runner image 0.13.0, with no lookup
table. That is also why image.tag defaults to empty and resolves to the chart's
own appVersion rather than to latest: --version 0.13.0 has to mean one
thing, where a floating tag meant whatever it had moved to overnight.
The chart provisions the static scaffolding: the runner Deployment (single
replica — the registry is in-memory and the runner reaps orphaned pods on boot),
Service, scoped RBAC, the telo-runner and restricted-PSS telo-sessions
namespaces, a ResourceQuota, and the session pod-to-pod NetworkPolicy. The
runner creates per-session objects at runtime. Nothing else is required — no
image registry, no build namespace, no push credentials.
For a kernel image or an operator catalog image held in a private registry,
create the dockerconfigjson Secret in telo-sessions yourself and point
session.imagePullSecret at it; the kubelet pulls with it.
Routing layer
Session hosts are <port>-<sessionId>.<base-domain> — a single label, so one
wildcard DNS record and one wildcard cert cover every session. Both current
routing APIs are supported, because clusters ship Ingress, Gateway API, or
both, and hardcoding either leaves the runner unroutable on the rest.
sessionRouting.mode picks the layer:
| mode | behaviour |
| --- | --- |
| auto (default) | Resolve from what is configured, then from what the cluster serves |
| ingress | networking.k8s.io/v1 Ingress |
| gateway | gateway.networking.k8s.io HTTPRoute (v1, falling back to v1beta1) |
| none | Publish nothing — logs-only even with a base domain set |
auto reads configuration before installation, and that order is the point: a
named Gateway wins, then a configured IngressClass, then a default-marked
IngressClass (ingressclass.kubernetes.io/is-default-class — the cluster has
nominated where an unqualified Ingress goes, and unqualified is what this runner
creates), and only then does it look at what merely exists. Gateway API CRDs are frequently present without being the intended
path, so "the API exists" is not evidence of intent. Where two usable layers
exist and nothing was configured, the runner refuses to start rather than
guessing — a wrong guess publishes routing objects nothing reconciles, which
produces a healthy pod and a 404 on every URL, with nothing reported anywhere.
# Gateway API
helm install telo-runner ./chart \
--set sessionRouting.mode=gateway \
--set sessionRouting.baseDomain=telo.run \
--set sessionRouting.gateway.name=public \
--set sessionRouting.gateway.namespace=gateway-system \
--set sessionRouting.dataPlaneNamespace=gateway-systemThe Gateway needs a listener whose hostname covers *.<base-domain> and whose
allowedRoutes.namespaces admits telo-sessions. A route it does not admit is
reported per session with the controller's own reason (NotAllowedByListeners).
Upgrading from sessionIngress.* is a breaking values change and the chart
refuses the old keys rather than ignoring them — Helm drops unknown values
silently, so an un-migrated file would otherwise boot a runner that logs
session routing disabled (logs-only) and drops every session URL.
One HTTPRoute per port. Gateway API scopes
hostnamesto the whole route while a rule matches on path, never host — so a single route carrying every session hostname would send them all to whichever rule matched/first. An Ingress rule owns its host, which is why that layer needs only one object.
sessionRouting.dataPlaneNamespace is not optional when routing is on. The
session NetworkPolicy admits traffic from that namespace only; without it the
route programs correctly and the workload is still unreachable.
Route verification
The runner already dials each declared port from its own network — that proves
the app is listening and says nothing about whether traffic can reach it.
So it separately verifies that what it published was actually programmed, and
reports each host as pending → programmed / unprogrammed on the session
event stream (type: "route"). Reported, never fatal: a slow controller and an
absent one look alike for the first few seconds.
Gateway API answers this precisely, from the Accepted / ResolvedRefs
conditions a controller writes. Ingress has no rejection signal — a controller
that refuses a rule simply never writes status — so there an unclaimed route is
caught by the timeout rather than by a reason the cluster gave.
Origin TLS (Cloudflare et al.)
Ingress mode only. Under Gateway API the origin certificate belongs to the
Gateway listener's own certificateRefs; setting SESSION_INGRESS_TLS_SECRET
alongside mode: gateway is refused at boot rather than silently ignored.
To have the per-session Ingress present an origin cert (so an upstream like
Cloudflare in Full (Strict) mode validates the origin), give the chart a
kubernetes.io/tls Secret in telo-sessions. The cert must cover the wildcard
*.<sessionRouting.baseDomain> — session hosts are a single label
(<port>-<sessionId>.<base-domain>). Two ways:
# A — reference a Secret you manage in telo-sessions (cert-manager, your own sync)
helm install telo-runner ./chart --set sessionRouting.ingress.tls.secretName=telo-origin-tls
# B — let the chart create the Secret from your cert + key
helm install telo-runner ./chart \
--set-file sessionRouting.ingress.tls.cert=origin.pem \
--set-file sessionRouting.ingress.tls.key=origin.keyEither wires SESSION_INGRESS_TLS_SECRET, and the runner stamps a spec.tls
block on every session Ingress. Leave all three empty to skip TLS at the origin
(terminated entirely upstream).
Egress note. Core NetworkPolicy is CIDR-only and cannot express the package-registry FQDN allowlist the session namespace needs — use a CNI with FQDN policy (Cilium) or an egress proxy to tighten it.
Development
pnpm --filter @telorun/k8s-runner build # tsc
pnpm --filter @telorun/k8s-runner test # vitest (pod specs, limits clamp, tar, API errors)The Kubernetes backend can't be exercised without a cluster; unit tests cover the backend-independent logic (the pod specs, limit clamping, the tar writer, and what an API rejection is allowed to tell a client).
