kuber

kuber is a Docker Compose to Kubernetes translation layer backed by a self-hosted management service (kuber-server). It reads a local Compose file, renders Kubernetes resources, and applies them to the cluster through an authenticated v2 API. Image builds happen inside the cluster with rootless BuildKit, so the workstation needs neither Docker nor kubectl.

Architecture

workstation (kuber CLI)  ──HTTPS──▶  kuber-server (in-cluster pod)
                                        │
                                        ├─ CAS (content-addressed blobs on RWX PVC)
                                        ├─ BuildKit Jobs (rootless) ──▶ registry
                                        └─ Kubernetes API (RBAC-scoped)

The CLI authenticates to the server with a Bearer token acquired by kuber login. It snapshots the repository into a content-addressed workspace, uploads only the blobs the server is missing, and submits a build request. The server materializes that workspace onto a shared RWX PVC and runs a rootless BuildKit Job that builds and pushes the image, then the CLI pins the resulting immutable registry digest into the rendered Deployment.

The server also owns reconciliation: it plans, applies, and prunes resources in a per-project namespace, reconciles managed Postgres and S3 claims, rolls deployments back, streams logs, and exposes interactive exec sessions over a WebSocket.

Directory Trust

Before kuber up, run kuber trust from the configured project directory. Trust is exactly the configured namespace plus a SHA-256 fingerprint of the resolved current working directory. The local mode-0600 store lets up fail before builds from an untrusted directory. The server stores only namespace and fingerprint registrations in labelled ConfigMaps in its kuber-system control plane namespace, then checks the pair when resource reconciliation begins.

This is an accidental-targeting safeguard, not a security boundary: a client that intentionally forges a registered fingerprint can pass it. kuber trust status shows local/server awareness without printing paths; kuber trust revoke removes the current directory registration.

Environment Assumptions

kuber targets a specific self-hosted cluster and workstation setup. It is not intended to run unchanged against an arbitrary Kubernetes environment. The expected setup:

• an account on the kuber-server management API
• a working ClusterRole/Role (see Server Deployment)
• a registry the in-cluster BuildKit can push to

Not required locally:

• Docker
• kubectl

API Origin

The v2 management API has one hard-coded origin:

https://kuber.astrxl.dev/api/v2

Authentication

kuber login dmgnr
kuber login dmgnr --persist
kuber whoami
kuber logout

The default login is stored with mode 0600 under $XDG_RUNTIME_DIR/kuber/session.json and disappears with the user runtime directory. --persist instead uses $XDG_CONFIG_HOME/kuber/session.json, or ~/.config/kuber/session.json when XDG_CONFIG_HOME is unset. Password input is never echoed. Runtime sessions last 24 hours and persistent sessions last 30 days; logout revokes the server-side session. readSession falls back to the persistent file when no runtime session exists.

The default login is scoped to the current user and the authenticated identity is available to every v2 command. login, logout, whoami, and global maintenance are the only commands that run without a loaded project configuration.

Roles and Authorization

The server grants capabilities through three roles:

• viewer — read-only cluster access (kubernetes:read)
• operator — viewer plus kubernetes:write and kubernetes:exec
• admin — all capabilities, including user administration (users:read, users:write, sessions:revoke, platform:adopt)

Administer users with the users command tree:

kuber users ls
kuber users add dmgnr --roles admin
kuber users update dmgnr --roles operator
kuber users update dmgnr --password
kuber users disable dmgnr
kuber users enable dmgnr
kuber users delete dmgnr
kuber users revoke dmgnr

users add, update --password, and delete prompt for password / written confirmation on an interactive terminal. Passwords are hashed with Argon2id on the server and never stored in plaintext. Updating a user's roles or password revokes all of that user's active sessions.

Inspect server-side operations and the audit trail:

kuber operations ls
kuber operations get <operation-id>
kuber audit ls

Server Deployment

The repository's compose.yml owns the kuber-system namespace, the kuber-server image, its Deployment, Service, and Ingress. .kuberrc.ts extends the rendered manifests with the server's ServiceAccount and RBAC:

• a namespaced Role/RoleBinding (kuber-server-auth) for the kuber-system Secrets, ConfigMaps, Pods, Jobs, and Leases the server itself reads and writes
• a ClusterRole/ClusterRoleBinding (kuber-server-manager) granting the cross-namespace verbs it needs to manage user projects

compose.yml runs the server as a non-root user (runAsUser/runAsGroup 1000), drops all Linux capabilities, uses a read-only root filesystem with a RuntimeDefault seccomp profile, and backs /data with a Longhorn PVC (kuber-build-data). The PVC hosts the CAS, materialized workspaces, and resumable upload bytes, and is shared with BuildKit Jobs.

Deploy the server with kuber itself:

KUBER_BOOTSTRAP_PASSWORD='replace-me' kuber up

The server creates an admin user (from KUBER_BOOTSTRAP_USERNAME, default dmgnr) on first boot only if that user does not already exist. The bootstrap Secret is only rendered while KUBER_BOOTSTRAP_PASSWORD is set. After logging in successfully, reconcile without that variable and restart once so kuber removes the stale bootstrap Secret and the password leaves the pod environment:

kuber up --no-b
kuber restart kuber-server

See Account Recovery for what to do if you are locked out.

Security Constraints

Because the server's ServiceAccount is scoped by the RBAC in .kuberrc.ts, it can only act on the resources kuber manages. The management layer additionally enforces ownership, so the server refuses to mutate resources that do not carry kuber's workspace labels. Deleting a resource requires its UID as an optimistic concurrency precondition, and workspace deletion also requires the namespace UID. Namespaces that are not owned by kuber, or owned by a different workspace, are never mutated (see Workspaces and Migration).

Workspaces and Migration

Each project maps to a Kubernetes namespace derived from the current working directory. The CLI records a "workspace" on the server keyed by project name and borrows the namespace UID to guarantee it owns the namespace before reconciling. Adopting existing resources relabels them (Server-Side Apply) only when they are already managed by kuber and not owned by another workspace.

up refuses to mutate a namespace when:

• the namespace already exists but does not carry kuber's managed-by label (external), or
• the namespace is labeled for a different workspace UID (different-workspace)

In those cases the CLI prints a hint to POST /workspaces/<project>/adopt with the namespace UID to complete a safe, explicit adoption. The platform namespace (kuber-system) is adopted through the admin-only platform adoption route.

Workspace state, revisions, operations, and audit events are stored as Secrets and ConfigMaps in kuber-system keyed by kuber's kuber.astrxl.dev/type label. Workspace updates are optimistic (If-Match on resource version) and immutable revisions are recorded so history survives. Expired sessions are cleaned up on an interval, and stale operations are marked failed on server startup recovery.

Migration Behavior

kuber up is idempotent and safe to re-run. Each run:

1. snapshots the workspace and uploads missing blobs to the server CAS,
2. ensures the workspace record (creating or updating it with an optimistic If-Match),
3. adopts the namespace and its kuber-managed resources,
4. reconciles managed Postgres and S3 claims,
5. renders manifests, plans the diff, applies desired resources, waits for rollout, and deletes stale resources.

Because resource references are immutable digests, re-running up only restarts deployments whose image content actually changed. start re-resolves published digests without building.

Registry Authentication

Building and pushing images from inside the cluster typically requires credentials for the target registry. These are read from a Docker config file (KUBER_REGISTRY_CONFIG, default /etc/kuber/registry/config.json) and mounted as an image pull secret named by KUBER_REGISTRY_SECRET.

Registry authentication is optional. If KUBER_REGISTRY_SECRET is unset, the server warns at startup and BuildKit uses anonymous registry access. This is intended for registries that allow anonymous push/pull. The same credentials are used when the server resolves a published image digest (start / export).

The server supports both standard registry bearer-token (WWW-Authenticate: Bearer) and pre-emptive Basic auth when resolving digests.

Build Registry Environment

The server's registry behavior is driven by a few closely related environment variables:

• KUBER_BUILD_REGISTRY (default registry.neko-piranha.ts.net): the registry the in-cluster BuildKit pushes built images to and kuber uses as the image namespace. It also supplies the registry host for authentication.
• KUBER_INTERNAL_REGISTRY_HOST: the host of an internal registry (for example the in-cluster distribution service) used for the BuildKit cache image and, when set, the image direct push target. When unset, push and cache fall back to KUBER_BUILD_REGISTRY.
• KUBER_INTERNAL_REGISTRY_INSECURE: set to "true" to push to the internal registry over plain HTTP instead of HTTPS. Only meaningful when KUBER_INTERNAL_REGISTRY_HOST is set.
• KUBER_PUSH_IMAGE_PREFIX (default kuber/): a prefix applied to project images pushed to the internal registry when KUBER_INTERNAL_REGISTRY_HOST is configured.
• KUBER_REGISTRY_RESOLVE_ORIGIN: an explicit origin used to resolve a published image digest (used by start / export). Useful when the digest must be resolved from a different endpoint than the build/push registry, such as an internal HTTP registry.
• KUBER_REGISTRY_CONFIG: path to the Docker config file with registry credentials (see above).
• KUBER_REGISTRY_SECRET: the image pull Secret mounted for BuildKit's registry access.
• KUBER_BUILDKIT_IMAGE: override the BuildKit runner image used for builds.
• KUBER_BUILD_DATA_CLAIM: the PVC claim backing builds.
• KUBER_BUILD_RECONCILE_MS (default 30000): the background build reconciliation interval in milliseconds. Values must be greater than zero and no greater than 2147483647 (the JavaScript timer maximum); invalid values use the default.
• KUBER_BUILD_RECONCILE_TIMEOUT_MS (default 20000, maximum 25000): the deadline for one background build-reconciliation scan in milliseconds. The maximum leaves time within the 30-second reconciliation lease for normal renewal or release. The reconciler renews both its workspace and per-build leases every 10 seconds for the lifetime of a generation, including while an observation, log read, or store call is pending. On timeout its abort signal is cancelled and logged, but Kubernetes requests may be unabortable; the generation remains active and continues its lease heartbeat until that request settles, so a later scan cannot overlap it.

See Server Deployment for how compose.yml wires the internal registry variables for the kuber-server pod.

Account Recovery

If you lose your credentials and cannot log in:

1. Recreate the bootstrap admin by deploying with KUBER_BOOTSTRAP_PASSWORD set again:

   KUBER_BOOTSTRAP_PASSWORD='new-password' kuber up --no-b
   kuber restart kuber-server

2. The server only creates the bootstrap user if the account does not already exist, so a fresh kuber login <username> with the new password works, or use the newly created admin to reset other accounts:

   kuber users update <username> --password

3. After recovering, reconcile without KUBER_BOOTSTRAP_PASSWORD and restart so the bootstrap Secret is removed and the password leaves the pod environment.

Because user records and session hashes are stored as Secrets in kuber-system, recovery relies on cluster administrators being able to redeploy the server with bootstrap credentials. users revoke <username> forcibly logs a user out across all devices.

Next.js Example

example/ contains a documented deployment template for adding kuber to an existing Bun-powered Next.js project without initializing or bundling an application in this repository. It includes a standalone-output Dockerfile, .dockerignore, compose.yml, and the required Next.js configuration.

Running

During development:

bun run index.ts up

Run the dedicated unit suite and type checks:

bun run test
bun run typecheck

Other useful commands:

bun run index.ts ps
bun run index.ts logs
bun run index.ts logs -f
bun run index.ts exec app sh
bun run index.ts start
bun run index.ts stop
bun run index.ts restart
bun run index.ts rollback
bun run index.ts fuck app
bun run index.ts db ls
bun run index.ts s3 ls
bun run index.ts s3 creds app
bun run index.ts s3 ui app
bun run index.ts login dmgnr
bun run index.ts users ls
bun run index.ts operations ls
bun run index.ts audit ls

All commands accept --config to use a configuration file other than .kuberrc.ts:

kuber --config production.kuberrc.ts up
kuber up --config production.kuberrc.ts

login, logout, and whoami are context-free and do not require a configuration.

Shell Completion

Generate and load completions for your shell:

source <(kuber complete zsh)
source <(kuber complete bash)

For a permanent setup, write the generated script to a file and source it from your shell configuration. Fish and PowerShell are also supported through kuber complete fish and kuber complete powershell.

Commands

• up [--no-b]: build images if needed, ensure the workspace, reconcile managed Postgres/S3, render manifests, apply them, wait for rollout, and delete stale resources
• start: like up but re-resolves the currently published image digests instead of building
• login [username] [--persist]: authenticate with the kuber API
• logout: revoke and remove the current API session
• whoami: show the authenticated API user and roles
• users: administer user accounts and roles
• operations: inspect server-side reconciliation operations
• audit: inspect the audit trail
• ps [-a]: print an ANSI graph of the current project namespace, hiding stopped deployments by default
• logs [deployment] [-f]: print (or follow) logs for one deployment or all managed deployments
• exec <deployment> <command...>: execute a command inside a running deployment pod over an interactive WebSocket
• restart [deployment]: roll out a restart across managed deployments
• stop: delete the matching name-scoped HPAs (so autoscaling cannot scale replicas back up) and scale managed deployments to zero
• rollback (alias fuck) [deployment]: roll one deployment back to its previous release, or all managed deployments when no name is given
• down [-f]: delete managed resources while keeping ingress, PVCs, managed databases, and managed S3 storage; -f also deletes those and the namespace
• db ls / db creds <service>: list or print credentials for managed Postgres claims
• s3 ls / s3 creds <service> / s3 ui <service>: list managed S3 claims, print their credentials, or print the Garage UI URL for a bucket
• export [-o file]: render manifests to a YAML file without applying them

Lifecycle commands (restart, stop, rollback, down) are idempotent: identical requests are deduplicated server-side and tracked as operations.

Configuration

Kuber optionally loads .kuberrc.ts from the working directory. The file must default export an object satisfying the published KuberConfig type:

import type { KuberConfig } from "@dmgnr/kuber";

export default {
  project: "my-app",
  composeFile: "compose.production.yml",
  registry: "registry.example.com",
  rolloutTimeoutMs: 10 * 60_000,

  async compose(compose) {
    const app = compose.services?.app;
    if (app && !Array.isArray(app.environment)) {
      app.environment ??= {};
      app.environment.NEXT_PUBLIC_BUILD_ID =
        await Bun.$`git rev-parse --short HEAD`
          .text()
          .then((value) => value.trim());
    }
  },
} satisfies KuberConfig;

Operational defaults:

• project: Compose top-level name, falling back to the current working directory name
• composeFile: the first recognized Compose filename in the working directory
• registry: registry.neko-piranha.ts.net
• rolloutTimeoutMs: 300000

Project-name precedence is .kuberrc.ts project, Compose top-level name, then the current working directory name.

The registry value controls which registry the CLI requests build images from and which the server uses to resolve published digests. rolloutTimeoutMs bounds how long up and rollback wait for a Deployment rollout.

Configuration hooks can be synchronous or asynchronous and receive mutable values:

• compose(compose, context): once after parsing and validation; affects every command that reads Compose
• preBuild(compose, context): before build eligibility is evaluated when builds are enabled
• postBuild(result, context): after images are built; result contains built and changed service names
• postRender(resources, context): after rendering and before reconciliation planning; also runs for export
• postApply(resources, context): after desired resources are successfully applied

Hook context contains the resolved cwd, project, composeFile, and optional configFile. A hook error aborts the command and is reported by the normal CLI error handler.

Registry and rollout configuration remain part of the CLI-facing contract. Image builder selection is not configurable: builds are scheduled, executed, and owned entirely by the server.

Compose Conventions

kuber supports a few project-specific Compose conventions on top of normal service translation.

Host-Based Ports

If a ports entry uses a hostname instead of a numeric published port, kuber treats it as an ingress host and routes traffic to the target container port.

Example:

services:
  app:
    ports:
      - somedomain.astrxl.dev:3000

That produces a Kubernetes Ingress rule for somedomain.astrxl.dev pointing at the service port for container port 3000.

Single-level wildcard subdomains are supported. Quote wildcard entries so YAML does not treat the leading * as an alias:

services:
  app:
    ports:
      - "*.astrxl.dev:3000"
      - "*.secure.astrxl.dev:3001:protected"

Protected routes use the kuber dialect and render Traefik IngressRoute resources instead of plain Kubernetes Ingress:

services:
  app:
    ports:
      - db.astrxl.dev:4984:protected
      - status.astrxl.dev:3001:protected(/dashboard,/socket.io)

Translation rules:

• host:port -> Kubernetes Ingress
• host:port:protected -> Traefik IngressRoute with middleware routing/cf-auth and host-wide matching
• host:port:protected(path1,path2,...) -> Traefik IngressRoute with middleware routing/cf-auth and explicit PathPrefix(...) matches only

Replicas and Autoscaling

deploy.replicas (or the top-level scale field) controls the Deployment replica count. A plain integer or numeric string renders a fixed replicas value, with scale taking precedence over deploy.replicas.

A "min-max" range string requests autoscaling instead of a fixed count:

services:
  app:
    image: app
    deploy:
      replicas: "2-6"

kuber renders:

• a Deployment with replicas set to the range minimum (2)
• a HorizontalPodAutoscaler (autoscaling/v2) targeting that Deployment, with minReplicas: 2, maxReplicas: 6, and a CPU target of 80% utilization
• a 100m CPU request injected into the container, unless x-container already specifies a CPU request (the HPA needs a CPU request to scale on)

For both fixed counts above one and autoscaled ranges whose maximum exceeds one, kuber also adds a topologySpreadConstraints entry spreading pods across hosts (kubernetes.io/hostname, maxSkew: 1, whenUnsatisfiable: ScheduleAnyway).

Malformed non-numeric replica values (for example "lots") are rejected with a clear error instead of silently defaulting.

Managed Postgres

You can declare a managed Postgres database with a pseudo-volume:

services:
  app:
    volumes:
      - postgresql:app

Or with an explicit username and database name:

services:
  app:
    volumes:
      - postgresql:user/database

This creates or reuses the managed CNPG role secret, reconciles the database resource, and injects DATABASE_URL and REDIS_URL=redis://redis.database.svc.cluster.local into the generated app secret in Kubernetes. REDIS_URL is only added to services with a managed Postgres claim. Running kuber db creds <service> performs the same focused Secret, role, and Database reconciliation before printing credentials.

Managed S3

Declare a Garage S3 bucket and access key with a pseudo-volume:

services:
  app:
    volumes:
      - s3:app

This creates GarageBucket/app and GarageKey/app in garage-system. To use different key and bucket names:

services:
  app:
    volumes:
      - s3:app-key/shared-assets

The Garage operator generates the credentials. kuber reads its generated Secret and injects these values into the service's <service>-env Secret:

• AWS_ACCESS_KEY_ID
• AWS_SECRET_ACCESS_KEY
• AWS_ENDPOINT_URL_S3
• AWS_REGION
• S3_BUCKET

One service can declare both postgresql:... and s3:...; all generated values are merged into the same service Secret. Managed Garage buckets and keys are retained by normal down and deleted by down -f.

Inspect a claim, print its generated credentials, or get its Garage UI URL:

kuber s3 ls
kuber s3 creds app
kuber s3 ui app

s3 creds prints AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, AWS_ENDPOINT_URL_S3, AWS_REGION, and S3_BUCKET as shell-style environment assignments. s3 ui only prints the URL; it does not open a browser.

Environment Files

env_file entries are read locally and turned into a Kubernetes Secret named <service>-env. Deployments then consume that secret through envFrom.

This is also where generated values such as DATABASE_URL and the managed S3 environment are injected.

Volumes

kuber treats different volume shapes differently:

• file bind mounts become ConfigMaps
• directory bind mounts become PVC-backed mounts
• named volumes become PVC-backed mounts
• tmpfs becomes emptyDir with memory backing
• postgresql:... is treated as a managed database claim, not as a filesystem mount
• s3:... is treated as a managed object-storage claim, not as a filesystem mount

Named volumes also support kuber-specific Longhorn storage hints. Kuber renders each distinct placement policy as a deterministic, reusable Longhorn StorageClass, then references that class from the PVC. The generated class uses Longhorn's numberOfReplicas, diskSelector, and dataLocality parameters; placement fields are never written directly to the PVC.

Default behavior:

# compose
services:
  app:
    volumes:
      - myvolume:/data

# effective kuber interpretation
services:
  app:
    volumes:
      - myvolume(1Gi on 2 fast):/data

Short syntax:

services:
  app:
    volumes:
      - data(20Gi):/data
      - archive(200Gi on archive):/archive
      - cache(10Gi on 1 fast):/cache

Meaning:

• name(20Gi):/path -> PVC size 20Gi
• name(20Gi on archive):/path -> PVC size 20Gi, with a StorageClass using diskSelector: "archive" and disabled data locality
• name(20Gi on 1 archive):/path -> PVC size 20Gi, with a StorageClass using one replica and diskSelector: "archive"

Explicit extensions are also supported.

Top-level named volume:

volumes:
  data:
    x-size: 20Gi
    x-diskTag: [archive]
    x-replicaCount: 1
    x-dataLocality: none

Long-form service mount:

services:
  app:
    volumes:
      - type: volume
        source: data
        target: /data
        volume:
          x-size: 20Gi
          x-diskTag: [archive]
          x-replicaCount: 1
          x-dataLocality: none

Precedence:

• short syntax like data(20Gi on 1 archive):/data
• long-form volume.x-*
• top-level volumes.<name>.x-*
• fallback default 1Gi on 2 fast

StorageClasses are cluster-scoped and content-addressed by policy. They are shared across projects and intentionally retained when a project is removed.

Building

When a service declares build, the CLI:

1. snapshots the repository into a content-addressed workspace (committed git state, tracked changes, untracked files, and ignored .env* files),
2. negotiates with the server and uploads only the blobs it is missing,
3. submits a build request; the server materializes the workspace from the CAS onto a shared RWX PVC and runs a rootless BuildKit Job that builds and pushes the configured image with registry cache.

Build containers are restricted: they run as non-root (runAsUser/runAsGroup 1000), do not mount the service account token, and only read the workspace (read-only mount) and a writable BuildKit state emptyDir. After a push, the registry's manifest digest is captured and embedded into the rendered Deployment as an immutable :latest@sha256:... reference, so a changed image naturally triggers a rollout. start and export look up the currently published digest without rebuilding, and fail if a buildable service has no published image yet.

export is side-effect-free: it refuses to render managed Postgres or S3 claims (because it cannot call the server to generate credentials) and instead tells you to run kuber up or remove the managed provider claims.

Rollback

kuber rollback [deployment] (alias fuck) rewinds managed Deployments to the previous release. ReplicaSets carry a deployment.kubernetes.io/revision annotation, and rollback restores the complete pod template from the next-older ReplicaSet via a JSON Patch, then waits for the rollout to complete. With no argument every managed Deployment is rolled back; pass a deployment name to target a single one.

Notes and limitations:

• Kubernetes only keeps its most recent ReplicaSets, so an older release may no longer be reachable after enough successful rollouts/rollbacks.
• Rollback restores the pod template (including image and environment), not live Secret, ConfigMap, PVC, database, or S3 state.
• A later kuber up or kuber start re-resolves :latest and returns the Deployment to the current desired state anyway, so rollback is the right tool for responding to a bad deploy, not for permanently pinning an old version.
• Rollback only considers Deployments managed by kuber (app.kubernetes.io/managed-by=kuber) and only runs within a workspace whose namespace kuber owns.

Workspace State and Operations

The server persists per-workspace state, revisions, operations, and audit events as Secrets/ConfigMaps in kuber-system. Mutations are idempotent:

• operations carry an idempotency key so retries do not double-apply,
• resource deletion requires a UID precondition,
• workspace replacement is guarded by If-Match.

The CLI is built with the normal build script for distribution and local use:

bun run build

Notes

• Resource names and namespaces are derived from the current working directory.
• The workspace snapshot is optimized for local iteration, not for producing a perfectly clean export of the repository.
• Managed database support is Kubernetes-only. It injects DATABASE_URL into the generated app secret and does not rewrite local .env files.
• kuber operates on managed resources in the namespace matching the current directory name.
• Builds are scheduled, executed, and owned by the server. There is no local SSH/daemon builder configuration; registry and rolloutTimeoutMs remain CLI-facing settings.