SkyKoi Runtime

The engine behind a koi: a personal AI agent that lives on a device, talks to its owner across every messaging channel, runs real work with tools, and remembers.

This repository is the npm package skykoi (CLI binaries koi and skykoi). One install bundles three things into one process: an agent, a gateway, and a multi-channel front door.

flowchart LR
    A["One npm install: skykoi"] --> B["Agent<br/>reasons, calls tools, remembers"]
    A --> C["Gateway<br/>always-on WS / RPC server"]
    A --> D["Front door<br/>WhatsApp, iMessage, Telegram..."]

Table of contents

• What it is
• System architecture
• Core vocabulary
• Quick start
• How the program starts
• Repository map
• Core subsystems
  • The koi agent loop
  • The gateway
  • Channels, routing and pairing
  • Providers and models
  • Configuration
  • Memory
  • Cron, infra and self-update
  • Device and browser
  • Plugins, extensions and native apps
• Lifecycle of an inbound message
• Where to add things
• Build, test and release
• Contributor conventions
• Security model
• Further reading

How to read this doc. It is the onboarding map, not the full manual. It is written to stay correct as the code moves, so it follows three rules. Please keep them when you edit: point to the source of truth, do not duplicate it (counts, versions and line numbers are deliberately avoided); key on directories, not files (directories are stable, files churn); and explain the why. The exhaustive manual is docs.skykoi.com; contributor rules live in AGENTS.md. This README bridges the two.

What it is

SkyKoi Runtime is the program that is a koi. It is published to npm and runs wherever the koi lives: today that is primarily the user's own device (the macOS menubar app, or any Mac/Linux/Windows host), reachable from the cloud through a tunnel. It can also run on a server or VM. (It is host-agnostic, so do not assume EC2; an earlier managed fleet has been retired.)

A single koi is an LLM-driven loop with a persistent workspace and memory. The runtime wraps that loop in an always-on gateway and connects it to the outside world through pluggable channels, the CLI/TUI, and native apps.

System architecture

The whole runtime at a glance: every way a message can arrive, funneled into one agent loop that is backed by a handful of subsystems.

flowchart TB
    owner(["Owner"])

    subgraph ways["Ways in"]
        direction LR
        CH["Channels<br/>WhatsApp - iMessage<br/>Telegram - Discord - Slack..."]
        GWc["Gateway clients<br/>web - mobile / desktop apps<br/>platform connect"]
        CLI["CLI / TUI<br/>local terminal"]
    end

    owner --> CH & GWc & CLI

    CH & GWc & CLI --> RP["Routing + Pairing<br/>which koi? who is this?"]
    RP --> KOI["The koi agent loop<br/>prompt then tool then result"]

    KOI --> PROV["LLM providers"]
    KOI --> TOOLS["Tools<br/>exec - browser - files - device"]
    KOI --> MEM["Memory<br/>sqlite + vectors"]
    KOI --> WSP["Workspace<br/>KOI / SOUL / USER / MEMORY.md"]
    KOI --> CRON["Cron<br/>scheduled turns"]

    classDef hot fill:#0aa,stroke:#066,color:#fff;
    class KOI hot;

Viewed as layers, the same system stacks like this. Surfaces depend on the gateway, which hosts the agent, which rests on the foundational services:

flowchart TB
    subgraph L4["Surfaces"]
        direction LR
        s1["CLI / TUI"]
        s2["Native apps"]
        s3["Channels"]
    end
    subgraph L3["Gateway  ·  src/gateway"]
        g1["WebSocket / RPC  ·  sessions  ·  heartbeat  ·  control UI"]
    end
    subgraph L2["Agent  ·  src/koi"]
        k1["agent loop  ·  tools  ·  prompt  ·  sessions"]
    end
    subgraph L1["Foundations"]
        direction LR
        f1["providers"]
        f2["config"]
        f3["memory"]
        f4["infra"]
    end
    L4 --> L3 --> L2 --> L1

Core vocabulary

These seven terms unlock the codebase.

| Term | Meaning | | ----------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | koi | One agent identity. Has a workspace, memory, model config, and sessions. A host can run several. | | gateway | The always-on process. Hosts the WebSocket/RPC server, channels, cron, and the koi runtime. | | session | One conversation thread with a koi, keyed by (koi, channel, account, peer). Persisted as a transcript. | | channel | A messaging surface (Telegram, iMessage, web...). A pluggable adapter translating native to internal messages. | | command vs tool | A command is owner-typed chat text (/model, /reset) intercepted before the LLM. A tool is an action the LLM itself calls (exec, browser, email). | | the "Pi" runtime | The embedded agent core (@mariozechner/pi-*). The koi loop wraps it; see src/koi/pi-embedded*. | | workspace | ~/.skykoi/...: the koi's editable brain on disk (KOI.md, SOUL.md, USER.md, MEMORY.md, sessions). |

Quick start

Requirements: Node >= 22, and pnpm (the version is pinned in package.json).

pnpm install
pnpm build                 # compile src/ to dist/ (tsdown). The CLI always runs dist/.

# Run the CLI from source during development:
pnpm dev <command>         # node scripts/run-node.mjs (rebuilds stale dist, then runs)
pnpm skykoi --help         # list every command

# Common loops:
pnpm gateway:dev           # run the gateway locally (channels skipped, fast)
pnpm gateway:watch         # gateway with auto-reload on change
pnpm tui:dev               # the terminal chat UI against a dev profile

Quality gates, before pushing anything with logic changes:

pnpm check                 # tsgo (typecheck) + lint + format
pnpm test                  # vitest suite (colocated *.test.ts)

[!IMPORTANT] dist/ vs src/. The koi binary (skykoi.mjs) imports dist/entry.js, so a code change is not live on the CLI until it is built. pnpm dev and pnpm gateway:watch rebuild for you; a bare koi ... runs whatever is already in dist/.

How the program starts

Follow this chain once and the entire CLI makes sense.

flowchart TB
    bin["skykoi.mjs<br/><i>npm bin</i>"] --> entry["src/entry.ts<br/>title, env, re-spawn, profiles"]
    entry --> main["src/cli/run-main.ts<br/>dotenv, first-run, parse argv"]
    main --> reg["src/cli/program/command-registry.ts<br/>catalog of subcommands"]
    reg --> cmd["src/commands/*<br/>the command implementation"]
    cmd --> deps["createDefaultDeps()<br/>injected config + helpers"]

| Piece | Role | | ----------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | src/cli/ | Commander-based CLI: program construction, argv helpers, help/version, and a fast-path router (src/cli/route.ts) that shortcuts common read-only commands past full parsing. | | src/cli/deps.ts | createDefaultDeps(): the dependency-injection container (config, workspace, koi helpers) passed into commands so they are testable. | | src/runtime.ts | The injectable log / error / exit surface, so tests can capture output. |

The default action: koi with no args checks login, then drops into the TUI chat, or shows the first-run welcome on a fresh machine.

Repository map

The product is a pnpm workspace: the root package (skykoi) plus extensions/* (plugins) and apps/* (native apps). Start with src/.

| Path | What lives here | Start here | | ---------------------------------------------------------------------- | ---------------------------------------------------------------------------------------- | ------------------------------------------ | | src/entry.ts, src/index.ts, src/runtime.ts | Process bootstrap + public library entry | src/entry.ts | | src/cli/, src/commands/ | The CLI: command registry + every subcommand | command-registry.ts | | src/koi/ | The agent. The Pi-embedded loop, tools, prompt assembly, sessions, models, sandbox | pi-embedded-runner/run.ts | | src/auto-reply/ | Inbound message to command-gating to koi turn to streamed reply | reply.ts | | src/gateway/ | The server. WebSocket + RPC, sessions, heartbeat, control UI, OpenAI-compatible HTTP | server.impl.ts | | src/gateway-client.ts | The WS client apps/devices use to connect | gateway-client.ts | | src/channels/ | Channel plugin system: adapter interfaces, discovery, allowlists, gating | plugins/types.core.ts | | src/telegram/ discord/ slack/ signal/ imessage/ web/ whatsapp/ line/ | Per-channel adapters (native API to internal messages) | each dir entry | | src/routing/, src/pairing/ | Map (channel, account, peer) to (koi, session); DM pairing and allowlists | routing/resolve-route.ts | | src/providers/, src/koi/models-config*.ts | LLM provider adapters and model selection | models-config.providers.ts | | src/config/ | Config schema, load/validate ~/.skykoi/skykoi.json, hot-reload, migrations | zod-schema.ts, paths.ts | | src/memory/ | Long-term memory: SQLite + vectors (sqlite-vec) + hybrid search | manager.ts | | src/world-model/ | The per-koi context block injected into the system prompt | renderer.ts | | src/cron/ | Scheduled, isolated koi turns | service.ts | | src/infra/ | Backoff/retry, update/self-update, exec-approval policy, heartbeat | by filename | | src/node-host/, src/browser/ | Device capabilities + Chrome automation (the nodes and browser tools) | node-host/runner.ts, browser/client.ts | | src/security/, src/gateway/device-auth.ts | Auth tokens, device pairing, exec policy | device-auth.ts | | src/plugins/, src/plugin-sdk/ | The plugin loader + the public SDK extensions/* build against | plugin-sdk/index.ts | | src/tui/ | The terminal chat UI | tui/ | | extensions/* | Workspace plugins: extra channels and memory backends | each package.json | | apps/macos ios android shared | Native clients (Swift/Kotlin) that speak the gateway protocol | per-app folder | | docs/ | The Mintlify docs site (docs.skykoi.com) | docs/index.md | | dist/ | Build output. Generated, never edit. | - |

Some directories hold hundreds of files. The "start here" file is the door; the imports take you the rest of the way. A directory's own README.md or doc page outranks this table.

Core subsystems

The koi agent loop

src/koi/ is the heart. A koi "turn" builds context, calls the LLM, runs any tools it asks for, feeds results back, and repeats until done. It wraps the embedded Pi agent core (@mariozechner/pi-*; local tweaks live in patches/).

flowchart LR
    A["Build context<br/>system prompt + world model<br/>+ workspace + tools"] --> B["Call the LLM"]
    B --> C{"Tool calls?"}
    C -->|yes| D["Run tools<br/>exec - browser - files - email..."]
    D --> E["Feed results back"]
    E --> B
    C -->|no| F["Persist session<br/>stream the final reply"]
    classDef done fill:#0a6,stroke:#063,color:#fff;
    class F done;

| Area | Where | Notes | | --------------- | ----------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Orchestration | pi-embedded-runner/ | run.ts drives the loop; attempt.ts is one step; compact.ts trims history; model.ts picks the model and fails over. | | System prompt | system-prompt.ts + src/world-model/ | Assembled from runtime metadata, the world model, workspace files, tools, and channel rules. | | Tools | src/koi/tools/ | One file per capability: exec, browser-tool, file read/write, email-tool, memory-tool, nodes-tool, cron-tool, sub-agents, per-channel *-actions. | | Sessions | src/koi/ | Persisted per (koi, sessionKey), guarded against concurrent writes and corruption. | | Auth and models | auth-profiles/, model-selection.ts, models-config*.ts | Resolve provider/key/model and fail over when one is unavailable. | | Sandbox | src/koi/sandbox/ | Docker isolation for non-main (group/channel) sessions and risky tool runs. |

The koi's brain on disk is plain Markdown it reads each turn and can edit:

flowchart LR
    subgraph ws["~/.skykoi workspace"]
        direction TB
        SOUL["SOUL.md<br/>behavior + rules"]
        KOIf["KOI.md<br/>config / personality"]
        IDN["IDENTITY.md<br/>name / avatar"]
        USR["USER.md + profile.json<br/>who the owner is"]
        MEMf["MEMORY.md + memory/*<br/>long-term memory"]
        HB["HEARTBEAT.md<br/>live state / tasks"]
    end
    ws --> sp["into every system prompt"]

The gateway

src/gateway/ is the always-on process that exposes the koi to the world. server.impl.ts boots the WebSocket server, an HTTP surface (control UI plus an OpenAI-compatible /v1/chat/completions), the channel manager, cron, heartbeat, and config hot-reload.

flowchart TB
    subgraph clients["Clients"]
        direction LR
        c1["macOS / iOS / Android apps"]
        c2["skykoi-live web surface"]
        c3["CLI / TUI"]
        c4["platform connect"]
    end
    clients -->|"WebSocket + RPC"| gw

    subgraph gw["Gateway process - src/gateway"]
        direction TB
        ws["WS server + RPC handlers<br/>server-methods*.ts"]
        chat["event fan-out<br/>server-chat.ts"]
        cm["channel manager"]
        http["HTTP: control UI + OpenAI-compatible API"]
    end

    gw --> koi["koi runtime - src/koi"]
    cm --> chans["Channels"]

[!NOTE] The wire protocol schema is defined in TypeScript and generated to dist/protocol.schema.json (plus Swift models) via scripts/protocol-gen*.ts. Breaking changes require updating all clients together: run pnpm protocol:check.

Channels, routing and pairing

Every messaging surface implements the same adapter interfaces (src/channels/plugins/types.core.ts: messaging / auth / outbound / group / command / setup). Channels are discovered and lazy-loaded; a configured channel that is not installed is logged, not fatal. Built-ins live in their own top-level dirs; extra channels are extensions/* packages built on the plugin-sdk.

flowchart TB
    M["Inbound native message"] --> ENV["Envelope<br/>normalized"]
    ENV --> RES["resolve-route.ts"]
    RES --> SK["session-key.ts<br/>channel + account + peer + scope"]
    RES --> PAIR{"known sender?"}
    PAIR -->|no| P["pairing + allowlist<br/>src/pairing"]
    PAIR -->|yes| T["target koi + session"]
    P --> T
    T --> KOI["koi loop"]

[!WARNING] When you change shared routing / allowlist / pairing / gating logic, change it for all channels at once. That is a recurring source of bugs (see AGENTS.md).

Providers and models

Adapters for Anthropic, OpenAI, AWS Bedrock, Google Gemini, Ollama, and others, plus the custom "skykoi" provider that proxies through the platform so a shared key never lands on the device. The actual LLM calls go through the Pi core.

flowchart LR
    sel["model-selection.ts<br/>capability + cost + availability"] --> p1{"primary<br/>available?"}
    p1 -->|yes| use["use it"]
    p1 -->|no| fb["fallback chain"]
    fb --> use
    use --> pi["Pi core makes the call"]
    cfg["models-config.providers.ts<br/>+ ~/.skykoi/models.json"] -.-> sel

Configuration

Config is ~/.skykoi/skykoi.json (JSON5: comments and trailing commas allowed, with ${ENV} substitution). paths.ts resolves locations; zod-schema.ts is the runtime source of truth for valid shape; TypeScript types mirror it in types*.ts. Old formats are upgraded by legacy-migrate.ts, and the gateway hot-reloads most changes.

[!NOTE] The root .env.example is a stale leftover, ignore it. Real configuration is ~/.skykoi/skykoi.json plus a handful of provider env vars; see the docs config pages.

Memory

Long-term recall beyond the live context window: messages and notes are embedded and stored in SQLite (sqlite-vec); retrieval fuses keyword (BM25) and vector similarity.

flowchart LR
    msg["messages + notes"] --> emb["embeddings"]
    emb --> db[("SQLite + sqlite-vec")]
    q["koi memory search"] --> hyb["hybrid: BM25 + vectors"]
    db --> hyb
    hyb --> ctx["into the prompt"]

Embeddings index asynchronously, so the newest items can lag the index slightly.

Cron, infra and self-update

src/cron/ runs scheduled, isolated koi turns (a reminder, a daily digest) with no live channel input, delivering the result to a target. src/infra/ is the cross-cutting glue: backoff, the exec-approval policy engine, heartbeat visibility, and the self-update path that pulls the latest npm version and restarts the gateway in place (how the fleet rolls forward).

Device and browser

src/node-host/ exposes the local machine's capabilities (screenshot, screen frames, location) that the koi drives via the nodes tool. src/browser/ is full Chrome automation over CDP plus Playwright, powering the browser tool and web-page capture.

Plugins, extensions and native apps

The plugin loader (src/plugins/) discovers packages declaring a skykoi.extensions entry and built against skykoi/plugin-sdk. This is how you add a channel or feature without touching core. apps/ holds the native clients (apps/macos is the menubar app that hosts the gateway today); they are clients of the gateway protocol.

Lifecycle of an inbound message

End to end, what happens when a message arrives from a channel:

sequenceDiagram
    actor User
    participant Ch as Channel adapter
    participant AR as auto-reply
    participant Rt as routing
    participant Koi as koi loop
    participant LLM as LLM provider
    participant Tool as Tools

    User->>Ch: native message
    Ch->>AR: normalize to Envelope
    AR->>AR: command? authorized?
    AR->>Rt: resolve koi + session
    Rt->>Koi: run turn
    loop until done
        Koi->>LLM: messages + tools
        LLM-->>Koi: text or tool call
        Koi->>Tool: execute tool
        Tool-->>Koi: result
    end
    Koi-->>AR: streamed events
    AR-->>Ch: final reply
    Ch-->>User: delivered

The gateway/app path is the same picture, except the entry point is a chat.send RPC instead of a channel adapter.

Where to add things

| You want to... | Go to | Notes | | -------------------------------------- | ------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------- | | Add a tool the koi can call | src/koi/tools/ | One file per tool; register it in the runner's toolset. Mind the tool-schema guardrails in AGENTS.md. | | Add a CLI command | src/commands/ + register in command-registry.ts | Take deps from createDefaultDeps(). | | Add a gateway RPC method | src/gateway/server-methods*.ts + protocol schema | Run pnpm protocol:gen and update clients. | | Add a channel | extensions/<name>/ against plugin-sdk (or a src/<channel>/ adapter) | Implement the types.core.ts interfaces; wire routing/allowlist/gating. | | Add an LLM provider/model | models-config.providers.ts | Add the client init + model defs; selection picks it up. | | Change the system prompt / context | system-prompt.ts, src/world-model/ | Affects every koi; test broadly. | | Change config shape | zod-schema.ts (+ types*.ts, add a migration) | Schema is the runtime source of truth. |

Build, test and release

Everything is in package.json scripts (pnpm run lists them). The ones you will use most:

pnpm build            # tsdown: src/ to dist/  (config: tsdown.config.ts)
pnpm check            # typecheck + lint + format (oxlint / oxfmt)
pnpm test             # vitest; pnpm test:coverage for V8 coverage
pnpm test:live        # tests that hit real provider keys (SKYKOI_LIVE_TEST=1)
pnpm test:docker:*    # end-to-end onboarding / gateway / install flows in Docker
pnpm protocol:check   # fail if the generated wire protocol is out of date

• Tests are colocated *.test.ts (e2e: *.e2e.test.ts); the full kit is documented in docs/help/testing.md.
• Releases are gated and manual: read docs/reference/RELEASING.md and docs/platforms/mac/release.md first, and never bump versions or npm publish without explicit owner sign-off (see AGENTS.md).
• Native version bumps touch several files at once: see AGENTS.md -> "Version locations".

Contributor conventions

AGENTS.md is the detailed playbook (commit/PR flow, docs/i18n rules, multi-koi git safety, channel-refactor checklists, release guardrails). The essentials:

• TypeScript, ESM, strict. Avoid any. Keep files reasonably small (~500 LOC guideline; pnpm check:loc). Run pnpm check before you commit.
• Add brief comments for non-obvious logic. Match the surrounding style.
• Touch one channel, consider them all when the change is to shared routing / allowlist / pairing / gating / onboarding logic.
• Never edit node_modules or dist/. Both are generated.
• Dependency patches/overrides need explicit approval and must be pinned to an exact version.

Security model

flowchart TB
    subgraph trusted["Host - full access"]
        main["main session<br/>tools run on the host"]
    end
    subgraph sandboxed["Docker sandbox"]
        grp["group / channel sessions<br/>+ risky tool runs"]
    end
    unknown(["Unknown sender"]) --> pair["DM pairing required"]
    pair --> main
    sensitive["Sensitive command"] --> approve["exec-approval system"]
    approve --> main
    device(["Gateway client"]) --> auth["device auth"] --> main
    creds["provider keys"] --> proxy["platform proxy<br/>keys stay off device"]

In one breath: the main session runs tools on the host with full access; non-main sessions can be Docker-sandboxed; unknown senders must pass DM pairing; sensitive commands go through exec-approval; device auth guards the gateway; and the shared-key platform proxy keeps provider credentials off the device. Report vulnerabilities per docs/SECURITY.md.

Further reading

• Full docs: docs.skykoi.com (sources in docs/: channels, gateway, tools, providers, platforms, config, releasing).
• Contributor playbook: AGENTS.md.
• The web/voice surface that talks to this runtime: the skykoi-live repo (it connects to this gateway for deep work).

License

BUSL-1.1: see LICENSE and docs/THIRD_PARTY_LICENSES.txt.