Sinain

Context OS — ambient intelligence for builders. Captures what you see and hear, distills it into a private knowledge graph, accessible from MCP, a web UI, and a screen-recording-invisible HUD overlay.

Quick Start · Docs · Privacy · Configuration · Contributing

You, Augmented

Sinain captures your screen and audio continuously, distills the stream into a structured live knowledge graph of facts, entities, and decisions, and exposes that graph through every interface where you might need it.

• Capture — screen frames + system audio, with <private> tag stripping and auto-redaction of credentials and tokens before anything leaves your machine.
• Distill — facts (atomic claims), entities (people / projects / topics), decisions (what was chosen and why) — extracted by an LLM, integrated by deterministic graph operations (no LLM in the integration step → 82.8% Information Preservation Rate on the LongMemEval benchmark (ICLR 2025, 500 questions), measured against full-context oracle).
• Access from anywhere — MCP server for agents (Claude Code, Codex, Goose, OpenClaude, Junie), web UI at localhost:9500/knowledge/ui/ for browsing, and a HUD overlay for in-the-moment recall.

The HUD overlay is invisible to screen capture — never appears in screenshots, recordings, or screen shares.

Agent-Agnostic

Sinain feeds the same screen and audio context to any MCP-compatible agent. Switch agents on the fly — no restart, no context loss.

• Tested with Claude Code, OpenClaude, Codex, Goose, and Junie. Any MCP-compatible agent works.
• Pick agents in the overlay's flash-icon selector — spawn tasks can route to any profile in your roster.
• Add custom profiles (personal Claude config, alternate models) by editing agents.json. The roster is the source of truth.
• Knowledge modules travel with you — export from one machine, import on another.

Privacy Controls

By default, sinain uses cloud APIs (OpenRouter) for transcription and analysis. When you need tighter control, switch privacy modes — no code changes, one env var.

• off → standard → strict → paranoid — four modes in ~/.sinain/.env.
• paranoid mode: Ollama + whisper.cpp. Zero network calls at runtime — the embedding model is pre-cached during the setup wizard (sinain setup-embedding), so subsequent runs are fully offline.
• HUD overlay is invisible to screen capture (NSWindow.sharingType = .none — see overlay/macos/Runner/AppDelegate.swift:70). Verifiable via split-screen recording with QuickTime + OBS + Loom — the HUD is absent from all three.

Quick Start

npx @geravant/sinain@latest start

That's it. On first run, sinain will:

1. Run an interactive setup wizard — transcription backend, API key, agent, privacy mode
2. Auto-download the overlay app (~17MB), sck-capture binary (~5MB), embedding model (~90MB), and Python dependencies
3. Start all services — sinain-core, sense_client, overlay, and agent

All assets are cached locally after the first install. In paranoid mode, subsequent runs are fully offline — no network calls at runtime.

Pin @latest on every invocation. npx @geravant/sinain (without @latest) caches forever against the unversioned spec — you'd silently keep running an old version for months. Sinain self-updates automatically when stale, but pinning @latest makes it explicit and saves a redundant relaunch.

Re-run the wizard anytime: npx @geravant/sinain@latest start --setup

Prerequisites

• macOS 12.3+ — Sinain uses ScreenCaptureKit (introduced in 12.3). Earlier versions are not supported in this release. Apple Silicon and Intel both work.
• Node.js 18+ — nodejs.org (LTS recommended)
• Python 3.10+ — brew install python3 (macOS) or python.org
• OpenRouter API key (optional for local-only mode) — openrouter.ai
• Network access during first install — the wizard downloads ~112MB total (overlay app, sck-capture binary, sentence-transformer embedding model). All cached locally; subsequent runs need network only for cloud LLM API calls (or zero network in paranoid mode).

Fully local? No API key needed. Ollama + whisper-cli = zero cloud at runtime. See Running Fully Local.

First install reproducibility? See docs/cold-install-verification.md for a step-by-step verified-on-fresh-user-account guide, including the timing measurement and the failure modes the audit caught + fixed.

macOS Permissions

1. System Settings → Privacy & Security → Screen Recording — add your Terminal, then quit and reopen Terminal (macOS TCC entitlements only apply to processes started after the grant)
2. System Settings → Privacy & Security → Microphone — same: add Terminal, then restart Terminal

Sinain detects when these permissions are missing and surfaces a clear restart-instruction banner; you'll never get a silent degraded mode.

Managing sinain

npx @geravant/sinain@latest stop             # stop all services
npx @geravant/sinain@latest status           # check what's running
npx @geravant/sinain@latest start --setup    # re-run setup wizard
npx @geravant/sinain@latest start --no-sense # skip screen capture
npx @geravant/sinain@latest start --no-overlay  # headless mode

Always pin @latest — see the note in Quick Start above.

Architecture

┌─── Your Device ─────────────────────────────────────────────────────┐
│                                                                     │
│  sck-capture (Swift)                                                │
│    ├─ system audio (PCM) ──► sinain-core :9500                      │
│    └─ screen frames (JPEG) ──► sense_client ─── POST /sense ──►    │
│                                                      │              │
│                              ┌────────────────────────┘              │
│                              │                                      │
│                         sinain-core                                 │
│                           ├─ audio pipeline → transcription         │
│                           ├─ agent loop → digest + HUD text         │
│                           ├─ knowledge graph (private, on-device)   │
│                           └─ WebSocket feed                         │
│                                  │                                  │
│                                  ▼                                  │
│                           overlay (Flutter)                         │
│                           private, invisible to screen capture      │
│                                                                     │
└─────────────────────────────────────────────────────────────────────┘

Components

| Component | Language | What it does | Docs | |---|---|---|---| | sinain-core | TypeScript | Central hub: audio pipeline, agent loop, knowledge graph, WS feed | README | | overlay | Dart / Swift | Private HUD (macOS), display modes, hotkeys | Hotkeys | | sense_client | Python | Screen capture, SSIM diff, OCR, privacy filter | sense_client/ | | sck-capture | Swift | ScreenCaptureKit: system audio + screen frames | tools/sck-capture/ | | sinain-agent | Bash | Shell harness that connects any agent to sinain-core | sinain-agent/ | | sinain-knowledge | TypeScript | Curation, playbook, eval, portable knowledge modules | Knowledge System | | sinain-mcp-server | TypeScript | MCP server exposing sinain tools to agents | sinain-mcp-server/ |

Configuration

Sinain splits config across two files in ~/.sinain/:

• .env — secrets (API keys) and infrastructure (ports, audio device, privacy mode, analyzer LLM).
• agents.json — agent roster (default agent, allowed-tools whitelists, analyzer pacing).

Both are created by the setup wizard. To re-run: npx @geravant/sinain start --setup.

Agents & profiles → agents.json

The agent roster lives in ~/.sinain/agents.json. Each entry is a profile mapping a name to a binary + behavior type + optional env, settings, and model overrides. The overlay's flash-icon selector lets you pick which profile handles spawn tasks at runtime. Custom profiles like pclaude (personal claude with its own config dir) are first-class — the dispatch decision keys off profile.type, not the profile name.

See Agent Roster & Profiles for the complete schema, recipes, and routing model.

Context Analysis (HUD summarizer) → .env

The context analysis loop runs every 3–30 seconds, sending recent audio/screen context to an LLM. It produces the short HUD text shown on the overlay plus a richer digest stored in the feed buffer for the knowledge graph.

| Variable | Default | Description | |---|---|---| | ANALYSIS_PROVIDER | openrouter | openrouter (cloud) or ollama (local, free) | | ANALYSIS_MODEL | google/gemini-2.5-flash-lite | Primary model for text analysis | | ANALYSIS_VISION_MODEL | google/gemini-2.5-flash | Auto-selected when screen images are present | | ANALYSIS_ENDPOINT | (auto per provider) | Override for custom OpenAI-compatible endpoints | | ANALYSIS_API_KEY | (from OPENROUTER_API_KEY) | API key; not needed for ollama | | ANALYSIS_FALLBACK_MODELS | gemini-2.5-flash,... | Comma-separated fallback chain |

Other Key Settings → .env

| Variable | Default | Description | |---|---|---| | OPENROUTER_API_KEY | — | Required (unless ANALYSIS_PROVIDER=ollama + local transcription) | | PRIVACY_MODE | off | off / standard / strict / paranoid |

See docs/CONFIGURATION.md for the full reference.

Privacy Modes

| Mode | What it does | |---|---| | off | All data flows freely — maximum insight quality | | standard | Auto-redacts credentials before cloud APIs (wizard default) | | strict | Only summaries leave your machine — no raw text sent to cloud | | paranoid | Fully local: Ollama + whisper.cpp. Zero network calls at runtime (embedding model pre-cached at install). |

See Privacy Threat Model for the full design.

Hotkeys

Global hotkeys use Cmd+Shift:

| Shortcut | Action | |---|---| | Cmd+Shift+Space | Toggle overlay visibility | | Cmd+Shift+M | Cycle display mode | | Cmd+Shift+/ | Open command input | | Cmd+Shift+H | Quit overlay |

See docs/HOTKEYS.md for all 15 shortcuts.

Running Fully Local

No cloud APIs needed. Local models handle everything:

# 1. Install local transcription
./setup-local-stt.sh

# 2. Install Ollama + vision model
brew install ollama && ollama pull llava

# 3. Start in local mode
./start-local.sh

| Model | Size | Speed | Best for | |---|---|---|---| | llava | 4.7 GB | ~2s/frame | General use (recommended) | | llama3.2-vision | 7.9 GB | ~4s/frame | Best accuracy | | moondream | 1.7 GB | ~1s/frame | Fastest, lower quality |

Setup Guides

| Setup | Guide | |---|---| | Agent Roster & Profiles | docs/AGENT-ROSTER.md — pick agents, add custom profiles | | Bare Agent | docs/INSTALL-BARE-AGENT.md — the default install path | | From Source | git clone, cp .env.example ~/.sinain/.env, ./start.sh |

Knowledge System

Sinain builds a persistent knowledge graph from everything it captures — audio transcriptions, screen OCR, and agent interactions. Facts are distilled incrementally (on buffer full and session end), stored in an EAV triplestore with graph relationships, and retrieved via hybrid search (FTS5 + tag-based + entity graph backrefs with RRF fusion).

The integration step is fully deterministic — no LLM decides what to store. Every extracted fact is preserved.

npx @geravant/sinain export-knowledge   # export playbook, modules, graph
npx @geravant/sinain import-knowledge ~/sinain-knowledge-export.tar.gz

See Knowledge System docs for architecture details.

Querying knowledge from any MCP agent

Sinain's knowledge graph is exposed to any MCP-aware agent via the bundled MCP server. See Connect Your Coding Agent (MCP) below for setup.

Connect Your Coding Agent (MCP)

Sinain ships an MCP server that exposes 15 sinain_* tools — including sinain_knowledge_query, sinain_get_knowledge, sinain_distill_session, sinain_get_context, and sinain_respond — to any MCP-aware agent. Register it once and the agent can read your knowledge graph and surface text on the HUD from any project.

npx @geravant/sinain@latest mcp install

The wizard detects which MCP agents you have installed and registers sinain for the ones you select. Re-runnable any time; idempotent.

| Agent | Setup | Config it touches | |---|---|---| | Claude Code | mcp install (auto via wizard) | ~/.claude.json (claude mcp add) | | Claude Desktop | mcp install (auto via wizard) | ~/Library/Application Support/Claude/claude_desktop_config.json (mac) | | Cursor | mcp install (auto via wizard) | ~/.cursor/mcp.json | | Codex | mcp install (auto via wizard) | ~/.codex/config.toml (codex mcp add) | | Goose | mcp install (auto via wizard) | ~/.config/goose/config.yaml | | Junie | mcp install (auto via wizard) | ~/.junie/mcp/mcp.json |

Already in sinain onboard — step 6 of the advanced flow runs the same registration. Quickstart asks once if any MCP agent is detected.

• See docs/MCP-INTEGRATION.md for setup details, troubleshooting, and the manual pclaude / alternate CLAUDE_CONFIG_DIR recipe.
• See docs/MCP-CAPABILITIES.md for what each tool enables, with example prompts and end-to-end recipes.

Deep Dives

| Topic | Doc | |---|---| | Knowledge System | docs/knowledge-system.md | | Knowledge API (HTTP) | docs/KNOWLEDGE-API.md | | MCP Integration (setup) | docs/MCP-INTEGRATION.md | | MCP Capabilities (tools + recipes) | docs/MCP-CAPABILITIES.md | | LongMemEval Audit | docs/LONGMEMEVAL-AUDIT.md | | Privacy Threat Model | docs/privacy-protection-design.md | | Full Configuration | docs/CONFIGURATION.md | | All Hotkeys | docs/HOTKEYS.md |

Source

Full source, issues, and contributing guide: github.com/anthillnet/sinain-hud

Contributing

See CONTRIBUTING.md.

License

MIT