Friend

A self-hosted personal AI assistant. It runs on your PC, talks to you from any screen over WebSocket, uses tools on your machine with risk-based approvals, and controls a full-screen sci-fi "stage" itself.

Phase 1: secure login, agent loop with Claude, CLI device node, approvals with PIN, and a minimal stage with a particle-sphere presence. Phase 2a: chart, gauge, timeline and graph objects, automatic layout, fade-out, orb and ring voice styles and mood intensity. Phase 5a: local voice. Hold Space and speak; Friend answers aloud. Phase 5b: only your voice commands it. Phase 2b: the agent can look through your camera, one approved picture at a time. Also built: memory (notes it keeps about you), tasks and reminders with a heartbeat, free-form pages in an isolated frame, a Docker code sandbox, and a plugin store. Models: Claude, or a local model through Ollama. See docs/superpowers/specs and docs/superpowers/plans.

Install (npm)

Needs only Node.js 20 or newer. Everything else is checked, and offered, by the setup wizard.

npm install -g friend-assistant
friend onboard

(or without installing: npx friend-assistant onboard)

friend onboard is a guided setup, one step at a time. Press Enter to take the suggestion at any question; run it again and it carries on where it stopped.

1. Welcome: what is about to happen.
2. What you want to use: pick any of: Claude through your subscription (Claude Code), Claude with an API key, a model on this PC (Ollama), voice, safe code running (Docker).
3. What this PC needs: it checks Node.js, plus only what your choices need: Python (voice), Docker, Ollama, Claude Code, and whether the port is free. For each missing one it shows the exact command (winget on Windows, brew on a Mac, apt on Linux, npm for Claude Code), asks before installing, runs it one by one with the output visible, and checks again. If a program was installed but this terminal cannot see it yet, it tells you to open a new terminal and run friend onboard again.
4. Your account: password and PIN (each typed twice) and the assistant's name.
5. How Friend thinks: signs in to Claude if needed (claude auth login), asks for an API key, or downloads the local model, and sets the safety net (the local model behind Claude).
6. Voice: a female or male voice, then the speech models (about 250 MB).
7. Your workspace folder, 8. Who can open Friend (only this PC, or your network, and the port), 9. How it looks (a theme), 10. Finish: a summary, and it can start Friend and open the browser.

Handy options: friend onboard --no-install (never installs anything, only tells you what to do), --yes (take every suggestion), --redo <step> (run one step again, for example --redo look), --reset (start from the beginning). friend doctor shows what this PC has and how to fix what is missing, and changes nothing. Progress is kept in onboarding.json in the data folder; it never contains a password, PIN or key.

If npm install fails on better-sqlite3 (it builds a small native part when no ready-made one fits your system), install the compiler tools it names (on Windows: the "Desktop development with C++" part of Visual Studio Build Tools) and run it again.

Command reference

Every command starts with friend. If you run Friend from the source code, write pnpm friend ... instead. friend on its own prints the short list. Most things can also be done inside the app (press Esc, then Settings), and a few have their own commands there (see "Commands inside the app" below).

Set up and check

| Command | What it does | Use it when | |---|---|---| | friend onboard | The guided setup in ten steps: asks what you want to use, checks this PC, offers to install what is missing (asking first), creates your account, then the model, voice, workspace, network and look. | The first time, or to finish something you skipped. It is safe to run again: it carries on where it stopped. | | friend onboard --no-install | The same, but never installs anything; it only tells you what to install. | You manage software yourself, or this is a work PC. | | friend onboard --yes | Takes every suggestion without asking (passwords and keys are still asked). | You want the defaults quickly. | | friend onboard --redo <step> | Runs one step again. Steps: welcome, features, prereqs, account, model, voice, workspace, network, look, finish. | You want to change one thing, for example friend onboard --redo look or --redo network. | | friend onboard --reset | Forgets what the wizard has done so far and starts over. Your account and settings stay. | The wizard's memory of its steps is wrong. | | friend doctor | Shows what this PC has (Node.js, Python, Docker, Ollama, Claude Code, the port) and the exact command to fix what is missing. Changes nothing. | Something does not work, or before asking for help. | | friend setup | Only creates the owner account: password, PIN, assistant name, workspace folder, API key, voice. | You want the plain version without the wizard. | | friend status | Prints the saved settings: data folder, owner, address, public URL, workspace, model, whether an API key is set. | A quick look at how Friend is set up. | | friend version | Prints the version. | Reporting a problem, or checking an update. |

Run Friend and other PCs

| Command | What it does | Use it when | |---|---|---| | friend start | Starts Friend: the web app, the assistant, the speech engine if installed, and your connectors. Prints the address (and your network address when that is on). Ctrl+C stops it. Friend's screen works in one browser window at a time: open it in a second place and that window shows "Use here", which moves Friend there and stops the first. | Every day. Open the address in a browser and sign in. | | friend pair | Prints a one-time pairing code, valid for 5 minutes, and the command to run on the other PC. | You want Friend to also work on a second PC. | | friend node --url ws://HOST:8787/device --code CODE | Run on the other PC: connects it to your Friend. Add --roots C:/work,D:/files for the folders it may use freely and --name laptop to name it. | The second step after friend pair. | | friend devices list | Lists the paired PCs and when each was last seen. | Checking what is connected. | | friend devices revoke <id> | Cuts a paired PC off at once. | A PC is lost, sold or no longer yours. |

Choose the model

| Command | What it does | Use it when | |---|---|---| | friend claude status | Shows whether Claude Code is installed and signed in, and the model and fallback in use. | Checking the Claude subscription route. | | friend claude setup | Checks that Claude Code is installed and signed in, and tells you what to fix. | You have a Claude plan and want to use it. | | friend claude setup --use | The same, and switches Friend to your Claude subscription (restart friend start). | You want Friend to answer through your plan, with no API key. | | friend ollama status | Shows whether Ollama is running, which models are installed, and which one Friend uses. | Checking the local model. | | friend ollama setup [model] | Starts Ollama if needed (in Docker when it is not installed), then downloads the model (default qwen2.5:7b). | You want a free, private model on this PC. | | friend ollama setup [model] --use | The same, and switches Friend to it. | Make the local model the main one. | | friend config set provider claude | Chooses the model by hand: claude (API key), claude-code (your subscription) or ollama (this PC). | You prefer a command to the Settings screen. | | friend config set apiKey <key> | Saves an Anthropic API key. | You use the API route. |

Voice

| Command | What it does | Use it when | |---|---|---| | friend voice setup | Installs speech (a private Python environment, Whisper for listening, Piper for speaking), asks which voice you want (female or male) and downloads about 250 MB. | You want to talk to Friend and hear it. | | friend voice setup --female | The same without the question: the first female voice (Amy). --male gives Ryan. | You know which kind you want. | | friend voice setup --voice jenny | The same with a voice by name or by its number in the list (--voice 4), or any other Piper voice name such as hi_IN-pratham-medium. | You want a specific voice. | | friend voice list | Shows the twelve voices (six female, six male), numbered, with the current one marked. | Choosing a voice. | | friend voice status | Shows whether speech is installed, which models it uses, and whether your own voice is enrolled. | Voice does not work, or you want to check. | | friend voice forget | Deletes your saved voice (asks for your PIN). Afterwards every voice is treated as yours until you enrol again. | You want to re-enrol, or switch recognition off. |

Network and port

| Command | What it does | Use it when | |---|---|---| | friend password | Sets a new sign-in password (asks twice, nothing is shown while you type). The old one is not needed, because you are at this PC, so it is also the way back in if you forgot it. Every browser and phone is signed out. | You forgot the password, or want a new one. From the screen: Esc → Settings → Security. | | friend pin | Sets a new PIN, the number Friend asks for before risky actions. | You forgot the PIN, or want a new one. | | friend daemon start | Runs Friend in the background, with no terminal window needed. Prints the address, keeps a log in the data folder (friend.log). | You want Friend always available, even after you close the terminal. | | friend daemon stop | Stops the background Friend and the speech engine with it. | Shutting it down. | | friend daemon restart | Stops it and starts it again. | After changing a setting such as friend lan on or a different model. | | friend daemon status | Says whether the background Friend is running, for how long, and at which address. Exit code 0 when running, 1 when not. | Checking, or in a script. | | friend upgrade | Updates Friend in one go: checks npm for the newest version, stops the background Friend, installs it, and starts it again. Add --force to reinstall the same version. If Friend runs in a terminal instead, it asks you to press Ctrl+C there first. (--auto is what Friend itself runs to update itself; it writes what happened to update.log in the data folder.) | A new version is out. | | friend usage [--days N] / clear | Shows the tokens Friend used, day by day (default 7 days), split by model and by where the request came from (voice, chat, scheduled tasks, messages). Only numbers are kept, never what was said; clear removes the saved counts. Counts older than a year go by themselves. | You want to see how much of your plan Friend is using. (Settings → Usage shows the same.) | | friend autostart on / off / status | Starts Friend by itself in the background when you sign in to this computer, so you never start it by hand after a restart. Windows: a hidden entry in your sign-in list; macOS: a launch agent; Linux: a user service (or an autostart entry when there is no user systemd). Only your own account's files are touched. on can be undone with off. | You want Friend always there after a restart. (Settings → Startup and updates does the same.) | | friend lan (or friend lan status) | Shows what Friend listens on and the addresses to open. | Checking, or finding the address for your phone. | | friend lan on | Lets other devices on your network open Friend. Prints the addresses and the things to know (plain http, no microphone on other devices, firewall prompt). | Using Friend from a phone or another PC. | | friend lan on --https | The same, with a secure https:// address (Friend makes its own certificate), so a phone may use the microphone and camera. | Talking to Friend from a phone. | | friend lan https on (or off) | Turns the secure address on or off by itself. | You already ran friend lan on, or want plain http back. | | friend lan on --port 9100 | The same on another port. | Port 8787 is taken. | | friend lan port 9100 | Changes only the port. | Another program uses 8787. | | friend lan off | Back to this PC only. | Closing it again. |

Restart friend start after any of these.

Your data and what Friend may do

| Command | What it does | Use it when | |---|---|---| | friend memory list | Lists the notes Friend keeps about you, numbered. | Seeing what it remembers. | | friend memory forget <number> | Deletes one note. | A note is wrong or private. | | friend memory clear | Deletes all notes (asks for your PIN). | Starting fresh. | | friend rules list | Lists the "always allow" rules you created by choosing Always allow on an approval card. | Reviewing what Friend may do without asking. | | friend rules remove <id> | Removes a rule, so Friend asks again. | You trust something less now. |

Plugins

| Command | What it does | Use it when | |---|---|---| | friend plugin catalog | Lists the plugins that come with Friend. | Looking for something to add. | | friend plugin list | Lists the installed plugins and whether each is on. | Checking what is installed. | | friend plugin install <name or folder> | Shows exactly what a plugin contains and asks before installing (add --yes to skip the question). A folder path installs your own plugin. | Adding a skill or a sandboxed tool. | | friend plugin enable <id> and friend plugin disable <id> | Switches a plugin on or off without removing it. | Pausing a plugin. | | friend plugin remove <id> | Deletes a plugin. | You no longer want it. |

Connectors (Gmail, Chrome control, any MCP service) are managed inside the app, not on the command line: Esc, then Connectors, or /connectors.

Settings by command

friend config set <key> <value> changes one setting; friend config keys prints the list with a line about each. Anything on the Settings screen can be changed there instead, with the same effect. The keys:

| Key | What it is | |---|---| | provider | Which model answers: claude (API key), claude-code (your subscription), ollama (this PC) or openai (any OpenAI-compatible server, such as OmniRoute). | | approvals | How much Friend asks before acting: manual (asks every time, the default), edits (file work and the sandboxed code runner go ahead) or auto (commands and tools go ahead too). Risky commands (they need your PIN), the camera, adding connectors or plugins, and Friend's own data folder always ask, in every mode. The AI cannot change this. Easier: Esc → Settings → Approvals. | | ownerName | Your name, shown on the profile badge (a shield in the corner) while you are signed in; it shows whoever Friend just recognized when they speak. Empty shows "Owner". | | apiKey, model | The Anthropic API key and the Claude model used with it. | | claudeCode.model, claudeCode.command, claudeCode.fallback | The subscription route: model (sonnet, haiku, opus), the command that starts Claude Code, and ollama or none for what to use when Claude is unavailable. | | claudeCode.webSearch | With Claude Code, lets Claude search the web with its own WebSearch tool (on by default), so questions about today (prices, news, scores) are answered from what it finds, with the source, instead of from memory. Only searching: Claude gets no other built-in tool, and a guest never gets it. Off: Claude answers from memory only. | | web.enabled | Gives every model the tools web.search (a list of results) and web.read (the text of one page) so it can answer about today (prices, news, scores) from the web instead of from memory. They run on this PC with no browser. A page can never give Friend orders, and Friend will not open addresses on this PC or your network. Off: neither tool exists. | | web.provider | Where web.search asks: duckduckgo (default, free, no key), brave or tavily (your own key), or searxng (your own server). If the chosen source cannot answer, Friend tries DuckDuckGo once and says so. | | web.searxngUrl | The address of your own SearXNG server (http or https, no login in it), used when web.provider is searxng. It must allow the JSON format. | | web.braveKey, web.tavilyKey | Your keys for Brave Search and Tavily. Write-only: never shown again, and an empty box leaves the saved key alone. Only the owner can change them, on the Settings screen. | | ollama.url, ollama.model, ollama.vision, ollama.contextTokens | The local model: server address, model name, whether it can see pictures, and how much it remembers at once. | | openai.url, openai.model, openai.apiKey, openai.vision | An OpenAI-compatible server: its address (ending in /v1), the model name it knows, its API key (or empty), and whether the model can see pictures. For OmniRoute (https://github.com/diegosouzapw/OmniRoute) run npm install -g omniroute and omniroute, then friend config set provider openai (the defaults are OmniRoute's: http://localhost:20128/v1 and model auto). Also works with LM Studio, LiteLLM, vLLM and OpenRouter. | | voice.hindiVoice, voice.marathiVoice | The voices that read Hindi and Marathi (Devanagari) text: hi_IN-priyamvada-medium, hi_IN-pratham-medium, hi_IN-rohan-medium, mr_IN-google-medium. Empty makes the main voice read that language. See "Hindi, Marathi and English together". | | updates.auto | Look for a newer Friend shortly after it starts and every few hours, and install it by itself while Friend is not busy (only a Friend running in the background restarts itself). Off: Friend only tells you a new version is out, and you run friend upgrade or press Update now in Settings → Startup. | | session.autoCompact | With Claude Code, Friend keeps one long conversation and re-reads all of it for every answer. On (default): when it grows past session.compactAtTokens, Friend writes short notes on it and starts a fresh conversation from them before the next answer. Off: it only tidies when you press Tidy up now. | | session.compactAtTokens | How big the conversation may grow before it is tidied up by itself, in tokens (50000 to 800000, default 100000). A fresh conversation already starts near 36000. | | voice.enabled, voice.sttModel, voice.ttsVoice, voice.language, voice.speakTyped, voice.duck, voice.duckPercent, voice.accept, voice.challenge | Speech on or off, the listening model, the speaking voice, a forced language, speaking typed replies, turning music and video down while you talk and while Friend speaks (and how far, in percent), and the two thresholds for recognising your voice. | | bind, port | Where Friend listens (easier: friend lan). | | https | true serves a secure https:// address with a certificate Friend makes for itself, so a phone may use the microphone (easier: friend lan https on). | | publicUrl, allowedOrigins, allowedHosts, trustProxy | For running behind your own domain or reverse proxy (see "Using your own domain later"). | | roots | The workspace folders Friend may use freely, separated by commas. |

Example: friend config set voice.speakTyped false keeps typed messages silent.

Environment variables

| Variable | Meaning | |---|---| | FRIEND_HOME | The folder where Friend keeps its data (default: .friend in your home folder). Use another folder to keep a separate test copy. | | FRIEND_DAEMON | Set to 1 by friend daemon start for the Friend it starts in the background. Only a background Friend may restart itself to install an update; one running in a terminal never does. You do not set it yourself. | | FRIEND_ANTHROPIC_KEY | An API key to use when none is saved. | | FRIEND_PYTHON | The Python program to use for voice, if the usual ones are not the right version. | | FRIEND_WEB_DIR, FRIEND_PLUGIN_CATALOG, FRIEND_VOICE_DIR, FRIEND_CHROME_CONTROL | Where the web app, the bundled plugins, the Python voice code and the Chrome control program (its index.js) are, for unusual installs. | | CHROME_BRIDGE_PORT, CHROME_BRIDGE_HTTP_PORT | Where Chrome control itself listens on this PC (its extension port, default 8765, and its MCP port, default 8766). Friend reaches both for you: other PCs only need Friend's own address. | | FRIEND_DEMO_PORT | The port pnpm demo listens on (default 8788). |

From the source code

| Command | What it does | |---|---| | pnpm friend ... | Runs any command above from the repository. | | pnpm demo | A scripted demo with no network and no key; open the printed address. | | pnpm test, pnpm typecheck, pnpm build | The checks and the web build. | | pnpm package | Builds the npm package in packages/friend (then npm publish from there). |

Commands inside the app

Type / in the message bar for a list that narrows as you type; they run on the screen and never go to the AI. /help opens the same list with a line about each.

| Command | What it does | |---|---| | /settings | Opens the settings. | | /look | Opens the settings on the voice presence tab (themes, colour, size, captions). | | /view minimal, /view companion, /view workspace | Chooses how much is on the screen besides the AI's pictures. | | /chat, /tasks | Shows or hides the conversation and the schedule (add on or off). | | /layout reset | Puts every panel back where it started. | | /connectors | Opens the connectors (Gmail, Chrome control, any MCP service). | | /plugins | Opens the plugins. | | /voice | Opens voice enrolment. | | /camera on, /camera off | Switches the camera. It only takes a picture when you approve. | | /mute, /unmute | Hides or shows everything on the stage. Remembered on this device. | | /quiet, /speak | Stops Friend speaking aloud (it still writes its answers, so you read the captions and the chat), or lets it speak again. Instant, no restart, remembered on this device. | | /clear | Takes everything off the screen. | | /status | Says what Friend is doing right now. | | /logout | Signs out on this screen. | | /lock | Emergency lock: ends every session and stops the agent. | | /help | Lists the commands. |

Quick start (from the source code)

pnpm install
pnpm build                 # builds the web client
pnpm friend setup          # password, PIN, assistant name, workspace folder, API key
pnpm friend start          # gateway + local device node, http://127.0.0.1:8787

Sign-in is guarded against guessing: after three wrong passwords (or PINs) from one address, that address is locked for a minute, and every wrong try after that doubles the wait (up to an hour). The sign-in screen shows the tries left and a countdown, and even the right password waits out a lock. Many addresses together are limited too. A refresh never shows any part of the home screen before sign-in: the page asks the server first and draws only the sign-in screen until you are in.

Open the address in a browser, sign in, and press any key to talk. Press Esc (or long-press the top-left corner) for the owner menu: voice on/off, mute stage, log out, emergency lock.

Preview the whole look with no API key and no network: pnpm build && pnpm demo, then open the printed address. Each message you send shows the next scene (gauges and charts, a timeline, a graph, the orb and ring voice styles, a warm mood).

Other commands: friend rules list|remove <id>, friend devices list|revoke <id>, friend status, friend config set <key> <value>, friend pair (code to connect another PC), friend node --url <ws-url> --code <code>.

Use your Claude subscription (Claude Code)

Instead of an API key you can let Friend think through the Claude Code app you already have signed in with your Claude plan. To the rest of Friend it is just another model: same tools, same approvals, same screen. Nothing else changes.

claude                          # once: sign in with your Claude account (if you have not already)
pnpm friend claude setup --use  # checks it works and switches Friend over; restart `friend start`
pnpm friend claude status

• One Claude Code process stays running, so replies start in about a second after the first message (a fresh start takes 3 to 9 seconds). Friend's conversation is remembered across restarts.
• Claude Code has no tools of its own here. Friend's tools reach it through a private local bridge (127.0.0.1, random token), and every call still goes through Friend's own approval and PIN rules and audit log.
• Unrecognized voices (guests) get a separate, tool-less, nothing-saved session.
• If your plan's usage limit runs out, Claude Code is signed out or missing, Friend answers with the local Ollama model and tells you once on screen (friend config set claudeCode.fallback none to see errors instead). Other settings: claudeCode.model (default sonnet), claudeCode.command.
• Caveats: it uses your plan's limits; using a subscription this way is a grey area in Anthropic's terms, so treat it as experimental; if Friend answered through the fallback, Claude does not know about that exchange.
• Opt-in live test against the real CLI: FRIEND_LIVE_CLAUDE=1 pnpm exec vitest run packages/server/src/provider/claude-code/live.test.ts.

Connectors (Gmail, Chrome control, any MCP service)

A connector gives Friend the tools of an outside program or service through the MCP standard. Open Esc, then Connectors (or type /connectors). Ready-made entries:

• Chrome control is a plugin that comes with Friend, so there is nothing else to install (no npm package, no claude mcp add). Open Plugins, install Chrome control (PIN), and Friend sets it up by itself from the copy inside it: the small helper Chrome needs and the extension folder. The one step only you can do is loading that folder in Chrome (chrome://extensions, Developer mode, Load unpacked; Configure shows the folder with a Copy button) and switching the extension on from its icon. Then switch the plugin on (PIN); Friend connects, and the plugin page shows whether the extension is linked. Every browser action still asks for your approval. Removing the plugin only removes Friend's connection; the helper and the extension stay, because other programs on your PC may use them. See docs/guides/chrome-control.md. From another PC it needs no extra port: switch on Allow other PCs in Configure and Friend serves Chrome control on its own address (/chrome/mcp for apps, /chrome/ws for the extension), with a download link for the extension and the key shown only after your PIN.
• Gmail: a community Gmail MCP server (@gongrzhe/server-gmail-autoauth-mcp). You create the Google OAuth credentials and sign in on Google's own page; Friend never sees your Google password. More mail and calendar connectors can be added the same way ("Something else": any program or any server by address).

How it is kept safe:

• Adding or switching one on needs your PIN, and the screen shows exactly which program will start (no shell is ever used) or which address it will reach. Plain http:// is only allowed for a server on this PC.
• Every call to a connector's tool asks for your approval (the usual card; "Always allow" works for the ones you trust). Tools appear to the AI as mcp.<connector>.<tool>.
• Secrets (tokens, keys) are typed on the screen only. They are saved in connectors.json next to the database (readable only by you) and are never shown again, never put in the AI's context, never in the audit log. The AI cannot supply or change secrets.
• What comes back is data, not orders. The AI is told that email, web pages and documents from a connector are untrusted and that it must never send or delete anything unless you asked for exactly that.
• A connector that stops working shows why; it is started again on the next use. Friend does not wait for connectors when it starts.
• Turning one off removes its tools at once. Removing deletes its saved secrets.

The AI can manage connectors too. Ask it to "connect X", "switch Gmail off", "reload Chrome control" or "remove it". Adding or switching on shows the same PIN card with the exact program; turning off and reloading need nothing; removing asks. Opt-in live check of the real chrome-control: FRIEND_LIVE_CHROME=1 pnpm exec vitest run packages/server/src/connectors/live.test.ts.

The AI writes its own skills and plugins

When it notices something you repeat, the AI may suggest a skill (instructions it loads later) or a plugin tool (a small python, node or shell program that runs in the code sandbox: no network, no access to your files, approval on every call). If you agree it calls plugins.create: the approval card shows everything it wrote, untruncated, and needs your PIN. It is then checked like any plugin, switched on and loaded straight away (plugins.reload re-reads everything; the Plugins panel has a Reload plugins button too). Creating the same id again installs a newer version. The AI can also list, switch off or on, and delete plugins.

Commands

/help opens the list of every command with what it does.

On-screen status, clock and settings

• Status line (bottom-left): always shows what Friend is doing in plain words: Ready, Listening, Understanding what you said, Thinking, Getting the weather for Pune, Creating a view, Running a command, Answering, Waiting for your approval, Reconnecting. It never shows command text or arguments.
• Clock (bottom-right): a small date and time, so the screen is never empty.
• Settings (Esc, then Settings): change the model provider (Claude subscription through Claude Code, Claude API key, or local Ollama), the model names, the fallback, the Ollama address, and the voice options without the terminal. A change is used straight away (no restart) except the speaking voice and listening model. Saving needs your PIN. The API key is write-only: it is never sent back to the screen. Network address, folders and safety rules are deliberately not in this screen; use friend config set for those.
• Slash commands: type / in the message bar for a list that narrows as you type (arrows or Tab to pick, Enter to run): /settings, /plugins, /voice, /camera on|off, /mute, /unmute, /quiet, /speak, /clear, /status, /logout, /lock, /help. They run on the screen and never go to the AI, so they work instantly and even when the model is down. Start a line with // to send a message that begins with a slash.
• PIN with the keyboard: wherever a PIN is asked for, type the digits, Backspace to correct, Enter to confirm (Enter never deletes your voice; that needs a click). The on-screen keys still work for touch.
• The AI knows the screen: its instructions include a layout guide (what is always on screen, the five regions, how to compose one answer, how long things should stay), and every request tells it what is on screen, where, for how long, and which regions are free. If it puts something on top of another object or over the captions, status line or clock, Friend moves it to a free spot (or the tidy auto grid) and tells it why, so it learns within the conversation. Nothing is refused.
• Voice presence look (Settings, tab "Voice presence", or /look): the settings dialog is wide and has tabs: Model, Voice and Voice presence. On the last one you choose a theme (Aurora, Ocean, Ember, Violet storm, Matrix, Rose, Ghost) with one click, or set everything yourself: the shape (sphere, orb, ring), the colour (swatches or any colour), brightness, size, how strongly it reacts to the voice, the soft glow behind everything, whether the live captions show and how big they are, and the view. A live preview shows the result before you save. Saving a change of look needs no PIN, and every open screen changes at once. Let the AI change the shape, colour and brightness to suit a topic can be switched off to keep exactly the look you chose (you can still ask it "make it purple" yourself). Pressing the colour box also works; the colour must be a plain #rrggbb.
• Views and panels: Esc, then View (or /view minimal|companion|workspace, or Settings) chooses how much is on the screen besides the AI's pictures. Minimal: only the voice presence (this hides the status line and the clock too; the Esc menu's View button never switches to it by accident, only Settings or /view minimal do, and Friend tells you once when the view is Minimal). Companion: plus the status line and clock. Workspace: also a Chat panel (what was said and answered, with a box to type in) and a Schedule panel (open tasks and reminders, soonest first, late ones in red). Turn any panel on or off from the Esc menu or with /chat, /tasks. Drag a panel by its title bar (or the status line and clock anywhere on them) to move it; arrow keys nudge it, double-click puts it back, /layout reset puts all back. Panels remember their place in this browser. Changing the view starts from that view's own set of panels.
• The AI can change settings too: ask it to switch the model, the fallback or the voice options, or to show a cleaner or fuller screen. Changing settings is a PIN-approval action (the same card as for any risky action); changing only the view needs no PIN. It can never change the API key or the Claude Code command (those stay on the settings screen), and a change of model takes effect after its current reply. The AI is also told which panels you have open and keeps its pictures out of their way.

Voice (local, optional)

pnpm friend voice setup     # one time: private Python environment + speech models (about 250 MB)
pnpm friend start           # starts the speech engine too

Hold Space (or long-press the screen) and speak; release to send. Friend answers in captions and aloud, and the sphere follows the real audio. Pressing Space while it talks interrupts it. Typed messages are answered silently.

• Female or male voice. Twelve voices to choose from, six female and six male, American, British and Scottish (Amy, Kristin, Hannah, Jenny, Alba, Kathleen; Ryan, Joe, Norman, Henry, Alan, Tom), plus the original voice (Lessac), which stays the default until you choose. You are asked at friend setup ("female or male") and again at friend voice setup (a number, female or male; or skip the question with --female, --male or --voice jenny). friend voice list shows them. Later, in Settings, Voice tab, pick one from the list and press Hear it to listen before you decide; the change needs no restart: the voice loads in the background (the first time a voice is used it downloads about 60 MB, then it works offline), you get a message when it is ready, and if it cannot be loaded Friend goes back to the previous voice instead of going silent. Any other Piper voice works too: friend voice setup --voice hi_IN-pratham-medium.
• Everything runs on your PC: speech to text with faster-whisper, text to speech with Piper. Recordings are never stored or sent anywhere else.
• Needs Python 3.10 or newer. friend voice status shows the state; friend config set voice.sttModel small (or voice.language hi, voice.enabled false) changes it, then run friend voice setup again for new models. The speaking voice is chosen as described above.
• Recordings are limited to 90 seconds and 4 MB. While Friend is busy with a request, a new recording is refused (409) instead of being dropped silently. Emergency lock also cancels a recording that is still being transcribed.
• The browser needs microphone permission and a secure origin (localhost, or HTTPS through your domain).
• Until you enroll your voice, anyone who can press Space on a logged-in browser speaks as you. The PIN still guards risky actions.

Only your voice commands Friend (speaker verification)

Press Esc (or long-press the top-left corner), choose Enroll my voice, read the phrases aloud and enter your PIN. Friend keeps only an averaged 256-number voiceprint, encrypted on disk (AES-256-GCM, key in voice.key next to the database). Recordings are never kept. After that every spoken request is scored against your voiceprint:

| Match | What happens | |---|---| | Your voice (0.72 or higher) | Runs as you, as before. | | Close, or too short to tell (0.58 to 0.72) | Nothing runs. A card asks "Was that you?"; the right PIN runs the words, a wrong PIN (3 tries), Dismiss or 60 seconds drops them. | | Another voice (below 0.58) | A guest: polite answer, no tools, none of your conversation or screen, nothing saved. You get a warning with what was said. |

• friend voice status shows whether a voice is enrolled; friend voice forget deletes it after the PIN (so does the enrollment panel). friend config set voice.accept 0.75 and voice.challenge 0.6 tune the two thresholds.
• Family voices. Once your own voice is saved, press Esc, then Family voices, to add your wife, son, daughter or anyone at home (up to eight people). Hand the device to them: they read a few sentences, you enter your PIN. From then on Friend knows who is talking, calls them by name, and keeps a short conversation with each of them. A recognized person can chat, get help, ask for the weather and have pictures shown. They cannot use your PC, files, camera, messages, settings, notes or schedule, never see your private things, and nothing they say is stored or logged. If two voices sound too alike to tell apart, Friend asks for your PIN instead of guessing. You can remove a person any time (PIN needed).
• Emergency lock drops any pending challenge and any enrollment in progress.
• A voice match is a convenience, not a lock: a recording or a cloned voice can fool any speaker model. That is why a borderline match needs the PIN and risky actions always ask for it.
• Python tests for the speech engine: cd packages/voice && python -m venv .venv && .venv/Scripts/pip install -e .[dev] && .venv/Scripts/python -m pytest.

Hindi, Marathi and English together

Speak or type in English, Hindi, Marathi, or a mix of them in one sentence ("आज का weather बताओ"). Friend answers in the language you used.

• Understanding you. Leave voice.language empty so each request is understood in the language you spoke. When it hears Hindi or Marathi it listens a second time with a hint that writes those in Devanagari and keeps English words in Latin letters. The default listening model (base) mishears Hindi and Marathi a lot: use friend config set voice.sttModel small (about 460 MB, a bit slower), run friend voice setup once, and restart.
• Answering you. The assistant is told to write Hindi and Marathi in Devanagari and English in Latin letters. Friend then says each piece with the right voice: Hindi with voice.hindiVoice (Priyamvada by default, or Pratham or Rohan), Marathi with voice.marathiVoice, English words with your main voice. Hindi or Marathi is told apart by the words used. The first time a voice is used it downloads about 64 MB (Settings → Voice picks them and starts the download). If one cannot be loaded, the main voice reads that language instead of staying silent.
• Typed text works the same: Devanagari in, Devanagari out, spoken with the Hindi or Marathi voice. Roman-letter Hindi ("kya haal hai") is understood by the model but is read by the English voice, so ask for Devanagari when you want it spoken well.
• Notes: the Hindi voices Priyamvada and Pratham are free for personal use but not for commercial use. A small local model (Ollama) writes Hindi and especially Marathi much worse than Claude does.
• Other languages are not set up with voices yet: the assistant may answer in them, but they are spoken by the English voice.

Your screen: it knows what it draws on, and you can keep widgets

• Friend knows the screen. Your browser tells Friend the size and kind of its screen (phone, tablet or desktop, portrait or landscape) when it connects and whenever you resize or turn it. The assistant lays things out for that screen: on a phone it shows one or two short pictures in a column, on a wide desktop it uses the whole stage. It also knows which view you chose and which pictures and widgets are showing.
• Temporary pictures and widgets. What Friend shows while it works is temporary: it takes it off the screen when the task is done, and the screen goes back to how you keep it. To keep something, press the pin on its corner, or ask "keep this on screen". A pinned widget is a widget: it stays across refreshes, signing out and in, and restarts, in the place you gave it. "Clear the screen" leaves your widgets alone; the × on a widget removes it for good (the assistant never removes or unpins your widgets unless you ask).
• Move and resize. On a computer or tablet, drag an object by the bar at its top and resize it by its bottom-right corner (or focus the bar and use the arrow keys; hold Shift to resize). Objects stay inside the part of the screen that is free of the menu, captions, status line and clock.
• The profile badge. A shield in the corner shows your name (friend config set ownerName "Your name", or Settings → Voice presence) while you are signed in. When Friend recognizes a family member's voice the badge shows their name; an unrecognized voice shows Guest. Drag it where you like; it is hidden in the Minimal view.

Showing files, and looking at its own screen

• Widgets. Besides card, stat, gauge, list, table, code, chart, timeline, graph and sandbox, Friend can show markdown (formatted text with tables), progress bars, checklists, key-value lists, a countdown, image, video (sound too), pdf and a gallery of pictures and clips. A group puts several of them into one object (a dashboard) and tabs put several views into one, so it does not need a card per fact.
• Your files. Ask "show me C:\Users\me\Pictures\trip.jpg" or "open my notes.md". Friend shows images and GIFs, video and sound, PDFs, Markdown, text, CSV (as a table) and code. Every file needs your PIN, and the approval card shows the full path. The file is served only to your signed-in browser, under a random address; nothing inside Friend's own data folder can be shown. A web address can never be used as a picture or clip, so a page or file cannot make Friend send data out through an image link.
• Looking at its own screen. The assistant can take a picture of the Friend screen to check its layout, after you enter the PIN, with no plugin and no screen-sharing prompt. It needs a model that can see pictures. Sandbox pages, video and PDFs appear empty in that picture. Pictures are never kept in the conversation.
• The assistant has a manual for the screen. It can read it any time (the stage.guide tool): every widget with its props, how to arrange them, the theme, files and ready recipes. It also knows what to do when the screen already has things on it: a short explanation goes on top as an overlay (it floats over a dimmed screen, moves nothing underneath, and leaves by itself after 30 seconds, or when you click away); something important clears the screen first (your pinned widgets stay) and is shown alone; a new topic replaces the old one; more of the same topic updates what is there.
• The theme. Friend is a HUD: a near-black blue background, glass panels with a thin glowing edge, one accent colour, small letter-spaced labels and monospace numbers. The assistant is told the exact colours and your current accent, and the pages it draws in a sandbox use your colours automatically (var(--accent), var(--ink) and so on).

Telegram, WhatsApp and SMS (plugins)

These are plugins: Esc → Plugins. Telegram and WhatsApp and SMS (Twilio) are listed under Available; press Install (PIN). Once installed they move to Installed with Configure (the steps left, and the form for the token, numbers and linking), Turn on / Turn off and Remove (which forgets the tokens). You can also just ask Friend: "set up Telegram for me". It can install the plugin (you approve with your PIN), tell you exactly which step is next, set what is not secret (a phone number, the on/off switch) and open the Configure page on your screen. A token or password is only ever typed by you, on that page. Two things come out of it: you can talk to Friend from your phone, and Friend can message people for you and read their replies.

• Telegram needs no public address. Create a bot with @BotFather (/newbot), paste the token, switch it on and save. Then press Get a link code (needs your PIN) and send /link CODE to your bot from your own Telegram. That chat is now yours: what you write there is answered like on your PC, in the same chat.
• SMS and WhatsApp use one Twilio account (account SID, auth token, your Twilio SMS number and WhatsApp sender, and your own phone number). Twilio delivers incoming messages to https://your-public-address/api/channels/twilio (set it in Twilio as the "a message comes in" webhook, HTTP POST), so Friend needs a public address (friend config set publicUrl https://…). Every request is checked against Twilio's signature, and a message Twilio sends twice is run once.
• Who counts as you. Only your linked Telegram chat and your own phone number. Everyone else who writes to your bot or number goes to an inbox as untrusted mail: nothing runs, nothing is answered, you get a notice on the screen, and you can ask Friend "did anyone reply?" (it reads them as data, never as instructions).
• Sending. "Message Anu on WhatsApp that I am late" uses the same approval rule as everything else (Settings → Approvals): in manual you read the recipient and the text and approve; in auto it goes without asking. Risky actions asked for from a phone are still decided on the PC.
• Safe by design. Tokens are write-only (never shown again, never sent to the screen), the AI cannot change channel settings, one sender cannot flood you (limits per sender and per minute), and a link code works once, for ten minutes, and dies after five wrong guesses. Calls are not built yet.

Portfolio and brokers (Zerodha)

A plugin: Esc → Plugins → Zerodha → Install (PIN) → Configure. Friend reads your Zerodha account through Zerodha's Kite Connect API and answers "how is my portfolio doing?", on screen or by voice, or sends a report to Telegram or WhatsApp at the times you choose (for example 9:30 and 3:45 on weekdays). The full walk-through, including the first live login, is in docs/guides/zerodha-setup.md.

• Read only. Friend can look at holdings, positions, funds and orders. It cannot place, change or cancel orders, never tells you what to buy or sell and never predicts prices. Settings → Trading shows Read only; the order levels are shown as "later version" and are locked. The assistant cannot change the broker settings or this level, even with your PIN.
• Cost. Nothing extra: no third-party service, Friend talks to Zerodha from your PC. Zerodha's free Personal API plan covers everything used here (live streaming and history are on their paid plan, which Friend does not need).
• Login. You sign in on Zerodha's own page, once a day (Zerodha ends the login about 6 am). Your password and 2FA never reach Friend. The API key and secret are write-only; the day's token is kept in a file only you can read. If the login has ended when a report is due, you get the login link on your phone instead, and the report follows once you sign in.
• Reports. Add them on the Configure page (PIN) or ask in chat. Currency sign and word, number format (en-IN, en-US, de-DE...), warning levels, cache time and limits are settings under Report options. Every figure is worked out by Friend from your account, never written by the AI model; a restart never sends one twice. Time zone, market-holiday dates and the login reminder (minutes before the first report) are on the same page. They are also settings: brokers.timezone, brokers.skipDates, brokers.loginReminderMinutes, and trading.level (only read), all changed from the Settings screen.
• Privacy. The figures you ask about pass through the AI model you chose (Claude from Anthropic, or Ollama on this PC); a report goes to the messaging app you pick (Telegram or WhatsApp). The audit log records that a report was made, never the amounts.
• More brokers. A broker is one file implementing the BrokerProvider contract plus the shared contract tests (runBrokerContract); reports, tools and screens do not change.

Alexa (plugin)

A plugin: Esc → Plugins → Alexa → Install (PIN) → Configure. You make a small private skill in your own Amazon developer account (free, left in development mode, never published) and Friend is its server, so "Alexa, ask jarvis assistant how my portfolio is" is answered by Friend, aloud and on the Friend screen if one is open. No AWS account and no paid plan. The full walk-through is in docs/guides/alexa-setup.md.

• What it does. Fixed questions (portfolio, weather, tasks, status) are answered straight from Friend's tools. Open questions go to the assistant: start with one of about 30 words ("how", "what", "tell me", "remind me", "can you", "find", "check", "explain", "calculate", "translate"...) and then say anything. By voice the assistant can look things up, answer questions and add reminders; anything beyond that (messages, files, commands, settings, email) is refused or asked on your phone first. English (India) and Hindi (India); Marathi is not available on Alexa.
• What it cannot do. Alexa cannot make an Echo speak on its own (use an Alexa Routine for a scheduled spoken report), and a slow answer is held ("Still working. Ask me if it is ready.") because Alexa waits only about 8 seconds.
• Safe by design. Every request is checked as really from Amazon (certificate, signature, 150-second timestamp), for your skill, from an Echo you allowed with your PIN. The voice can only read: anything that would ask you is sent to your Telegram or WhatsApp for a YES (or skipped), and anything needing your PIN is refused. The assistant cannot change Alexa settings.
• Configurable. Skill name, languages, answer and permission times, request and Echo limits, and extra tools a voice may use are settings on the Configure page (Advanced), with safe ranges; nothing is tied to one person's setup.
• Setup. Friend needs a public HTTPS address with a real certificate (friend config set publicUrl https://your-address). The Configure page shows the endpoint, lets you download the skill files (manifest and English and Hindi models) to paste into the console, and lists your Echo devices. If your address changes later, paste the new endpoint into the console again; nothing else changes.

Music player and screen widgets

Friend's screen has a media player and four widgets drawn in a HUD style (angled corners, thin lines, your accent colour): a profile tag, a date and time, quick-launch buttons and the music player. Esc → Settings → Widgets (PIN) sets them all.

• Music. Pick a folder on the PC that runs Friend (sub-folders are searched; the folder is checked when you save). Friend reads the tags and cover art from the files (free, no account) and plays them in the browser, with a playlist, shuffle, repeat, volume and search. The file types, the most tracks, how deep to search, the start volume, repeat, shuffle and the moving bars are all settings. Files are only read from inside the folder; links that point out of it are not followed.
• Spotify (plugin). Connect your own Spotify account with OAuth (Plugins → Spotify): the assistant and the player can then search and play from it next to your folder, and the player's Library tab gets a Spotify choice. You sign in on Spotify's own page; Friend never sees your password. Playing needs Spotify Premium on your account and a free app of your own at developer.spotify.com (the guide walks through it: docs/guides/spotify-setup.md). Any plan or approval a service asks for is on your own account; Friend never pays for or shares one. JioSaavn is not supported: it has no public API and its policy forbids unlicensed access.
• SoundCloud (plugin). The same for your own SoundCloud account (Plugins → SoundCloud): you register a free app of your own (SoundCloud decides who gets one), paste its client id and secret (the secret is write-only), and sign in on SoundCloud's own page. Tracks stream through Friend to the screen, or play from the PC through mpv. See docs/guides/soundcloud-setup.md.
• Files and pictures to your Telegram or WhatsApp. Ask Friend to send you a photo, a clip, a song or a document that is on the PC (message.send_file). You approve every file after seeing its name, every time. Telegram takes up to 50 MB; WhatsApp up to 16 MB and needs Friend's public address (Twilio fetches the file from a link that works for ten minutes). Only to you, never as SMS; programs (.exe, scripts) and Friend's own data folder are never sent.
• Voice messages in Telegram and WhatsApp. Send Friend a voice message from your own linked Telegram chat or your own WhatsApp number and it is understood and answered like a typed one (the reply starts with what it heard). It uses the same voice engine as the PC microphone, so that must be running; if not, you are told. Up to 5 minutes or 8 MB; from anyone else voice messages are ignored.
• Audius (plugin). Free music from Audius, an open music service: no account, no key, no payment. Install it from Plugins and switch it on; the assistant and the player can then search and play it, on the screen or from the PC. See docs/guides/audius-setup.md.
• Web video (plugin). Ask Friend to play a video or a song from YouTube or almost any video site: "play lofi hip hop on YouTube", or give it a link. It plays inside Friend, in a card on your Friend screen (a browser, also a phone) or in a window on the PC, as you choose in the plugin's Configure page; Friend can pause, jump to a time, change the volume and stop it, and it never opens YouTube in your Chrome. Nothing to install: when you turn the plugin on, Friend downloads the two free programs it needs (yt-dlp and ffmpeg) by itself from their projects' own release pages, checks them against the published checksums, keeps them in its own folder and keeps yt-dlp up to date. (mpv is optional, for a window of its own on the PC; without it a browser window is used.) Safety: only public http and https links, never an address on your own network, a limit on length and an optional list of allowed sites; nothing is saved to your disk. See docs/guides/web-video.md.
• Play from the PC's speakers. Settings → Widgets → Where the sound comes out: the screen's browser (default) or the PC that runs Friend, with no browser open. The folder's music is played by mpv (free; "Check mpv" finds it or shows the install command for your system) and Spotify on a Spotify device you name. The screen then shows what the PC plays and controls it. See docs/guides/host-output.md.
• Control by the assistant. Say or type "play something by ...", "next", "turn it down", "shuffle", "what's playing". The assistant can play, pause, skip, seek, change volume, shuffle, repeat and edit the queue, and search the library; it cannot change the folder or any widget setting. Over Alexa the music tools are off unless you add them to the voice tools yourself.
• Quick launch. Up to 12 buttons, each opening an address (http or https only), asking the assistant something, opening a panel (chat, tasks, plugins, settings, music) or controlling the music.
• Date and time, profile. 12 or 24 hour, with or without seconds, in any date language; the profile tag shows the assistant's name, a label you choose, and who Friend is talking to.
• Animations. Tabs slide and light up on hover, ripple and flicker "ENGAGED" when pressed; while music plays the dial's needle sweeps, the bars follow the sound and the cover turns. Settings → Animations can reduce them or turn them off, and the device's own "reduce motion" is always respected. Esc → Music and Quick launch hide or show them; /music and /launcher do the same.

Pictures first: weather, stats and graphics

Friend is meant to show answers, not only say them. It controls the whole screen and picks the object that fits (gauge, chart, table, timeline, graph, or its own animated page).

• Weather: "what is the weather in Pune?" gets live data from Open-Meteo (no account or key) and puts temperature, humidity, wind and a 24-hour chart on screen.
• Computer stats: "how is my computer doing?" puts CPU load, memory and disk gauges, uptime and CPU on screen.
• For these two, Friend calls the tools itself as soon as the request is clear (a named city, or a computer word plus a status word), so the graphs appear immediately even with a small local model; the model then only says a short summary. Asked together, both sets stay on screen. Anything else is decided by the model, which is told never to invent numbers and never to announce an action without doing it (if a small model does, Friend asks it once to actually call the tool).

Voice replies

Friend answers aloud to both spoken and typed messages (friend config set voice.speakTyped false to keep typed ones silent). The voice follows the text: each sentence is spoken as soon as it is written, and code blocks are never read aloud.

Browsers keep sound blocked until the page has had a click, tap or key press. Friend unlocks sound on the first one anywhere on the page, and shows a one-time hint if sound is still blocked. Needs friend voice setup once, and a restart of friend start afterwards.

Memory

Tell Friend "remember that my sister is Priya" and it keeps a short note. The 20 newest notes are in front of the model in every conversation; older ones are found on demand ("what do you know about Priya?").

• Up to 200 notes of 500 characters each, stored in the local database. When full it refuses and asks you to forget something; it never drops notes silently.
• You control all of it: friend memory list, friend memory forget <number>, friend memory clear (asks for the PIN). You can also just say "forget note 7".
• Notes are treated as data about you, never as instructions, so text inside a note (or text a web page tricked the agent into saving) cannot give the agent orders. Guests and unrecognized voices never see your notes. The agent is told never to save passwords, PINs or keys.

Plugins

Plugins add skills (instructions the agent loads when they fit, for example "write my stand-up") and tools (small programs the agent can call, for example an exact unit converter). Three examples are bundled in plugins/: unit-converter, standup-writer, rubber-duck.

• Manage them from the screen (Esc, then Plugins) or the terminal: friend plugin catalog | list | install <name|folder> | enable <id> | disable <id> | remove <id>.
• Installing shows what is inside and needs your confirmation (the PIN in the browser). Switching a plugin on needs the PIN too; switching off and removing do not.
• A plugin's tools can never be more powerful than the code sandbox: they run in a throwaway Docker container (no network, no access to your files, time and memory limits), get their arguments as /work/args.json, and ask for your approval on every call. Skills are text only: they give the model words, never power, and cannot override your rules or the approval system.
• Plugins are validated strictly (lowercase ids, size and file-count limits, no links, every file must be inside the plugin folder) and re-validated each time Friend loads; a broken plugin is switched off with a visible reason.
• Make your own: a folder with plugin.json plus the files it names, then friend plugin install ./my-plugin. See docs/superpowers/specs/2026-10-08-friend-plugins-design.md for the format. Downloading plugins from the internet is deliberately not supported yet (it needs signed packages to be safe).

Running code safely (Docker)

The agent can write a program (Python, Node or shell), run it, and read the result: calculations, data crunching, quick experiments. It needs Docker Desktop running; without it the tool says so and nothing runs. (Windows Sandbox is not needed, so Windows Home works.) The first use of each language downloads a small image (about 50 MB), so that one run needs internet; after that it works offline.

Every run asks for your approval, and the card shows the whole program. Each run is its own throwaway container:

• --network none: no internet and no DNS; it cannot send anything anywhere.
• It sees none of your files: only its own code is mounted, read-only. The root filesystem is read-only; only a 64 MB scratch /tmp is writable.
• Not root (user 65534), all capabilities dropped, no-new-privileges, 512 MB memory (no swap), 1 CPU, at most 128 processes.
• A time limit (10 s by default, 60 s at most); after that the container is killed and removed. Output is cut at 20,000 characters.

pnpm test includes a check against real containers that runs hostile code (network attempts, writes outside /tmp, memory hog, endless loop, fork bomb) and confirms each is stopped.

Free-form pages (sandbox)

When none of the standard objects fits, the agent can draw its own small page on the stage: a custom visualization, a calculator, a timer, a game. It is HTML, CSS and JavaScript written inline (up to 100,000 characters) and runs in an isolated frame:

• sandbox="allow-scripts" without allow-same-origin, so the page has an empty origin: it cannot read your session, storage or the app, cannot call Friend's APIs, navigate the window, open popups or submit forms.
• A fixed Content-Security-Policy placed before anything the agent wrote: default-src 'none' (no network of any kind, no external scripts, fonts or frames), inline script and style only, images only as data: URLs.
• The app never listens for messages from the frame, and every object has a dismiss (×) button.
• Checked in a real browser: the demo's page tries fetch, parent.document and localStorage and reports each as blocked (pnpm demo, fifth message).

Tasks and reminders

Say "remind me in 30 minutes to call the bank", "remind me tomorrow at 5pm to send the report", "every Monday at 9 remind me to back up" (hourly, daily or weekly repeats), or just "add milk to my list". "What is on my list?" shows your open tasks; "done with 3" or "remove 4" tidies up.

• A heartbeat inside the server checks every 15 seconds. When a reminder is due, an alert appears, a red card shows on the stage for 90 seconds and Friend says it aloud (when voice is set up).
• A reminder only notifies you. It never starts a conversation with the model and never runs a command on its own, so nothing happens on your PC while you are away.
• If no screen is open (or Friend is mid-conversation) the reminder waits and is shown as soon as you are back. A repeating task skips any slots it missed instead of replaying them.
• Times need no special format: Friend knows your local time and converts. Up to 200 open tasks.

Camera (owner-controlled)

Ask "what is this?" while you hold something up to the camera, and Friend can look. It works one picture at a time:

1. The camera is off until you turn it on: press Esc, choose Camera: off (it becomes Camera: on). The switch is not remembered; a reload turns it off again.
2. When the agent wants to look it calls camera.snap, and an approval card appears: "Take one picture with the camera and show it to the AI model". There is no "Always allow" for the camera.
3. After you approve, the browser opens the camera, takes one frame (JPEG, longest side 1024 px), closes the camera and uploads the frame. While the camera is open a red Camera on badge with a small live preview shows in the corner; the agent cannot hide it.

• The picture is shown to the model for that turn only. It is never written to disk, never saved in the conversation history (history keeps "[camera picture, not kept]"), and the audit log records only its size. Guests can never use the camera.
• Emergency lock, logout, switching the camera off, or closing the page cancels a pending request and releases the camera.
• With Claude as the provider the picture is sent to Anthropic's API. With Ollama it stays on your PC, but only a vision model can use it: ollama pull llava (or another vision model), then friend config set ollama.model llava and friend config set ollama.vision true. Note a vision model may not support tools, so Claude is the practical choice today. Without vision the tool says so befor