kolaby

v0.4.0

Published

18 hours ago

Pick an element in the browser and send its exact file:line, props and screenshot to your AI coding agent

0High
0Medium
0Low

derlys.f

vite-plugin nextjs react devtools mcp claude-code ai element-picker inspector

kolaby

Pick an element in the browser and hand your AI agent the exact file:line, the component's props, the render tree, and a screenshot.

It removes the three manual steps: take a screenshot, find the file, describe where it is.

Alt + hover  ->  highlights the element: CtaButton › button · src/CtaButton.tsx:3:5
click        ->  captures screenshot + metadata
type         ->  "make this button green and smaller"
Send         ->  the agent edits the file, HMR reloads the page

You choose where the work happens:

| Mode | Where the work shows up | What you do | |---|---|---| | Deliver (recommended) | In the Claude Code you already have open — the panel follows along | Pick the ● session and hit Send | | Listening | In that open conversation, dedicated to it | Type once: "call wait_for_selection" | | Automatic | In a fresh claude -p, with a live log in the panel | Pick "New session" | | Manual | In the agent, whenever you ask | Type "look at the element I selected" |

How it works

| Piece | What it does | |---|---| | Vite plugin | In dev, stamps data-kolaby-loc="src/File.tsx:line:column" onto every DOM tag in your JSX and serves the overlay. Never runs in a production build (apply: 'serve'). | | Overlay | Hover highlights, click captures. Screenshot via foreignObject + canvas, no external dependency. Component tree read from the React fiber. | | MCP server | Exposes get_selection and wait_for_selection to Claude Code, Codex, or any MCP client, returning text plus images. | | Runner | Automatic mode: launches claude -p in the session you pick and streams progress to the panel over SSE. | | Service | Everything behind Send: writes the selection, snapshots files for undo, and decides who handles it. Vite and Next share it. |

The bridge between the dev server and the MCP server is the .kolaby/ folder in your project root — separate processes, no socket.

Only DOM tags (div, button, ...) get stamped. React components don't: the attribute would become a prop and pollute the component's API.

Install

npm i -D kolaby

In your React project's vite.config.ts:

import { defineConfig } from 'vite'
import react from '@vitejs/plugin-react'
import { kolaby } from 'kolaby'

export default defineConfig({
  plugins: [kolaby(), react()],
})

Put kolaby() before react(). It runs with enforce: 'pre' and stamps the raw JSX.

Add to your project's .gitignore:

.kolaby

Next.js

Next has no equivalent of Vite's transform hook, so the stamping runs as a webpack pre-loader (and as a Turbopack rule, since Turbopack reads webpack loaders).

// next.config.mjs
import { withKolaby } from 'kolaby/next'

export default withKolaby({ runner: 'claude' })({})

Two pieces are yours to add, because Next won't let a plugin inject a script or a route:

// app/layout.tsx
import { Kolaby } from 'kolaby/next/component'

export default function RootLayout({ children }) {
  return (
    <html><body>
      {children}
      {process.env.NODE_ENV === 'development' && <Kolaby />}
    </body></html>
  )
}

// app/kolaby/[...route]/route.ts
import { createRouteHandler } from 'kolaby/next/route'

const handler = createRouteHandler({ runner: 'claude' })

export function GET(req: Request) { return handler.GET(req) }
export function POST(req: Request) { return handler.POST(req) }
export const dynamic = 'force-dynamic'

The folder is kolaby, no underscore. In the App Router a folder starting with _ is private and never becomes a route — which is why the Next base is /kolaby and not /__kolaby.

Export GET/POST as named functions. Next reads the exports of route.ts statically and won't see export const { GET } = ....

Queue, history, undo and redo behave identically in both adapters: the logic lives in a single service.

example-next/ is a working app:

cd example-next && npm install && npm run dev

Deliver mode (recommended)

The request arrives as a message inside the conversation you already have open. You type nothing beforehand, and the answer comes back in your terminal, not the browser.

What each agent needs is different. Only Claude Code needs anything wired up:

| Agent | To receive a selection in an open conversation | |---|---| | Claude Code | kolaby({ runner: 'claude' }) in your config, and the MCP server registered — that is how the session fetches the screenshot | | Codex | nothing. Just have a conversation open in the project | | opencode | a server running (opencode serve); see below |

Any other MCP-speaking agent still works, through listening mode.

Then pick the element, choose the session marked ●, and hit Send. In your terminal:

> Message from @courier: Browser selection: CtaButton at src/CtaButton.tsx:3:5 - make this button teal

That Claude Code calls get_selection, pulls the screenshot and the file:line, and edits.

How it works: the dev server launches a throwaway claude -p that acts purely as a courier — all it does is call SendMessage targeting the session you picked. The real work happens in your conversation, not in a hidden process.

Codex works the same way, by a different route. codex queue --thread <id> --message <text> hands the request to the app-server daemon, which drops it into the conversation you have open. No courier process, and nothing to paste beforehand.

opencode works the same way too, over HTTP. POST /session/:id/prompt_async drops the request into the conversation and returns immediately.

opencode has to be running as a server. The TUI on its own does not open one — verified on 1.18.25, where the process holds no listening socket at all, whatever the docs say. So:

opencode serve                          # in your project
opencode attach http://127.0.0.1:4096   # optional: the TUI, as a client of it

Sessions live in a shared database, so a server started in the project also sees conversations you began in a plain TUI there.

You do not have to pin a port. kolaby asks the operating system which ports the opencode processes have open, then asks each one for its sessions — and each session carries its own directory, so several instances open at once sort themselves out.

Because the message travels on a command line, the full report would not fit — Windows caps it at 8191 characters. So the report is written to .kolaby/entries/<id>/request.md and the message points at it, with the file:line and your request inline.

The panel still follows along. Each session's history is an append-only .jsonl file; the dev server records its size at delivery time and tails whatever comes after. You see the same progress in the browser that shows up in the terminal: Codex records the same way, in ~/.codex/sessions/, and marks the end of a turn explicitly with task_complete — Claude Code has to be inferred from the silence.

example-78 working
  delivering to example-78...
  > mcp__kolaby__get_selection
  > Edit CtaButton.tsx
  Applied pink. src/CtaButton.tsx:5 — background: '#db2777'.

Tailing stops on its own after a few seconds of silence from the session.

Listening mode

This is what makes the work land in the terminal you already have open, with all the context of that conversation.

call wait_for_selection and do whatever the selection asks; repeat

The session parks and waits. In the browser panel it shows up as:

▶ example-6b - waiting now

Pick it in the selector, hit Send, and the request lands in that conversation — with the screenshot, file:line, and component. It edits, HMR reloads the page, and the answer comes out in your terminal, not the browser.

wait_for_selection hands control back when time runs out (timeoutSeconds, default 120, max 600) and tells the agent to call again. That keeps a session from parking forever.

Works with any MCP-speaking agent. The panel shows which tool is waiting.

For Claude Code, Codex and opencode, prefer deliver mode. Listening means typing that line once per session; delivery means typing nothing at all.

Automatic mode

import { kolaby } from 'kolaby'

export default defineConfig({
  plugins: [
    kolaby({ runner: 'claude' }),
    react(),
  ],
})

Needs claude on your PATH. No MCP server and no open session required — the dev server calls the CLI directly.

Heads up: the default is permissionMode: 'acceptEdits'. The agent edits files without confirming, triggered by a click in the browser. That's the point of automatic mode, but know what you're turning on. Use permissionMode: 'plan' if you want the plan without the edit.

One run at a time: sending again while the agent works returns "agent still busy".

The agent is launched by the executable's absolute path, with no intermediate cmd.exe — one less process to fail, and the exit code you get is the agent's, not the shell's.

If the process dies before it starts (on Windows, STATUS_DLL_INIT_FAILED / STATUS_DLL_NOT_FOUND), the panel explains what happened and the runner retries once on its own. Those failures are usually transient — Windows out of session resources, or a program update touching DLLs.

Choosing the session

The panel has an Agent session selector. Each row says what it is:

New session
  ── Open now ──
  ▶ example-6b - waiting now
  ● project-c-4a [project-c] - open in terminal (busy)
  ● example-2f - open in terminal (idle)
show earlier conversations (9)

Only what's open right now shows by default. A project's history tends to hold dozens of conversations and is rarely what you want, so it sits behind the link and only expands on its own if the session you picked last time is in there.

| Marker | What it is | What the runner does | |---|---|---| | ▶ name | Agent parked inside wait_for_selection | Hands it over — no new process | | ● name | Claude Code open in a terminal right now | Sends the request into that conversation (SendMessage) | | codex name | A Codex conversation for this project | Queues the request into it (codex queue) | | opencode name | An opencode conversation, from a running instance | Posts it in (POST /session/:id/prompt_async) | | picker | An earlier send from this plugin | --resume <id> | | conversation | An older Claude Code conversation | --resume <id> | | New session (default) | — | --session-id <fresh uuid> |

Live sessions come first. The label uses the name Claude Code gives a session (project-c-4a) when it's open, and what you asked for when the picker created it — not the prompt's boilerplate header.

Related directories are included. An app in repo/web sees sessions opened in repo, and vice versa. The bracket shows where the session lives when it differs from the app root. In that case the runner runs from the session's directory and gets --add-dir <app root>, and the prompt carries the explicit root so file:line paths resolve correctly.

Why an open session isn't resumed with --resume: two processes writing the same history trip over each other. So an open session receives a message instead of being resumed — it does the work itself, in the terminal, with no parallel process.

A freshly opened session with no messages yet works fine in deliver mode: the message doesn't depend on there being a conversation on disk.

Your choice is kept in localStorage and survives a reload. If the session disappears, it falls back to "New session".

Registering the MCP server

Needed for manual mode and listening mode, and — for Claude Code only — for deliver mode too, since that is how the session pulls the screenshot. Codex and opencode deliver without it.

Claude Code

From the root of the project you're editing:

claude mcp add kolaby -- npx -y kolaby-mcp

Or a .mcp.json committed to the project:

{
  "mcpServers": {
    "kolaby": {
      "command": "npx",
      "args": ["-y", "kolaby-mcp"]
    }
  }
}

Codex

codex mcp add kolaby -- npx -y kolaby-mcp

Or in ~/.codex/config.toml:

[mcp_servers.kolaby]
command = "npx"
args = ["-y", "kolaby-mcp", "/path/to/your/project"]

With no argument the server uses the agent's working directory. Pass an explicit root if the agent runs from somewhere else, or set KOLABY_ROOT.

Verified against codex-cli 0.150.1.

opencode

{
  "mcp": {
    "kolaby": {
      "type": "local",
      "command": ["npx", "-y", "kolaby-mcp", "/path/to/your/project"],
      "enabled": true
    }
  }
}

In ~/.config/opencode/opencode.json, or opencode.json in the project — the project file is better, because then the root is the working directory and you can drop the path argument.

Only needed for listening mode. Delivery into an open opencode conversation goes over HTTP and needs no MCP at all.

Usage

npm run dev in your project.
Hold Alt and hover. Or click the button in the bottom-right corner to lock the mode on.
↑ / ↓ walk up and down the tree (button → div → whole header).
Click to select. The + button keeps the crosshair on so you can pick several.
Write the request and hit Send.
Automatic mode: the panel turns into a live log (> Read CtaButton.tsx, > Edit CtaButton.tsx) and the page reloads itself. Manual mode: in the agent, "look at the element I selected" — it calls get_selection.

Esc leaves the crosshair; a second Esc closes the panel.

What the agent receives

- Viewport: 900x700 @1x - breakpoint md (768-1023px)
- The selection was made at this width. If the change only applies at this
  size, use the breakpoint; if it applies everywhere, edit the base style.

## Element 1: <button>

- **File to edit**: `src/CtaButton.tsx:3:5`
- **Declared in component**: CtaButton
- **Render tree**: App > CtaButton
- **Warning**: instance 1 of 2 on the page rendered from the SAME line of code.
- **Warning**: this DOM node is rendered by a library component
  (MuiButtonBase > MuiButton). The file above is where YOU use the component,
  not where the tag is written. Adjust via props; don't edit node_modules.
- **Component props**:

```json
{ "label": "Buy now", "variant": "primary", "size": "lg" }
```

- **Computed style**, **rendered HTML**, **[PNG screenshot attached]**

The props are the most useful item on that list. With variant="primary" in hand, the agent knows which branch of the component to touch instead of inferring it from the CSS that came out the other end.

Plugin options

kolaby({
  enabled: true,        // turn off without removing it from the config
  outDir: '.kolaby',    // where selections are written
  hotkey: 'Alt',        // 'Alt' | 'Control' | 'Meta' | 'Shift'
  maxHtml: 2000,        // outerHTML character cap per element
  locale: 'en',         // 'en' | 'pt' — panel and agent prompt

  runner: false,        // false | true | 'claude' | 'codex' | { command, args, format }
  permissionMode: 'acceptEdits', // 'acceptEdits' | 'auto' | 'bypassPermissions' | 'manual' | 'plan'
})

Language

Two surfaces speak a language, and they're set separately.

The panel follows locale when you set it, and the browser's language when you don't. So a Brazilian developer on a project configured in English still reads the panel in Portuguese.

The agent prompt follows locale only, and defaults to English. This is the one that matters most: an agent instructed in English answers in English, in the terminal of whoever asked.

kolaby({ runner: 'claude', locale: 'pt' })

Currently en and pt. Adding a language means adding its code to LOCALES in src/shared/locale.ts and one object to each dictionary — TypeScript then points at every key you still owe. Missing keys fall back to English instead of showing a raw identifier, so a partial translation is safe to ship.

There is no i18n runtime: the overlay is a bundle injected into your app, and a translation library would weigh more than all the text it carries.

MCP tools

| Tool | Parameters | What it does | |---|---|---| | wait_for_selection | timeoutSeconds (default 120, max 600), includeImages | Blocks until you hit Send, then returns the selection. | | get_selection | includeImages (default true), consume (default false) | Reads the pending selection without waiting. | | clear_selection | — | Discards the pending selection. |

Queue, history and undo

Every send becomes its own entry under .kolaby/entries/<id>/. A send used to overwrite the previous one; now clicking twice in a row loses nothing.

The panel lists recent sends with each one's state:

queued        Second request
working       example-6b  12s ago
done          3min ago                        undo

The colored bar on the left carries the state without you having to read it: grey queued, lime working, red failed. Click a row to open what was selected, which files were touched, and what the agent did.

Undo restores the files to their contents before the run. The copy is taken the moment you hit Send, of the files the selection points at — which are exactly the ones the agent was told to edit. Redo brings the agent's work back: the post-run state is saved at the moment you undo.

The guarantee covers the target files. If the agent decides to touch a different file, that one doesn't come back — the button only appears when a copy exists, and it restores what was saved.

Security

The dev server accepts connections from any browser tab, and the send route launches an agent with write permission. Unprotected, any page you happened to have open could trigger edits in your project.

Two barriers, in both adapters:

Origin: present and from another host, refused with 403.
X-Kolaby header: there is no "simple request" carrying a custom header, so the browser is forced to send a preflight first — which we don't answer.

Both are CSRF defenses: they stop another page in your browser from driving Kolaby. They do not authenticate a direct request, and are not meant to.

Never reachable in production

Kolaby is a development tool and refuses to exist outside next dev / vite dev. Every entry point is switched off independently, so no single mistake exposes it:

| Entry point | How it turns itself off | |---|---| | Vite plugin | apply: 'serve' — never part of a production build | | Next config | bails when NODE_ENV === 'production', so no stamping | | <Kolaby /> | returns early in production, so the overlay never loads | | Route handler | every route answers 404 in production |

The route handler matters most, because unlike the others it lives in your repository and ships with your deploy. In production it returns 404 — not 403, so it does not even confirm the route exists — before reading the request, touching the disk, or starting an agent. enabled: false has the same effect in development.

Dev server routes

| Route | What it does | |---|---| | GET /__kolaby/client.js | Serves the overlay bundle. | | GET /__kolaby/sessions | Lists the project's sessions, including whoever is waiting. | | GET /__kolaby/history | Queue and history of sends. | | POST /__kolaby/revert | Restores (or re-applies) a send's files. | | GET /__kolaby/events | SSE with agent progress, replaying from the start. | | POST /__kolaby/selection | Receives the selection and dispatches. |

Limitations

Library components: a MUI <button> lives in node_modules, which isn't stamped. In that case the selection reports the file where you use the component and tells the agent the tag isn't in your code.
Screenshots don't reproduce ::before/::after pseudo-elements or the contents of <canvas> and <video>. Cross-origin images that can't be inlined become transparent, so the canvas isn't tainted.
React only for the component tree, which is read from the fiber. data-kolaby-loc works in any JSX; the component chain is empty outside React.
One agent run at a time. The rest wait in the queue, in order.
Undo covers the target files, not everything the agent might have touched.
In automatic mode answers land in the browser panel, not the terminal. To work in the open conversation, use deliver mode.
Deliver mode needs the target session to have the MCP server registered — get_selection is how it fetches the screenshot.
The target session needs a name (Claude Code assigns one automatically). Without a name the courier has no address.
wait_for_selection occupies the session while it waits: it does nothing else until you click or the timeout hits.
Delivery works with Claude Code (SendMessage), Codex (codex queue) and opencode (HTTP). Any other MCP-speaking agent uses listening mode instead.
opencode only appears while a server is running (opencode serve or opencode web). Its TUI alone opens no port, so there is nothing for kolaby to reach.
Finding that server needs lsof on the PATH on Linux and macOS; on Windows it uses netstat and tasklist, which always exist. Without it, opencode simply never shows up in the list.
Codex conversations only appear once they hold at least one message — a freshly opened one has written no rollout file yet. Claude Code sessions do not have this limitation, because claude agents --json reports them.
"Open now" for Codex is inferred from recent activity, since Codex publishes no liveness signal. A conversation idle for a while is grouped under the earlier ones; delivery works either way.
The session list reads Claude Code's internal layout (~/.claude/projects/, ~/.claude/sessions/). A change there breaks the list — "New session" keeps working regardless.
Freshly opened sessions with no messages at all don't appear in the list.

License

Kolaby is proprietary software — not open source. See LICENSE for the full terms.

In short: it is free to install and use for building your own software, including commercially. You may not redistribute it, resell it, bundle it into another product, create derivative works, or use the Kolaby name for anything other than referring to Kolaby itself.

Future releases may move some features to paid tiers. Nothing here obliges any feature to stay free forever.

For commercial licensing or redistribution rights, get in touch.

Published

Vulnerabilities

Links

Maintainers

Keywords

Readme

kolaby

How it works

Install

Next.js

Deliver mode (recommended)

Listening mode

Automatic mode

Choosing the session

Registering the MCP server

Claude Code

Codex

opencode

Usage

What the agent receives

Plugin options

Language

MCP tools

Queue, history and undo

Security

Never reachable in production

Dev server routes

Limitations

License