panelist

What panelist is: Synthetic user panels for any artifact, run across multiple model providers to correct for self-preference bias. Most tools of this kind return a warmth score, which invites "the AI loved it" misuse. Panelist models behavior instead: the exact point a persona quits, dismisses, or refuses to click like.

Differentiator: panelist's primary output is a deal-killer / cut-list: the exact point a persona would quit, dismiss, refuse to click, or refuse to forward — abandonment is behavioral and far more reliably simulable than affect. Panels are cross-model by default (fighting same-model self-preference bias), personas differ by what makes them quit rather than demographics, and every output auto-stamps the honesty caveat that this is a pre-filter, not user research.

Why: a 9/10 score from a model that wants to please you is worthless as a filter. A specific point where three independent, cross-model personas all disengage is an actionable signal — that's the difference panelist is built around.

⚠️ What panelist is and is not for. A synthetic persona is a model of a customer, and your model of the customer is wrong — that is why you talk to real people. panelist is legitimate as a drafting aid, a cheap pre-filter to kill obviously-weak drafts before spending a human's attention, and a way to catch obvious misses. It is not evidence about real readers, not a substitute for talking to them, and not validation. "Our personas responded well to this" is a sentence this tool is designed to make hard to write.

Persona isolation, by construction (panelist#72/#75). "A persona sees the artifact and nothing else" is a structural claim, not an aspiration: every execution plane — spawn/runPersona, score/scoreCandidate, and the multi-turn runJunctionLoop — grants a persona no tools by default: no MCP server, no web search, no filesystem search. Wildcard/"grant everything" opt-ins are rejected outright, so a tool-discovery capability can never be smuggled in by granting some other tool. Every response reports the effective granted set (isolation.tools) and any attempted-but-denied call (isolation.denied), so a contaminated run is visible instead of indistinguishable from a clean one. See "Tool isolation".

Why it's different

Most tools that spawn personas emit a warmth/viability score — the exact output that invites "look, the AI loves it" misuse. panelist's primary output is a deal-killer / cut-list: where would this persona stop, dismiss, or refuse to click, forward, or buy? Abandonment is behavioural and far more robustly simulable than affect.

• Deal-killers, not scores. The default verdict is a cut-list, not a rating.
• Cross-model panels. ≥2 providers by default (designed to wrap a provider layer you supply, e.g. PromptFoo/LiteLLM — not hand-rolled) to counter same-model self-preference / sycophancy bias.
• Diversity by kill-condition. Personas differ by what makes them quit, not by demographics. Identity is behavioural (rewards / punishes / quitsWhen), never age/employer/tenure.
• Honesty by construction. Every panel output auto-stamps the caveat above.
• Isolated by construction. No tool is reachable unless explicitly granted — see above.
• Calibration hooks. Join synthetic verdicts to real downstream signal and rank personas by how well they predict, not how well they read.

See docs/synthetic-persona-best-practices.md for the governing rules behind these design choices (specificity over decoration, demographics vs. behaviour, anti-sycophancy, panel diversity, calibration, and the honesty line).

Install

npm i panelist

panelist is published to the public npm registry as the unscoped package panelist (via OIDC Trusted Publishing — no token needed): owned by Kromatic Innovation; published to npm from the trikro account. No .npmrc changes required.

Bring your own model client — required. panelist bundles no live model. You inject a provider adapter as spawn's third argument (deps.client); the default client throws rather than run without one, so a panel can never silently run un-modelled. In production this wraps a provider layer you supply (e.g. PromptFoo/LiteLLM). See the prerequisite below for the adapter shape.

Use cases

panelist models a synthetic user reacting to any artifact — not just prose. Reading is one behavior among several; quitting, dismissing, refusing to forward, refusing to click, and refusing to buy are all in scope:

• Copy and long-form: "Given this paragraph, will you read the next one?" — resume review, book-chapter reader panels, blog-post pre-filters.
• Interface and flow: "Where in this onboarding would you give up?"
• Commercial: "Would you pay for this? At what point do you stop believing the pricing page?"
• Developer-facing: "Would you install this? Where does the README lose you?" — OSS-README review.
• Decision review: "Given this plan, what would make you walk away?"

The shipped packs/business pack — a B2C consumer and a B2B buyer — exists for the commercial use cases above; it is as central to panelist as packs/review's OSS-code-review archetypes, not an afterthought bolted onto a prose-review tool.

Multi-turn junction walks

For the "will you read the next one?" use cases — walking a persona through a book (branching hub-and-spoke) or a blog post (a linear chain) one junction at a time, behind a structural information barrier — see the junction contract. It covers graph authoring, the engine guarantees (barrier, always-available bail, patience budget), and the consumer verdict hook, with two runnable worked examples (branching, linear-chain).

Personas: identity is data, task is ephemeral

A persona is a durable identity record (what it rewards, punishes, and quits over). The task (vote / comment / converse) is supplied by the caller at invocation time — so one definition answers any instruction, with no new consumer script per use case. See docs/invocation-contract.md for the formal task/response envelope. For the agentic plane, src/lib/runner.mjs (renderRunnerPrompt / runPersona, backing .claude/agents/persona.md) is a single generic runner for ANY registered persona by id — no per-persona agent files.

Prerequisite: panelist bundles no live model. You inject a client as spawn's third argument (deps.client) — its shape is { model, complete: async ({ prompt }) => ({ ok, text, model }) }, the same adapter score.mjs uses. In production this wraps a provider layer you supply (e.g. PromptFoo/LiteLLM); the default client throws so you can never accidentally run without one. See docs/invocation-contract.md.

import { spawn, registerPersonas } from "panelist";
import reviewPack from "panelist/packs/review";

registerPersonas(reviewPack);

// A trivial stub client — swap the body for a real provider call.
const client = {
  model: "your-model",
  async complete({ prompt }) {
    // call your provider with `prompt`, return its raw text
    return {
      ok: true,
      text: '{ "verdict": "keep", "note": "stub" }',
      model: "your-model",
    };
  },
};

const readmeText = "# my-project\n\nInstall: npm i my-project\n"; // the draft under review
const verdict = await spawn(
  "drive-by-installer",
  {
    mode: "vote",
    artifact: readmeText,
    instruction: "Would you stop reading before you found the install command?",
  },
  { client },
);

Persona packs (toggleable)

Packs are exported but nothing auto-registers — a consumer opts in explicitly. Shipped example packs:

• packs/review — OSS-code-review archetypes (drive-by-installer, production-evaluator, maintainers-maintainer, drive-by-contributor).
• packs/business — a middle-class B2C consumer and a B2B buyer, for the "would you click / would you buy / would you read on" use cases.

These are examples. Real, private persona rosters live in their owner's repo and register at runtime; they are never shipped here.

Status

Early. Developed internally, then open sourced. Apache-2.0.