@lnittman/pi-interview

High-trust interview extension for pi.

pi-interview turns the next-step decision after an agent turn into a grounded interview:

• multi-select questions with notes
• cold-start support (Ctrl+I / /interview ask before any assistant turn)
• rich grounding from files, diagnostics, trajectory, project state, and ~/.agents/**
• provider-first settings with separate interviewModel and handoffModel selection from the same interview fleet
• curated context-richness profiles (focused, balanced, rich) instead of raw context-engineering knobs
• workflow-aware prompting that treats debrief as a phase-transition tool instead of a generic default
• viewport-aware interview rendering for long option descriptions and grounding, with grounding hidden until you reveal it in the interview UI
• composed follow-up prompts that send a synthesized handoff brief plus an exact interview appendix into the next agent turn

Commands

• /interview ask — trigger manually
• /interview settings — choose interviewModel, handoffModel, allowed model fleet, context profile, capability-aware thinking level, and trigger mode
• /interview status — show config, usage status, and latest interview/handoff runtime details
• /interview audit — inspect the current context packet
• /interview trace — toggle trace-rich context auditing
• Ctrl+I — manual interview shortcut

Development

cd ~/Developer/packages/pi/pi-interview
pnpm install
pnpm build
pnpm test
pnpm typecheck

Subjective quality evals

This repo includes a Braintrust-based subjective eval stack in addition to deterministic regression tests.

Commands:

pnpm telemetry:evals:list
pnpm telemetry:evals:quality
pnpm telemetry:evals:pairwise
pnpm telemetry:evals:workflow
pnpm telemetry:evals:portfolio
pnpm telemetry:evals:handoff
pnpm telemetry:evals:handoff:variants
pnpm telemetry:dataset:seed

Use Doppler for secrets:

doppler run -- pnpm telemetry:evals:quality
doppler run -- pnpm telemetry:evals:pairwise
doppler run -- pnpm telemetry:evals:workflow
doppler run -- pnpm telemetry:evals:portfolio
doppler run -- pnpm telemetry:evals:handoff
doppler run -- pnpm telemetry:evals:handoff:variants
doppler run -- pnpm telemetry:dataset:seed

Example portfolio sweep, including gpt-5.4 as a real fallback generator candidate:

doppler run -- env PI_INTERVIEW_EVAL_MODELS=zai/glm-5.1,minimax/MiniMax-M2.7-highspeed,openai/gpt-5.4 pnpm telemetry:evals:portfolio

Example replay-backed handoff variants sweep with explicit interviewModel × handoffModel × receiverModel matrices:

doppler run -- env PI_INTERVIEW_HANDOFF_SOURCE=replay PI_INTERVIEW_EVAL_INTERVIEW_MODELS=zai/glm-5.1,minimax/MiniMax-M2.7-highspeed PI_INTERVIEW_EVAL_HANDOFF_MODELS=zai/glm-5.1,minimax/MiniMax-M2.7-highspeed PI_INTERVIEW_EVAL_RECEIVER_MODELS=zai/glm-5.1 PI_INTERVIEW_HANDOFF_VARIANTS=legacy,static,appendix,agentic pnpm telemetry:evals:handoff:variants

Full eval playbook:

• docs/evals.md

License

MIT