corva-fe-ai-tools

Two things in one repo:

1. @corva/fe-ai-tools — the standalone, versioned Ask Corva app-tools kit (result builders, createToolset, the register/eval hooks, the param readers) imported by DC and platform FE apps. Source in src/; yarn build (tsup) / yarn test (Vitest) / yarn lint. React is a peer dependency. Plan: features/app-tools-kit.
2. The coverage playbook — instruction library for covering Corva FE apps with Ask Corva AI tools, eval suites, dialogue scenarios, scripts, and coverage tracking (everything below).

Use this repo as the working playbook. Links to corva-web-frontend are freshness references only — an agent should start here, pick the right workflow, and execute without first reading another repo.

Start here — what do you need to do?

| I need to… | Go to | | --- | --- | | Onboard an app — add Ask Corva tools and cover it | The onboarding pipeline ↓ (section below) | | Test an app / see how the agent uses its tools | workflows/testing/ — Recipe A (explore) / E (accept) | | Understand how Ask Corva works (concepts first) | New to Ask Corva? ↓ (next section) | | Work on platform tools / multi-app routing | workflows/platform.md | | See what's already covered / the backlog | docs/coverage/ | | Plan or track multi-wave work (tiers, statuses, waves) | plans/README.md |

The rule that routes most decisions: only anchor apps (high usage × risk — see docs/coverage/coverage-policy.md) get tools + a full graded eval suite. The long tail stops at discovered queries + a few live scenarios. So "onboard" means different depth depending on whether the app is an anchor — the pipeline below makes that branch explicit.

New to Ask Corva? Read this first

If you have never seen Ask Corva, spend five minutes on the mental model before touching a workflow. The workflow READMEs assume you know these two things — they own the process, not the concepts.

1. docs/reference/architecture.md — how a user prompt becomes a UI change, what a "deferred tool" is, how a tool reaches the agent (prompt → agent → app-call-tool → your handler → state). The why behind every convention; includes the agent roster + glossary.
2. docs/reference/tool-authoring.md — what a tool is, the two kinds, and exactly how one looks (folder shape, envelope, the [WHAT]/[WHEN]/[RETURNS] description standard).
3. docs/reference/eval-harness.md — how the harness drives the real UI and grades state (read when you reach testing/evals).

The onboarding pipeline

The per-app journey, in order. Each step is a markdown entry point — pass one absolute app path, no labels (full rules in the "How to invoke" section below).

discover-queries → tools → evals (author + drive to green) → testing
   queries.json      AI tools   eval suite + scenarios.md     live A/E
        │              │              │                          │
   every app      anchor only     anchor only              every app

queries.json is the single source of truth the rest derive from. Anchor app → run all four steps. Long-tail app → step 0 + a few live scenarios (steps 1–2 are anchor-only). There is intentionally no separate "full-cover" workflow — this pipeline is the navigation; start at the step your task needs.

0. Discover real user queries — every app

@workflows/discover-queries/README.md /absolute/path/to/app

Investigates the app domain + source, then writes {app}/ai-tests/queries.json (≥30 real user questions classified support / clarify / invalid-value / undo / coverage-gap / domain-limitation). Re-run whenever user needs shift — it preserves query IDs and diffs against the previous file.

1. Add tools — anchor app, needs queries.json

@workflows/tools/README.md /absolute/path/to/dev-center-app-repo

Builds the Ask Corva tool surface inside a Dev Center app repo (the ai-tools/ folder, createToolset, the kit bridge, a unit test per handler).

2. Author + green the eval suite — anchor app

@workflows/evals/README.md /absolute/path/to/app

Authors the anchor eval suite (every tool + app-specific subsets + ground-truth) and {app}/ai-tests/scenarios.md from queries.json, then drives it to green.

3. Live & interactive testing — every app

@workflows/testing/README.md /absolute/path/to/app

Talk to the real agent on a local build and watch it use the app's tools. Recipe A (interactive, one prompt at a time: which tool + args the agent picks) and Recipe E (graded multi-turn scenario acceptance) for every app; the eval suite (full graded batch) for anchors only — plus Recipe C/D utilities. The whole journey, arrow-by-arrow: workflows/testing/testing-flow.md.

Platform tools + their eval suite are already built — maintained via workflows/platform.md, not a per-app workflow.

How to invoke

Every workflow is a markdown entry point. Pass one positional path when the workflow targets an app — no labels. Use the exact entry shown in each workflow README.

@workflows/<workflow>/README.md /absolute/path/to/app

Platform-level work takes no app path unless its README says otherwise.

App launchers (onboard / test)

Two launchers turn a few answers into a ready start: they locate the app repo (cloning it with --clone if absent), seed its live-test .env, write a machine-readable JSON result, and print where to start + the operational gotchas, then point you back into this README for the navigation. Run them from the repo root (or by absolute path — they resolve their own location):

# Cover an app (add AI tools) — the slug is all it needs (interactive prompt):
scripts/launch/onboard.sh

# …or headless (how the agent runs it, after taking the repo from you):
scripts/launch/onboard.sh --repo directional-surveys-and-projections [--clone] [--dry-run]

# Test an app (drive the agent live) — repo + recipe:
scripts/launch/test-app.sh --repo directional-surveys-and-projections --recipe E [--clone]

# Print the questions a script asks (the agent reads this to know what to ask you):
scripts/launch/onboard.sh --print-questions

The app name is derived from the slug; onboarding runs the whole pipeline in order (the anchor decision is made inside the workflow, not asked here).

--repo takes a slug (corva-dc-fe-<slug> is inferred for DC apps), owner/name, a GitHub URL, or a local path. --dry-run previews clone/.env actions without performing them; an existing .env is never overwritten.

Library map

Where everything lives (the journey above is the order; this is the index).

| Path | Purpose | | --- | --- | | workflows/discover-queries/ | Discover real user queries → {app}/ai-tests/queries.json. | | workflows/tools/ | Add Ask Corva tools to a Dev Center (anchor) app repo. | | workflows/evals/ | Author an anchor eval suite + scenarios.md from queries.json, and drive to green. | | workflows/testing/ | Live & interactive agent testing for any app or multi-app dashboard (Recipes A–E + the eval suite). | | workflows/platform.md | The platform tools + eval suite (already built) — maintenance note. | | docs/reference/ | Source-of-truth concept docs: architecture, tool authoring, eval harness, eval architecture. | | docs/coverage/ | Tool coverage inventory + DC-app backlog owned by this repo. | | plans/ | Planning + execution framework (task tiers, waves, changelog) for features and bugs. | | scripts/ | Deterministic validation + pre-flight guardrails, plus the launch/ folder (onboard.sh / test-app.sh + shared lib/common.sh) that locates/clones an app repo, seeds its .env, and routes into a workflow. |

Workflow folders use short, repeated filenames: README.md (entry point + execution contract), checklist.md (final quality gates, where kept separate), schema.json / example.json (machine-readable contract + example), examples/ (worked-example files for that workflow only).

Contracts and ownership

This repo is the source of truth and owns: the workflow playbooks; scripts and validation gates; the coverage inventory + DC-app backlog; and the concept reference in docs/reference/ — architecture.md (data flow, agent roster, glossary), tool-authoring.md, eval-harness.md, and eval-architecture.md (the anchor/concentrate model).

CWF (corva-web-frontend/docs/ai-agent/) keeps only the code-coupled docs that must update in the same session as the source: code-map.md (file map), ownership.md (decision log), capabilities.md (matrix tied to platform eval case IDs). Re-sync the reference docs here against those three when the FE changes.

Live testing produces dated run reports with the code under test — platform sessions in corva-web-frontend/ai-tests/runs/, per-app sessions in {app}/ai-tests/runs/ — evidence logs, not reference docs, and not in this playbook.

Validation

Before considering changes done, run the gates (a non-zero exit is a hard stop):

scripts/audit-docs.sh            # doc hygiene: stale wording, JSON, links, shell, whitespace
plans/scripts/check-playbook.sh  # plans gate: waves, changelog freshness, template index
git diff --check                 # no whitespace errors / conflict markers

Changes under plans/features|bugs|testing/ also need a same-session plans/changelog.md entry naming each touched area — the playbook gate enforces it.