GuideGen

Turn a guide.json file into an in-app product tour and walkthrough videos.

• No runtime LLM. The JSON is the source of truth.
• One install: React tour UI + Studio + guidegen video / from-doc CLI.
• Works with React and Next.js (App Router).
• Tour playback uses PageAgent’s cursor on the live page (no LLM at runtime).

npm install guidegen

npm — published package is guidegen. This repo’s workspace version is 0.1.2 (Studio, cursor playback, and JSON pruning). Internals @guidegen/core / @guidegen/react / @guidegen/video are bundled into guidegen; do not import them from an app.

What you need

1. A guide.json (write one, or generate it from an SRS with Studio / guidegen from-doc).
2. Stable selectors on your UI (data-testid is best).
3. For videos: a running app, ffmpeg, and Playwright Chromium.

1. Sample guide.json (copy this)

Save as guides/onboarding.json.

{
  "version": "1.0.0",
  "guideId": "admin-onboarding",
  "title": "Admin Onboarding",
  "flows": [
    {
      "id": "create-project",
      "title": "Create a new project",
      "steps": [
        {
          "order": 1,
          "route": "/projects",
          "action": "highlight",
          "narration": "Welcome to the Projects page. This guide will show you how to create a new project.",
          "durationMs": 2500,
          "pauseAfterMs": 400
        },
        {
          "order": 2,
          "route": "/projects",
          "action": "click",
          "target": {
            "cssSelector": "[data-testid='new-project-btn']",
            "elementText": "New Project",
            "role": "button"
          },
          "narration": "Click New Project to open the creation form.",
          "durationMs": 2200,
          "pauseAfterMs": 400
        },
        {
          "order": 3,
          "route": "/projects",
          "action": "input",
          "target": {
            "cssSelector": "[data-testid='project-name-input']",
            "elementText": "Enter a project name",
            "value": "Launch Campaign"
          },
          "narration": "Enter a project name for your new project.",
          "durationMs": 2600,
          "pauseAfterMs": 400
        },
        {
          "order": 4,
          "route": "/projects",
          "action": "click",
          "target": {
            "cssSelector": "[data-testid='create-project-btn']",
            "elementText": "Create",
            "role": "button"
          },
          "narration": "Click Create to add the project to your list.",
          "durationMs": 2400,
          "pauseAfterMs": 0
        }
      ]
    }
  ]
}

Match those selectors in your UI:

<button type="button" data-testid="new-project-btn">New Project</button>

<input
  data-testid="project-name-input"
  placeholder="Enter a project name"
/>

<button type="button" data-testid="create-project-btn">Create</button>

Enable JSON imports in tsconfig.json:

{
  "compilerOptions": {
    "resolveJsonModule": true
  }
}

2. Generate guide.json from a document

Compile-time only. Paste an SRS, PRD, or any written how-to. An LLM plans the flows, then PageAgent runs each intent on your live app and the recorded actions become guide.json. Runtime tours still use the JSON with no LLM.

In-app studio

Mount GuideStudio in the root layout so it survives navigation (PageAgent has to drive the real screens).

'use client'

import guide from '@/guides/onboarding.json'
import { useRouter } from 'next/navigation'
import { Guide, GuideStudio } from 'guidegen/next'

export function Providers({ children }: { children: React.ReactNode }) {
  const router = useRouter()

  return (
    <Guide guide={guide} widget="create-project">
      {children}
      <GuideStudio
        onNavigate={(route) => router.push(route)}
        videoEndpoint="/api/guidegen/video"
        llmProxyEndpoint="/api/guidegen/llm"
      />
    </Guide>
  )
}

The Studio button (bottom left) lets you:

1. Paste a document
2. Enter an OpenAI-compatible model, base URL, and API key (session-only)
3. Generate JSON — PageAgent clicks/types on the live UI and maps actions to the guide schema (stable data-testid / CSS selectors when it can)
4. Download the JSON
5. Optionally generate videos if you pass videoEndpoint or onGenerateVideo

Recording ignores Guide/Studio overlay chrome. After PageAgent finishes, GuideGen prunes exploratory scrolls, failed retries, and steps that only have elementIndex (those cannot be replayed as video). Keep the SRS faithful to real UI — do not describe rename/delete if those controls do not exist.

Pass llmProxyEndpoint (demo: /api/guidegen/llm) so planner and PageAgent calls go through your server. Browser-direct calls to many LLM APIs are blocked by CORS.

LLM proxy (demo):

// app/api/guidegen/llm/route.ts
import { NextResponse } from 'next/server'

export async function POST(request: Request) {
  const target = request.headers.get('x-guidegen-target')
  const apiKey = request.headers.get('x-guidegen-api-key')
  if (!target) {
    return NextResponse.json({ error: 'Missing x-guidegen-target header.' }, { status: 400 })
  }

  const response = await fetch(target, {
    method: 'POST',
    headers: {
      'Content-Type': 'application/json',
      ...(apiKey ? { Authorization: `Bearer ${apiKey}` } : {}),
    },
    body: await request.text(),
  })

  return new NextResponse(await response.text(), {
    status: response.status,
    headers: { 'Content-Type': response.headers.get('Content-Type') || 'application/json' },
  })
}

A Next.js demo route that wraps the existing video CLI:

// app/api/guidegen/video/route.ts
import { mkdtemp, writeFile } from 'node:fs/promises'
import { tmpdir } from 'node:os'
import { join } from 'node:path'
import { compileGuideVideos } from 'guidegen/video'

export async function POST(request: Request) {
  const guide = await request.json()
  const dir = await mkdtemp(join(tmpdir(), 'guidegen-'))
  const guidePath = join(dir, 'guide.json')
  await writeFile(guidePath, JSON.stringify(guide, null, 2))

  await compileGuideVideos({
    guidePath,
    outputDir: join(process.cwd(), 'public/videos'),
    url: 'http://localhost:3000',
  })

  return Response.json({
    videos: [{ flowId: 'create-project', url: '/videos/create-project.mp4' }],
  })
}

CLI

App must be running with <GuideStudio /> mounted (it registers window.__guidegenRecordFlow).

npx guidegen from-doc ./guides/sample-srs.md \
  --url http://localhost:3000 \
  --out ./guides/onboarding.json

Optional --video renders MP4s after the JSON is written.

| Flag | Default | |------|---------| | --url | GUIDEGEN_URL or http://localhost:3000 | | --out | ./guides/onboarding.json | | --model | GUIDEGEN_LLM_MODEL or gpt-4o-mini | | --base-url | GUIDEGEN_LLM_BASE_URL or https://api.openai.com/v1 | | --api-key | GUIDEGEN_LLM_API_KEY or OPENAI_API_KEY | | --video | off |

3. In-app tour — Next.js (copy this)

app/providers.tsx

'use client'

import guide from '@/guides/onboarding.json'
import { Guide } from 'guidegen/next'

export function Providers({ children }: { children: React.ReactNode }) {
  return (
    <Guide guide={guide} widget="create-project">
      {children}
    </Guide>
  )
}

widget options:

• "create-project" — floating button for that flow
• true — floating button for the first flow in the JSON
• omit — no floating button (use <GuideWidget /> yourself)

app/layout.tsx

import { Providers } from './providers'

export default function RootLayout({ children }: { children: React.ReactNode }) {
  return (
    <html lang="en">
      <body>
        <Providers>{children}</Providers>
      </body>
    </html>
  )
}

Styles are injected automatically. A Guide button appears at the bottom-right. During playback the PageAgent cursor moves to each control and clicks/types; numbered step badges stay off. Stop / × remain clickable (the cursor overlay does not capture pointer events). When the tour ends, the cursor is removed.

If you use next/image or a custom tsconfig paths alias, keep @/ pointing at the app root so @/guides/onboarding.json resolves.

4. In-app tour — React (copy this)

import guide from './guides/onboarding.json'
import { Guide } from 'guidegen'

export function App({ children }: { children: React.ReactNode }) {
  return (
    <Guide guide={guide} widget="create-project">
      {children}
    </Guide>
  )
}

With React Router, pass navigation:

import { useLocation, useNavigate } from 'react-router-dom'
import { Guide } from 'guidegen'
import guide from './guides/onboarding.json'

export function AppGuide({ children }: { children: React.ReactNode }) {
  const navigate = useNavigate()
  const location = useLocation()

  return (
    <Guide
      guide={guide}
      widget="create-project"
      getCurrentPath={() => location.pathname}
      onNavigate={(route) => navigate(route)}
    >
      {children}
    </Guide>
  )
}

5. Extra components (copy this)

Use these inside <Guide> / <Guide> from guidegen/next.

import { GuideWidget, GuidePlayer, GuideVideo, useGuide } from 'guidegen'

// Floating button for one flow
<GuideWidget flowId="create-project" label="Guide" />

// Embedded player with Play / Pause / Stop
<GuidePlayer flowId="create-project" />

// HTML5 video player (after you generate an MP4)
<GuideVideo flowId="create-project" />

Start a flow from your own button:

'use client'

import { useGuide } from 'guidegen'

export function StartTourButton() {
  const { play, pause, resume, stop, state, currentStep } = useGuide('create-project')

  return (
    <div>
      <button type="button" onClick={() => void play()}>
        Start tour
      </button>
      <p>Status: {state.status}</p>
      <p>{currentStep?.narration}</p>
    </div>
  )
}

From Next.js, the same components are re-exported:

import { GuideWidget, GuidePlayer, GuideVideo, useGuide } from 'guidegen/next'

6. Videos (copy this)

One-time setup

# ffmpeg (macOS)
brew install ffmpeg

# Playwright browser used by the recorder
npx playwright install chromium

Generate MP4s

Start your app, then:

npx guidegen video ./guides/onboarding.json --url http://localhost:3000

That:

1. Opens --url in headless Chromium
2. Waits for the app to hydrate (so React click handlers are attached)
3. Replays every replayable flow step in guide.json (hunt/retry/elementIndex-only steps are skipped)
4. Screenshots before each click/input/select (target is highlighted)
5. Overlays the step narration
6. Writes ./public/videos/<flow-id>.mp4

CLI flags

| Flag | Default | Meaning | |------|---------|---------| | --url | GUIDEGEN_URL or http://localhost:3000 | Running app origin | | --out | ./public/videos | MP4 output folder | | --assets | directory of the JSON file | Root for ttsPath / optional screenshots | | --in-place | off | Write videoPath back into guide.json | | --width / --height | 1280 / 720 | Capture viewport | | --from-assets | off | Stitch existing screenshotPath files instead of capturing |

npx guidegen video ./guides/onboarding.json \
  --url http://localhost:3000 \
  --out ./public/videos \
  --in-place

package.json script

{
  "scripts": {
    "guide:video": "guidegen video ./guides/onboarding.json --url http://localhost:3000 --out ./public/videos"
  }
}

Play the video in the app

After generate, public/videos/create-project.mp4 is served at /videos/create-project.mp4.

<GuideVideo flowId="create-project" />

Or set "videoPath": "videos/create-project.mp4" on the flow (or pass src).

Optional TTS: put an mp3 next to the guide and set "ttsPath": "audio/create-project/step-1.mp3" on the step. If it is missing, the video is silent plus the caption overlay.

7. guide.json reference

Guide

| Field | Required | Description | |-------|----------|-------------| | version | yes | Schema version, e.g. "1.0.0" | | guideId | yes | Stable id for this guide | | title | yes | Display title | | flows | yes | One or more flows |

Flow

| Field | Required | Description | |-------|----------|-------------| | id | yes | Used as flowId and as the MP4 filename | | title | yes | Shown in the player | | steps | yes | Ordered steps (order is used if they are unsorted) | | videoPath | no | Path/URL for <GuideVideo /> |

Step

| Field | Required | Description | |-------|----------|-------------| | order | yes | 1-based order | | action | yes | See actions below | | route | no | Path to be on before the step, e.g. "/projects" | | target | for click/input/select | How to find the control | | narration | no | Tooltip + video caption | | durationMs | no | Video segment length (default 2000) | | pauseAfterMs | no | Pause after the action (default 500) | | value | on target | Text to type / option to select | | ttsPath | no | Audio file for this step | | screenshotPath | no | Only used with --from-assets | | scrollDown / scrollPages | no | For scroll |

Actions

| action | What it does | |----------|----------------| | highlight | Show narration; no click | | click | Click target | | input | Fill target with target.value | | select | Choose option target.value or elementText | | scroll | Scroll the page (scrollDown, scrollPages) | | navigate | Go to route | | wait | Wait durationMs |

How target is resolved

1. cssSelector
2. role + elementText
3. elementText
4. elementIndex (in-app tour only — not enough for video capture)

Always prefer data-testid / cssSelector.

8. Theming (copy this)

:root {
  --guidegen-accent: #2563eb;
  --guidegen-overlay: rgba(15, 23, 42, 0.55);
  --guidegen-surface: #ffffff;
  --guidegen-text: #0f172a;
}

9. Imports cheat sheet

// React
import { Guide, GuideWidget, GuidePlayer, GuideVideo, GuideStudio, useGuide } from 'guidegen'

// Next.js App Router
import { Guide, GuideWidget, GuidePlayer, GuideVideo, GuideStudio, useGuide } from 'guidegen/next'

// Node (build scripts)
import { compileGuideVideos } from 'guidegen/video'

Programmatic video build:

import { compileGuideVideos } from 'guidegen/video'

await compileGuideVideos({
  guidePath: './guides/onboarding.json',
  outputDir: './public/videos',
  url: 'http://localhost:3000',
})

10. Troubleshooting

| Problem | Fix | |---------|-----| | Flow "…" was not found | widget / flowId must match flows[].id | | Tour clicks the wrong control | Add data-testid and set target.cssSelector | | Video capture cannot find a step | Video needs cssSelector or role + elementText (not elementIndex alone) | | ffmpeg is required | Install ffmpeg and ensure it is on PATH | | Could not reach --url | Start the app first, then run guidegen video | | JSON import type error | Set "resolveJsonModule": true in tsconfig.json | | Guide button missing | Pass widget or render <GuideWidget flowId="…" /> inside <Guide> | | No GuideStudio recorder | Mount <GuideStudio /> in the app layout, keep the app running, then retry from-doc | | Planner / PageAgent LLM error | Check --base-url, --model, and --api-key (OpenAI-compatible /chat/completions). In the browser, set llmProxyEndpoint. | | Studio / PageAgent leftover mask or numbered badges | Click Guide or Stop (cleanup runs on mount too). Hard-refresh after a package rebuild. Recording must not click Guide/Studio chrome. | | Generated JSON is a long retry/scroll loop | SRS described UI that is not on the page (e.g. rename). Re-generate; GuideGen now prunes hunts and index-only clicks. | | Video times out on #project-name (or a form field) | The opening click must run after React hydrates, and the JSON must include that click before the input. Prefer data-testid. | | Generated JSON has only elementIndex | Add data-testid on controls so the recorder can emit cssSelector | | Studio video job fails | App must be reachable at --url; ffmpeg and Playwright Chromium must be installed |

11. This repository

Monorepo. Public package is guidegen; the demo at examples/nextjs-demo imports the built workspace package.

npm install
npm run build
npm test
npm run dev:demo          # http://localhost:3000

After changing packages/*, rebuild then hard-refresh the demo. Contributor context for agents lives in AGENTS.md.

License

MIT