@haystackeditor/verify

Configure Haystack to verify PRs on your repository with dev server setup, verification commands, and fixtures.

Quick Start

The fastest way to get started:

npx @haystackeditor/cli skills install

Then invoke the setup in your coding CLI:

| CLI | How to invoke | |-----|---------------| | Claude Code | /setup-haystack | | Codex CLI | /setup-haystack or ask "set up haystack verification" | | Cursor | /setup-haystack (in Composer) |

Alternative Installation

If you prefer to register the MCP server manually:

claude mcp add haystack-verify -- npx @haystackeditor/verify

What /setup-haystack Does

This diagnoses your project and guides you through complete setup:

• Creates .haystack.json config
• Configures your bundler (Vite/Next.js/Webpack) for sandbox compatibility
• Fixes localhost/hostname issues in your code
• Sets up API keys and secrets
• Adds verification flows

Available Skills

| Command | Purpose | |---------|---------| | /setup-haystack | Start here - diagnoses project, routes to sub-skills | | /prepare-haystack | Add aria-labels and data-testid for browser automation | | /setup-haystack-secrets | Configure API keys, LLM credentials, secrets |

2. Or Create Config Manually

Create .haystack.json in your project root:

version: "1"
name: my-app

dev_server:
  command: pnpm dev
  port: 3000
  ready_pattern: "Local:"
  env:
    SKIP_AUTH: "true"  # Bypass auth in sandbox

verification:
  commands:
    - name: build
      run: pnpm build
    - name: lint
      run: pnpm lint
    - name: typecheck
      run: pnpm tsc --noEmit

  fixtures:
    "/api/users/*":
      source: "file://fixtures/users.json"
    "/api/config":
      source: "https://staging.myapp.com/api/config"

How It Works

When Haystack runs in a sandbox:

1. Clone & Install - Your repo is cloned and dependencies installed
2. Run Verification Commands - Runs each command in verification.commands during sandbox creation
3. Cache Results - Command outputs are cached for instant restoration
4. Start Dev Server - Runs dev_server.command with dev_server.env variables
5. Wait for Ready - Waits for dev_server.ready_pattern in stdout
6. Load Fixtures - Intercepts API calls matching fixture patterns

The sandbox caches verification results, so subsequent sandboxes restore instantly.

Configuration Reference

Simple Mode (Single Dev Server)

For projects with a single service:

version: "1"
name: my-app

dev_server:
  command: pnpm dev           # Command to start server
  port: 3000                  # Port number
  ready_pattern: "Local:"     # Stdout pattern indicating ready
  env:                        # Environment variables
    NODE_ENV: development
    SKIP_AUTH: "true"         # Bypass auth in sandbox

verification:
  commands:
    - name: build
      run: pnpm build
    - name: lint
      run: pnpm lint

Monorepo Mode (Multiple Services)

For monorepos with multiple services (frontend, backend workers, etc.):

version: "1"
name: my-monorepo

services:
  # Frontend web app
  frontend:
    root: ./                    # Directory relative to repo root
    command: pnpm dev
    port: 3000
    ready_pattern: "Local:"
    env:
      SKIP_AUTH: "true"

  # Backend API worker
  api:
    root: infra/api-worker
    command: pnpm wrangler dev --local
    port: 8787
    ready_pattern: "Ready on"
    env:
      MOCK_EXTERNAL_APIS: "true"

  # Batch job (not a long-running server)
  analysis:
    root: analysis
    type: batch               # Run once, don't wait for port
    command: pnpm start
    env:
      LOCAL_OUTPUT: "true"

verification:
  commands:
    - name: build
      run: pnpm build
    - name: lint
      run: pnpm lint

Authentication Bypass

Auth bypass is configured via environment variables in dev_server.env or services[].env:

dev_server:
  command: pnpm dev
  port: 3000
  env:
    # Common patterns - use whichever your app supports
    SKIP_AUTH: "true"
    BYPASS_AUTH: "true"
    AUTH_DISABLED: "1"
    NEXT_PUBLIC_MOCK_AUTH: "true"

Your app code should check these env vars and skip authentication when set:

// Example auth bypass in your app
const skipAuth = process.env.SKIP_AUTH === 'true';
if (skipAuth) {
  // Return mock user, skip token validation, etc.
}

Verification Commands

Verification commands run during sandbox creation. Results are cached.

verification:
  commands:
    - name: build
      run: pnpm build
    - name: lint
      run: pnpm lint
    - name: typecheck
      run: pnpm tsc --noEmit
    - name: test
      run: pnpm test --passWithNoTests

Common command templates by project type:

| Project Type | Typical Commands | |-------------|-----------------| | Frontend (React/Vue) | pnpm build, pnpm lint, pnpm tsc --noEmit | | Node.js API | pnpm build, pnpm lint, pnpm test | | Cloudflare Worker | pnpm wrangler deploy --dry-run, pnpm tsc --noEmit | | Python | pytest, mypy ., ruff check . | | Monorepo | pnpm -r build, pnpm -r lint |

Fixtures (Optional)

Fixtures define mock data for API endpoints. They're keyed by URL pattern:

verification:
  fixtures:
    # Local file (relative to repo root)
    "/api/config":
      source: "file://fixtures/config.json"

    # Remote URL with API token
    "/api/users/*":
      source: "https://staging.example.com/api/users"
      headers:
        Authorization: "Bearer $STAGING_TOKEN"

    # Remote URL with session cookies (login-required endpoints)
    "/api/dashboard":
      source: "https://staging.example.com/api/dashboard"
      cookies: "$STAGING_SESSION"

    # AWS S3
    "/api/data":
      source: "s3://bucket/data.json"

    # Cloudflare R2
    "/api/reports":
      source: "r2://bucket/reports.json"

    # Dynamic generation via script
    "/api/dynamic":
      source: "script://fixtures/generate.ts"

    # Pass through to real API (no mocking)
    "/api/public/*":
      source: passthrough

Feeding Fixtures to Your App

Defining fixtures in .haystack.json tells Haystack what data to use. You also need middleware to intercept requests:

Option 1: Vite Plugin (Recommended for Vite apps)

// vite.config.ts
import { haystackFixtures } from '@haystackeditor/verify/vite';

export default defineConfig({
  plugins: [
    haystackFixtures(), // Reads .haystack.json, intercepts matching routes
  ],
});

The plugin activates when HAYSTACK_FIXTURES=true is set in your dev_server.env.

Option 2: MSW (Framework-agnostic)

// src/mocks/handlers.ts
import { haystackHandlers } from '@haystackeditor/verify/msw';

export const handlers = [
  ...haystackHandlers(), // Auto-generated from .haystack.json
];

Option 3: Manual Middleware

// Express example
app.use(async (req, res, next) => {
  if (process.env.HAYSTACK_FIXTURES !== 'true') return next();

  const fixture = findMatchingFixture(req.path); // from .haystack.json
  if (fixture) {
    const data = await loadFixture(fixture.source);
    return res.json(data);
  }
  next();
});

Choosing a Fixture Source

| Source | Best For | Tradeoffs | |--------|----------|-----------| | file:// | Small, stable data (<50KB) | Committed to repo, versions with code | | https:// | Dynamic data from staging | Requires network access, may be flaky | | s3:// | Large datasets (>50KB) | Needs AWS credentials, not in repo | | r2:// | Large datasets on Cloudflare | Needs R2 credentials, not in repo | | script:// | Generated/computed data | Runs at startup, flexible but slower |

Rule of thumb:

• Auth mocks, config, small responses → file:// (commit to repo)
• Large API responses, analysis data → s3:// or r2://
• Data that changes per-run → script://

Login-Required Data (Cookie-Based Auth)

For endpoints that require a logged-in session (not just an API token), use the cookies field:

Step 1: Log into your staging site in a browser

Step 2: Export your session cookies

# Chrome DevTools: Application > Cookies > [your-domain]
# Copy the cookie string, e.g.: "session=abc123; csrf=xyz789"

Step 3: Store as a secret

haystack secrets set STAGING_SESSION "session=abc123; csrf=xyz789"

Step 4: Reference in .haystack.json

verification:
  fixtures:
    "/api/dashboard":
      source: "https://staging.example.com/api/dashboard"
      cookies: "$STAGING_SESSION"

Note: Session cookies typically expire after some time. You'll need to re-export them periodically.

For long-lived access, consider:

• Creating a dedicated test account with non-expiring sessions
• Using API tokens instead of cookies where possible
• Using script:// fixtures that perform OAuth programmatically

MCP Tools

Setup & Configuration

| Tool | Description | |------|-------------| | haystack_setup_start | Start interactive setup wizard with auto-detection | | haystack_setup_answer | Provide answer to setup question | | haystack_load_config | Load .haystack.json configuration | | haystack_validate_config | Validate config for errors and warnings | | haystack_detect_project | Auto-detect framework, monorepo, services | | haystack_get_instructions | Get context-aware next steps |

Fixtures

| Tool | Description | |------|-------------| | haystack_load_fixtures | Load fixtures from all sources | | haystack_generate_fixture | Generate skeleton fixture file | | haystack_record_fixture | Record API response as fixture file |

Verification

| Tool | Description | |------|-------------| | haystack_get_verify_targets | Get dev server config and verification commands | | haystack_get_auth_config | Get auth bypass environment variables | | haystack_get_flow_templates | Get pre-built verification command templates |

Secrets

| Tool | Description | |------|-------------| | haystack_secrets_info | Get documentation about the secrets system | | haystack_secrets_list | List configured secrets (keys only, never values) | | haystack_secrets_set | Store a secret with zero-knowledge encryption | | haystack_secrets_delete | Delete a stored secret |

Auto-Detection

The haystack_setup_start and haystack_detect_project tools automatically detect:

• Framework: Vite, Next.js, Remix, Nuxt, SvelteKit, Astro, CRA, Cloudflare Workers
• Package Manager: npm, pnpm, yarn, bun (from lockfiles)
• Monorepo: pnpm workspaces, Lerna, Nx, Turborepo, Rush
• Services: Scans workspace packages for runnable services
• Auth Bypass: Checks .env.example for common auth skip patterns

This provides smart defaults in the setup wizard, reducing manual configuration.

Verification Command Templates

Use haystack_get_flow_templates to get pre-built command sets:

| Template | Description | |----------|-------------| | frontend | React/Vue/Svelte apps (build, lint, typecheck) | | node_api | Node.js APIs (build, lint, test) | | cloudflare_worker | Cloudflare Workers (deploy --dry-run, typecheck) | | python | Python projects (pytest, mypy, ruff) | | monorepo | Monorepo with pnpm workspaces | | minimal | Just build command |

Environment Variables & Secrets

For fixtures from external sources or other sensitive values, use Haystack Secrets:

# Store a secret (zero-knowledge encrypted)
haystack secrets set STAGING_API_TOKEN "your-token"
haystack secrets set AWS_ACCESS_KEY_ID "AKIA..."
haystack secrets set AWS_SECRET_ACCESS_KEY "..."

Then reference in .haystack.json:

fixtures:
  "/api/data":
    source: "https://staging.example.com/api/data"
    headers:
      Authorization: "Bearer $STAGING_API_TOKEN"

  "/api/reports":
    source: "s3://bucket/reports.json"
    # Uses $AWS_ACCESS_KEY_ID and $AWS_SECRET_ACCESS_KEY automatically

Common secrets:

| Variable | Purpose | |----------|---------| | STAGING_API_TOKEN | Token for staging API access | | AWS_ACCESS_KEY_ID | AWS S3 credentials | | AWS_SECRET_ACCESS_KEY | AWS S3 credentials | | R2_ACCESS_KEY_ID | Cloudflare R2 credentials | | R2_SECRET_ACCESS_KEY | Cloudflare R2 credentials | | CLOUDFLARE_ACCOUNT_ID | For R2 bucket access | | DATABASE_URL | Database connection string |

See /setup-haystack-secrets skill for full documentation.

Schema Reference

Full .haystack.json schema:

version: "1"                    # Required, must be "1"
name: my-app                    # Optional display name
package_manager: pnpm           # Optional: npm, pnpm, yarn, bun (auto-detected)

# Simple mode: single dev server
dev_server:
  command: pnpm dev             # Required: command to start server
  port: 3000                    # Required: port number
  ready_pattern: "Local:"       # Optional: stdout pattern when ready
  env:                          # Optional: environment variables
    SKIP_AUTH: "true"

# Monorepo mode: multiple services (mutually exclusive with dev_server)
services:
  frontend:
    root: ./                    # Optional: directory relative to repo root
    command: pnpm dev           # Required: command to start service
    port: 3000                  # Required for servers, omit for batch jobs
    ready_pattern: "Local:"     # Optional: stdout pattern when ready
    type: server                # Optional: 'server' (default) or 'batch'
    env:                        # Optional: environment variables
      SKIP_AUTH: "true"

# Verification configuration
verification:
  commands:                     # Optional: commands run during sandbox creation
    - name: build               # Human-readable name
      run: pnpm build           # Shell command to run

  fixtures:                     # Optional: API mock data sources
    "/api/users/*":             # URL pattern to intercept
      source: "file://..."      # Data source (file://, https://, s3://, r2://, script://, passthrough)
      headers:                  # Optional: headers for remote sources
        Authorization: "Bearer $TOKEN"