contextplusplus

contextplusplus turns public URLs and provider responses into clean, LLM-friendly Markdown. A shared Effect-TS v3 engine powers a CLI, an Express HTTP API, and an MCP server. A public Next.js website and authenticated dashboard provide the product experience around that engine.

This README is the developer's first-day guide: it explains the system model, where changes belong, how to run the useful surfaces, and how work reaches production. Detailed inventories and runbooks live under .agent-docs/reference/; executable engineering contracts live under .agent-docs/spec/.

What the product includes

| Surface | What users do with it | Main owner | | --- | --- | --- | | CLI (contextplusplus, ctxpp) | Install integrations, authenticate, convert URLs, render saved data, and manage API keys/connections | apps/cli | | HTTP API | Convert URLs, run research, expose service metadata, and serve auth, usage, billing, and admin flows | apps/api | | MCP server | Give agents hosted planning, search, and evidence extraction | apps/mcp | | Website | Learn about the product, pricing, providers, and CLI/MCP installation | apps/website | | Dashboard | Sign in, manage account/API keys/MCP connections, inspect usage, and manage billing | apps/dashboard | | Email relay | Deliver the app-owned CLI device-verification email through Cloudflare Email Service | workers/email-sender |

🤖 AI Agent Fast-Track (Copy & Paste Prompt)

To configure contextplusplus with your coding agent (Claude Code, Cursor, Codex, Gemini, Antigravity, OpenCode, Devin, Zed), copy and paste this prompt to your agent:

Install and configure contextplusplus for my coding environment:
1. Ask me for my email address to register my account (or confirm if I want to use --anonymous).
2. Run `npx -y contextplusplus onboard --email <my_email>` (or `--anonymous`).
3. Verify the setup by running `npx -y contextplusplus doctor`.
For full instructions and options, see https://contextplusplus.com/agent.md

Automated agent documentation is served directly at agent.md and contextplusplus.com/agent.md.

Human / Interactive Installation

To run the guided interactive installer directly in your terminal:

npx -y contextplusplus install

The implemented commands, routes, tools, and screens are catalogued in Use Cases, Screens, and Flows.

The system in one request

All three conversion boundaries converge on the same core workflow:

user, client, or agent
        |
        +---- CLI -------- hosted API or --local core runtime
        +---- HTTP API --- authentication, quota, and response negotiation
        +---- MCP -------- authentication, tool schema, and research budgets
                           |
                           v
                   ProcessRunWorkflow
                           |
                           v
                 ProviderBatchDispatcher
                           |
                           v
                    ProviderRegistry
                           |
             +-------------+--------------+
             |                            |
       exact provider              generic fallback
       strategy chain          scrape.do -> Kernel browser
             |                            |
             +-------------+--------------+
                           |
                cache + decode + render
                           |
                optional artifact bundle
                           |
                           v
         JSON, Markdown, or structured MCP response

The important distinction is between a provider definition and a live strategy. Pure endpoint definitions under packages/core/src/providers/definitions/<service>/<endpoint>/ parse URLs and describe endpoints. Strategies under packages/core/src/infrastructure/providers/ build requests, call external systems, decode responses, and render output. Catalog assembly validates and connects both sides. See Architecture and Provider Catalog and Rendering.

Architectural boundaries

packages/core is the center of gravity. Interface apps should remain thin; the website and dashboard use a separate browser-safe shared-package path.

apps/api, apps/cli, apps/mcp
              |
              v
    runtime composition / infrastructure
              |
              v
      application workflows + ports
              |
              v
         domain schemas and errors

apps/website ----\
                  +--> packages/ui --> packages/contracts --type-only--> core domain
apps/dashboard --/          |
                           +--> packages/markdown

Keep these invariants in mind before editing backend code:

• Domain code owns schemas, branded IDs, requests, results, provider contracts, and tagged errors.
• Application code owns workflows and ports. It must not import infrastructure or interface code.
• Infrastructure implements application ports and composes live Effect layers.
• Effect.runPromise, runFork, and runSync stay in API, CLI, and MCP boundary entrypoints.
• NodeNext TypeScript uses .js extensions for relative backend imports.
• Broad rules belong in deterministic tests or scripts/check-rules.mjs, not only in prose.

The nearest AGENTS.md contains local deltas. Read it before changing a nested area; its sibling CLAUDE.md points to the same instructions.

Five-minute developer setup

Prerequisites:

• Node.js >=22
• Corepack with pnpm 10.28.2
• Git
• A same-filesystem location for linked task worktrees

Keep the primary checkout clean and create a worktree for real tasks:

git clone https://github.com/yigitkonur/contextplusplus.git
cd contextplusplus
corepack enable

git fetch origin --prune
git worktree add .worktrees/my-task -b my-task origin/main
cd .worktrees/my-task
make worktree-bootstrap
pnpm run agent:check

# When finished and merged:
make worktree-clean

make worktree-bootstrap installs the frozen lockfile from pnpm's shared store, sets the repository hooks, and copies safe ignored configuration from the primary checkout when it exists. It never shares node_modules and never overwrites an existing destination.

For a task that needs backend credentials, create an ignored .env in the primary checkout before creating the worktree. Start from .env.example, fill only the keys you need, and never commit a secret. The bootstrap copies that file into the task and derives apps/api/.env and apps/mcp/.env because those processes load environment files from their package directories.

For frontend work, create the app-local files in the task worktree:

cp apps/website/.env.example apps/website/.env.local
cp apps/dashboard/.env.example apps/dashboard/.env.local

Use local or non-production values and edit only the needed public NEXT_PUBLIC_* keys. These values are bundled into browser code.

Task workflow

Agent Docs keeps implementation decisions and verification attached to the task, while Git keeps code changes isolated:

origin/main
    |
    +--> task worktree and branch
             |
             +--> Agent Docs task -> plan -> implement -> focused checks
             |
             +--> commit -> push -> exact-SHA CI
                                      |
                                      +--> pnpm run land -> exact-SHA main CI

For non-trivial work:

1. Read root and nearest AGENTS.md files.
2. Create or resume one task under .agent-docs/tasks/ through the matching Agent Docs skill; do not hand-edit the current-task pointer.
3. Load agent-docs-before-dev before implementation so the owning package specs are selected.
4. Make the smallest source-backed change and run focused local guards.
5. Commit task-owned files, push the task SHA, and require terminal exact-SHA CI.
6. Finish/archive the task, then use the guarded landing command.

The canonical lifecycle is .agent-docs/workflow.md. Archived tasks preserve provenance but are never current implementation policy.

Workspace ownership

| Path | Responsibility | Read before broad changes | | --- | --- | --- | | packages/core | Domain, Effect workflows/ports, infrastructure, providers, cache, rendering, templates, and runtime composition | Architecture, Data and State | | apps/api | Express boundary, middleware, routes, readiness, metrics, and HTTP response shaping | Interfaces and Integrations | | apps/cli | contextplusplus/ctxpp commands, hosted/local execution, installer, auth, and local state | Use Cases, Screens, and Flows | | apps/mcp | MCP host, connection route, auth middleware, and research tool adapters | Interfaces and Integrations | | apps/website | Public marketing, pricing, legal, install, and try-it pages | Frontend Architecture | | apps/dashboard | Auth, account, usage, API keys, MCP connections, billing, and device approval | Frontend Architecture | | packages/ui | Shared raw-source React components, API client, Supabase browser client, styles, and Tailwind preset | Frontend Architecture | | packages/contracts, packages/markdown | Browser-safe API types and Markdown utilities | Frontend Architecture | | packages/tsconfig | Shared NodeNext and Next.js TypeScript bases | Codebase Map | | workers/email-sender | Authenticated Cloudflare transactional-email relay | Email Sender Runbook | | sdk | Generated TypeScript and Python clients | Interfaces and Integrations | | supabase | Local Supabase config, seed, and forward-only migrations | Data and State | | test | Unit, integration, E2E, real-Postgres, fakes, and fixtures | Security, Testing, and Risks | | scripts | Rule checks, generators, CI planners, release helpers, and operational tools | Config, Build, Release, and Operations | | plugins/contextplusplus | Distributed Claude/Codex plugin, onboarding assets, and product skills | Codebase Map | | .agent-docs | Workflow, specs, current references, task records, and ignored personal workspace | Agent Docs reference index |

The workspace is declared by pnpm-workspace.yaml: sdk/typescript, apps/*, packages/*, and workers/*, plus the implicit root package. Turborepo owns the declared workspace task graph; root scripts remain the command source of truth.

Run the useful surfaces locally

The Makefile is the safest backend entrypoint because it checks port ownership and will not kill an unrelated process.

| Surface | Command | Local address or behavior | | --- | --- | --- | | API | make local | http://127.0.0.1:3456 | | API on LAN | make local-lan | Port 3456; binds 0.0.0.0 | | MCP | make mcp MCP_PORT=3002 | http://localhost:3002/mcp | | CLI help from source | make cli | Runs the CLI entrypoint through tsx | | Website | pnpm --filter @contextplusplus/website dev | http://localhost:3000 | | Dashboard | pnpm --filter @contextplusplus/dashboard dev | http://localhost:3001 | | Email Worker | pnpm --filter contextplusplus-email-sender dev | Wrangler local runtime |

MCP normally defaults to 3001, which conflicts with the dashboard; the table uses 3002 so all product surfaces can run together. Running the API package directly defaults to 3000, which conflicts with the website, so prefer make local or set an explicit PORT.

Useful API probes after startup:

curl --fail http://127.0.0.1:3456/health
curl --fail http://127.0.0.1:3456/ready
curl --fail http://127.0.0.1:3456/v1/services

/health proves process liveness. /ready probes dependencies and can report cache degradation separately from fatal Supabase readiness. Neither proves an authenticated conversion or a production deployment.

Configuration model

Backend configuration is defined with Effect Config in packages/core/src/application/ports/config-service.ts; .env.example is the human-facing synchronized inventory. Frontend browser values are owned by each app's .env.example and packages/ui/src/env.ts.

| Group | Representative keys | Needed when | | --- | --- | --- | | Providers and research | SCRAPE_CREATORS_API_KEY, SCRAPEDO_API_KEY, KERNEL_API_KEY, LLM_* | Exercising live provider or optional LLM-assisted research paths | | Cache and artifacts | CACHE_BACKEND, REDIS_URL, ARTIFACT_ROOT | Changing persistence or running Redis-backed cache locally | | HTTP and MCP | HOST, PORT, MCP_PORT, MCP_URL, HTTP_REQUEST_TIMEOUT_MS, HTTP_TRUST_PROXY_HOPS | Running or changing server boundaries | | Auth and data | AUTH_MODE, SUPABASE_*, API_KEY_HASH_SECRET, MCP_CONNECTION_CREDENTIAL_HASH_SECRET | Exercising auth, usage, quota, API-key, or connection flows | | Billing | STRIPE_* | Exercising Checkout, Portal, subscription, or webhook behavior | | Email | EMAIL_SENDER_*, EMAIL_FROM_ADDRESS, REOON_API_KEY | Exercising CLI registration and device-verification delivery | | Observability | SENTRY_*, OTEL_EXPORTER_OTLP_ENDPOINT, LANGFUSE_*, METRICS_AUTH_TOKEN | Exercising telemetry, tracing, metrics protection, or operations | | Browser apps | NEXT_PUBLIC_SUPABASE_*, NEXT_PUBLIC_API_BASE, NEXT_PUBLIC_APP_URL, NEXT_PUBLIC_SITE_URL, optional Turnstile site key | Running website/dashboard flows |

Production startup validates required provider, cache, Supabase, Stripe, and URL configuration and rejects AUTH_MODE=optional-dev. Secrets belong in ignored local files or provider/GitHub stores; never place them in docs, tests, task artifacts, logs, or NEXT_PUBLIC_* variables.

The full variable model and hosted topology are documented in Config, Build, Release, and Operations.

Route common changes correctly

| You are changing | Primary owners | Load these project skills | Focused proof starts with | | --- | --- | --- | --- | | Domain schemas, errors, services, layers, or Effect workflows | packages/core/src/{domain,application,infrastructure} | build-effect-ts-v3 | Owning unit/integration test and pnpm run lint:rules | | Provider detection, endpoints, strategies, templates, fixtures, or provider docs | Core provider definition + strategy + template + fixture/catalog consumers | contextplusplus-provider-change; add build-effect-ts-v3 for Effect code | pnpm run fixtures:check-live and pnpm run docs:mintlify:check | | HTTP/OpenAPI/public SDK contract | apps/api, .agent-docs/reference/contracts/openapi.yaml, SDK generators and outputs | contextplusplus-contract-regeneration | pnpm run sdk:check and pnpm run sdk:check:py | | Supabase migration, RLS/RPC, auth, quota, usage, or billing | supabase, generated DB types, core services, boundary consumers | contextplusplus-data-migration-safety + supabase; add supabase-postgres-best-practices for SQL | Migration helper static checks and pnpm run supabase:check | | Website, dashboard, shared UI/contracts/Markdown, or frontend auth | Frontend app plus packages/ui, packages/contracts, and packages/markdown consumers | contextplusplus-frontend-change + nextjs; add react-best-practices for multi-component/hook/performance work | Narrow app/component lint or test selected by the skill | | README, AGENTS, Agent Docs specs/references, or Markdown links | Current source authority and all live link consumers | contextplusplus-documentation | pnpm run docs:links:check and pnpm run agent:check | | CI, landing, npm publishing, migrations, Railway/Vercel, or runbooks | .github, release/deploy scripts, and platform contracts | contextplusplus-release-deploy | Owning planner/contract test plus exact-SHA CI | | Review, security audit, launch readiness, or production risk | The complete affected trust boundary | contextplusplus-production-review | Evidence matrix from the skill; no mutation during review-only work |

Project-scoped skills live in .agents/skills/<name>/SKILL.md and are mirrored under .claude/skills/. Load the matching project skill before editing, then add the technology skill where the table calls for it. Project-specific rules win over generic vendor advice. The root agent manual is the compact trigger index; Agent Docs skills own task lifecycle rather than product behavior.

Generated surfaces and source authority

Do not patch generated output in isolation. Change the authority, run the generator, commit the synchronized output, and run its drift check.

| Generated output | Authority and generator | | --- | --- | | sdk/typescript/src/schema.d.ts and TypeScript SDK surfaces | .agent-docs/reference/contracts/openapi.yaml -> scripts/generate-sdk.ts | | sdk/python/contextplusplus/* | .agent-docs/reference/contracts/openapi.yaml -> scripts/generate_sdk_py.py | | packages/core/src/infrastructure/supabase/database.types.ts | supabase/migrations/* -> scripts/gen-supabase-types.sh | | mintlify/providers/** | Provider catalog/endpoint sources -> scripts/generate-mintlify-docs.ts | | .agent-docs/reference/providers/integrations/** | Provider catalog/endpoint sources -> provider-doc generator |

Build output and local state are not source guidance: node_modules/, dist/, .next/, .turbo/, *.tsbuildinfo, .worktrees/, .claude/worktrees/, and Supabase temp/branch directories can be regenerated. to-delete/ is ignored, recoverable quarantine; do not treat it as current documentation or delete it as part of an unrelated task.

Focused local verification

Local checks should be narrow, deterministic, and tied to the changed contract. Examples:

# Documentation and agent guidance
pnpm run docs:links:check
pnpm run agent:check

# Backend architectural rules
pnpm run lint:rules

# One focused Vitest file
pnpm exec vitest run test/integration/cache-key.test.ts

# Generated-provider, contract, or database drift
pnpm run fixtures:check-live
pnpm run docs:mintlify:check
pnpm run sdk:check
pnpm run sdk:check:py
pnpm run supabase:check

# Always before committing
git diff --check

CI is the build machine. Do not run repo-wide typecheck, full Vitest, next build, verify, or other full-workspace gates on the agent workstation. The owning skill may require a narrower local check; report exactly which proof rung ran rather than describing a focused check as full validation.

CI, landing, release, and deployment

Push routing is fail-closed:

• Executable, generated, MDX, mixed, or uncertain ranges run full CI.
• Approved Markdown/task-only ranges run Docs CI.
• .agents/** or .claude/** changes alone intentionally register no workflow.
• Branch workflows validate only; package and database writes are main-push-only.

After focused local proof:

git push -u origin my-task
python3 scripts/ci-watch.py --sha "$(git rev-parse HEAD)" --branch my-task

The watcher is bounded and ends with CI-DONE <verdict>. Only CI-DONE success for the checked-out SHA is green. For a narrow remote feedback iteration, use:

pnpm run ci:feedback -- <typecheck|build|lint|test|affected|full> [filter]

Inspect a failure with avr run view <run-id> --steps --log-failed --no-pager, fix it, push a new commit, and watch the new SHA. Do not substitute a branch's latest run or a stale green run for exact-SHA evidence.

Land only after branch proof:

pnpm run land

The landing script fetches, rebases if origin/main moved, reclassifies and revalidates the exact branch range, fast-forwards main, pushes it, and waits for exact-SHA main validation.

| Proof tier | What it establishes | | --- | --- | | Focused local guard | The specific changed contract observed by that command | | Exact task-SHA CI | Branch validation; never production side effects | | Exact main-SHA CI | Validation plus successful planned npm/migration reconciliation | | Deployment workflow | Exact Railway deployment ID or Vercel prebuilt deployment reached its terminal gate | | Live probe | The deployed API/MCP/site/dashboard behavior answered on the intended production surface |

Validated main pushes reconcile npm and Supabase state. Package-affecting commits publish core before CLI; safe new migrations can apply automatically, while destructive SQL requires the protected reviewer environment. Full main CI triggers Railway and Vercel only after every planned promotion succeeds. Docs-only main validation reconciles cumulative promotion state but does not trigger deployment. See the operations reference for recovery paths and live-proof requirements.

Common mistakes and quick diagnosis

| Symptom | Likely cause and next step | | --- | --- | | API/MCP cannot see values from root .env | The process loads from its package directory. Use make worktree-bootstrap from a configured primary checkout or copy the task's .env to apps/api/.env and apps/mcp/.env. | | Dashboard or MCP will not start on 3001 | Their defaults collide. Keep dashboard on 3001 and run make mcp MCP_PORT=3002. | | Website or direct API will not start on 3000 | Their defaults collide. Prefer make local for API on 3456. | | A relative backend import fails | NodeNext source imports must include the emitted .js extension. | | A core file needs Effect.runPromise | The program is crossing the runtime boundary in the wrong layer. Return an Effect and run it from the API/CLI/MCP entrypoint. | | Shared UI compiles but has missing styles | Preserve both apps' transpilePackages entries and Tailwind content scan for packages/ui/src. | | A generated drift check fails | Edit the authority/generator input, regenerate, and commit all synchronized outputs. | | CI appears green but the SHA differs | It is stale evidence. Run scripts/ci-watch.py with the full checked-out SHA and branch. | | No workflow appears for a skill-only push | .agents/** and .claude/** alone are intentionally ignored; validate with pnpm run agent:check. | | Historical instructions disagree with code | Current code/tests win. Use the live spec/reference authority and record the supersession; do not revive archived tasks or quarantine. |

Documentation authority and reading order

Use this order when learning or resolving a contradiction:

1. Current code, tests, config, schemas, and generators.
2. Applicable root and nested AGENTS.md instructions.
3. .agent-docs/spec/ for short executable contracts.
4. .agent-docs/reference/ for detailed current knowledge, inventories, and runbooks.
5. The active .agent-docs/tasks/ record for task-local decisions.
6. Archived tasks only for provenance, never current policy.

Recommended deeper reading:

| Question | Authority | | --- | --- | | Where is a file or change owned? | Codebase Map and Onboarding | | How does the runtime/provider pipeline fit together? | Architecture | | What commands, routes, tools, and screens exist? | Use Cases, Screens, and Flows | | What are the external API, SDK, Supabase, Stripe, and observability boundaries? | Interfaces and Integrations | | What schemas, cache, artifacts, and persisted state exist? | Data and State | | How do providers, strategies, templates, and fixtures stay aligned? | Provider Catalog and Rendering | | How do the website, dashboard, auth UI, and shared packages fit together? | Frontend Architecture | | How do environment, CI, publishing, migrations, deployment, and operations work? | Config, Build, Release, and Operations | | What are the security controls, test gates, and known risks? | Security, Testing, and Risks |

If a detailed fact belongs in one of those references, update that authority and keep this README as the approachable map into it.

Hosted research

Configure a Context++ account through the existing installer or ctxpp login. Research uses server-held providers and models with the same HTTP client in npm and plugin distributions.

ctxpp research plan --objective "Compare the current options"
ctxpp research search --query "official documentation"
ctxpp research extract --url https://example.com --requirement "Dated supporting evidence"
ctxpp research schema --command extract

Planning is optional. Search returns leads; extraction returns snapshot-grounded findings. Guidance never executes additional work. JSON is the default output; --output jsonl|markdown and atomic --output-file export are available. Every call supplies its own context. Partial responses identify unfinished URLs; the caller may retry eligible URLs once, for two total submissions. See the research contract.