System2

System2 is a multi-agent system for data engineering, analysis, and statistical reasoning. It adapts to your existing data stack or builds one from scratch. You talk to the Guide. It spawns a team that plans, builds pipelines, analyzes data, catches statistical fallacies, and delivers interactive output you can inspect end to end.

Named for Kahneman's deliberate, quantitative mode of reasoning, System2 is the bicycle for your analytical mind.

It exists to help people think more clearly about complex questions and empower everyone, regardless of skill, to acquire and interpret data, and share rigorous analysis grounded in evidence and methods they can inspect.

https://github.com/user-attachments/assets/d2f688d9-15fe-41a0-a4d0-46b2e87d2236

Quick start

Prerequisites

1. Node.js 20+ and pnpm 8+ (in a terminal, run node -v and pnpm -v to check)

2. LLM credentials (at least one required). Two tiers, combinable, both configurable via system2 config:

   • OAuth (subscription accounts): use your existing AI subscription. Recommended for sustained workloads, since the provider bills a flat subscription fee instead of per token. Each provider runs one model family for all agent roles (Claude Opus for Anthropic, GPT-5.x for OpenAI Codex, GPT-5.x for GitHub Copilot); see docs/agents.md for the resolver and override path. Supported: Anthropic (paid plan required), OpenAI, and GitHub Copilot. Free tiers vary by provider — see each provider's pricing page.
   • API keys: an alternative tier. Generally costlier per use than a subscription; free-tier models avoid the cost but come with tight rate limits and higher latency. OpenRouter gives access to multiple models with one key; Anthropic, Google, OpenAI, and others also work. You can add multiple keys for rotation and multiple providers for failover.

   Combining both tiers gives you OAuth as the primary path with API keys as fallback when a subscription rate-limits.

3. Brave Search API key (highly recommended). Enables agents to search the web and fetch web pages content. This is useful for researching APIs, documentation, and data sources on the web. Get one here.

Install and run

Open a terminal and run:

pnpm setup                                # one-time per machine: configures pnpm's global bin dir + PATH (skip if already done)
# Close and reopen your terminal here so the new PATH is picked up.
pnpm add -g @diegoscarabelli/system2      # install System2 globally
system2 init                              # first-run setup (see below)

If system2 isn't found after install, your pnpm global bin directory isn't on PATH. Run pnpm setup (it's idempotent), close and reopen your terminal, and try again. pnpm bin -g should print a directory that's on your shell's PATH. There's no need to clone this repo to install — npm has the package.

system2 init creates ~/.system2/ (with config.toml initialized from a template and an empty auth/ dir) and immediately hands off to system2 config, which creates ~/.system2/auth/.auth.toml with your LLM credentials (OAuth and/or API keys) and (optionally) the Brave Search key in the same flow.

system2 start            # starts the server and opens the browser

system2 start launches the server and opens your browser at http://localhost:4242. On first launch, the Guide walks you through the UI, learns your preferences, detects your existing infrastructure, and sets up your data stack and development environment.

The Guide detects that the knowledge files are still templates and runs the onboarding skill through the web UI. It introduces itself, learns about you and your goals, and detects your system. Unless you direct it otherwise, the recommended stack includes:

• An analytical database (PostgreSQL with TimescaleDB by default)
• A shared Python environment with notebooks and data libraries
• An ETL framework from openetl_scaffold
• An orchestrator (Prefect or Airflow)

The Guide adapts to what you already have: if you have an existing database, orchestrator, or pipeline repo, it integrates with those instead. By the end, you have a working data stack, a code repository ready for pipelines, and knowledge files populated with your setup.

Managing and updating

Once the server is running, a small set of commands covers the day-to-day lifecycle: checking status, shutting down, managing credentials, and upgrading.

Daily operation. Use system2 status to confirm the server is up, and system2 stop to shut it down when your work ends. The next system2 start creates a timestamped backup of ~/.system2/ before initializing.

system2 status           # check whether the server is running

system2 stop             # shut down gracefully

Credentials and services. Use system2 config to add or remove OAuth providers, add or rotate API keys, set the Brave Search key, and reorder failover priority across credentials.

system2 config           # interactive: manage OAuth, API keys, and services

Operational parameters. Hand-edit ~/.system2/config.toml to tune per-agent behavior, register databases, or adjust operational defaults. The Guide also edits this file during onboarding (for example, to register database connection parameters of your data stack). The command system2 init initializes this file to a template if it is not found.

Upgrading. Pull the latest release from npm:

pnpm update -g @diegoscarabelli/system2

Key capabilities

Multi-agent system built for data work. The Guide is your single point of contact with long-term memory of your interactions. Work is organized into projects, each with its own team: a Conductor that plans and orchestrates, a Reviewer that catches statistical fallacies, and optional Workers that execute in parallel. Multiple projects can run concurrently, each with dedicated agents. A Narrator curates memory on a schedule.

Structured collaboration. Agents coordinate through real-time messages and manage work on a database-backed kanban board (visible to you) with task hierarchies, dependencies, and comment threads. Every project follows a plan-approve-execute cycle: the Conductor proposes, you review and approve before work begins.

Knowledge that accumulates. Every conversation builds on the last. Agents refine git-tracked markdown files storing your preferences, infrastructure setup, lessons learned, and reusable skills. Daily summaries and project stories are written automatically so nothing is lost. Stop the server, restart it days later: the team picks up where it left off.

Interactive artifacts. Agents can build and display anything in the UI on demand: live dashboards querying your databases (PostgreSQL, ClickHouse, DuckDB, Snowflake, BigQuery, MySQL, MSSQL, SQLite), research articles, Jupyter notebooks, reports. Artifacts appear alongside the conversation with live reload, so you see results the moment they're ready and iterate on them in place.

Built-in tools and domain skills. Agents come equipped with shell access, file operations, database queries, web search, inter-agent messaging, and self-scheduling reminders, all with safety guards and role-based restrictions. Built-in skills cover data infrastructure (Airflow, Prefect, TimescaleDB, SQL modeling), live dashboard building, statistical analysis, code review, and reasoning fallacy detection. Agents can also create new skills at runtime.

Any LLM, automatic failover. OpenRouter, Anthropic, Google, OpenAI, Cerebras, Mistral, Groq, xAI, and any OpenAI-compatible endpoint. Automatic key rotation, provider failover with backoff, and time-based cooldowns.

Configuration

Settings live in two files under ~/.system2/:

• config.toml — operational settings you hand-edit: per-agent overrides, database connections, scheduler cadence, backup policy, log rotation, and other tunables. Created by system2 init.
• auth/.auth.toml — credentials and service toggles, written exclusively by system2 config (OAuth providers, API keys, Brave Search, web-search enablement).

System2 supports an OAuth subscription tier (Anthropic, OpenAI Codex, GitHub Copilot, tried first) and an API key tier (Anthropic, Google Gemini, OpenAI, OpenRouter, Cerebras, Groq, Mistral, xAI, and any OpenAI-compatible endpoint, used as failover). Either tier alone is enough; combining them gives subscription-first with per-token failover.

See docs/configuration.md for the full schema, model selection rules, and failover behavior.

Tech stack

| Layer | Technology | | ----- | ---------- | | Runtime | Node.js, TypeScript | | Agent SDK | pi-coding-agent | | HTTP / WebSocket | Express, ws | | Database | SQLite (WAL mode) | | UI | React 18, Zustand, Vite | | Scheduling | Croner | | Package manager | pnpm |

What lives where

System2's home directory is ~/.system2/. It holds all system state: configuration, the internal database (projects, tasks, agents), knowledge files, project workspaces, chat histories, and logs. The directory is a git repository, so knowledge file changes are version-tracked and reversible.

~/.system2/
├── .gitignore
├── app.db                           SQLite database (gitignored)
├── config.toml                      Operational settings, hand-edited (0600, gitignored)
├── auth/                            Machine-managed credentials (0700, gitignored)
│   ├── .auth.toml                    Auth + service config, written exclusively by `system2 config` (0600)
│   └── {provider}.json              OAuth tokens (one file per logged-in provider, e.g. anthropic.json, openai-codex.json) (0600)
├── knowledge/                       Persistent knowledge (git-tracked)
│   ├── conductor.md                 Conductor role-specific knowledge
│   ├── daily_summaries/             Daily activity logs
│   ├── guide.md                     Guide role-specific knowledge
│   ├── infrastructure.md            Your data stack, servers, tools
│   ├── memory.md                    Long-term learnings (Narrator-maintained)
│   ├── narrator.md                  Narrator role-specific knowledge
│   ├── reviewer.md                  Reviewer role-specific knowledge
│   ├── user.md                      Your background and preferences
│   └── worker.md                    Worker role-specific knowledge
├── logs/                            Server logs (gitignored)
├── projects/
│   └── {dir_name}/                  {id}_{slug} from project record (e.g. 1_linkedin-campaign)
│       ├── artifacts/               Published reports and dashboards
│       │   ├── plan_{uuid}.md       Conductor's proposal (pre-approval)
│       │   └── project_story.md     Final narrative (Narrator)
│       ├── log.md                   Continuous project log (Narrator)
│       └── scratchpad/              Working files
├── artifacts/                       Project-free artifacts (not tied to any project)
├── server.pid                       PID file when server is running (gitignored)
├── sessions/                        Agent conversations as JSONL (gitignored)
├── skills/                          User-created workflow instructions
└── venv/                            Shared Python environment (notebooks, data libraries) (gitignored)

Agents run in a shell and can work with any directory on your machine. For data engineering work, they typically create and manage code in external repositories (e.g. ~/repos/system2_data_pipelines), keeping pipeline code separate from the System2 home directory.

On every system2 start, the system creates a timestamped backup of ~/.system2/ before the server initializes (stored as ~/.system2-auto-backup-YYYY-MM-DDTHH-MM-SS/, with a 24-hour cooldown between copies and automatic pruning to keep only the 3 most recent). Both limits are configurable via [backup] in config.toml.

See docs/knowledge-system.md for how knowledge files are injected into agent prompts, file ownership, and the git tracking model. See docs/architecture.md for the full runtime directory layout.

Documentation

For a deeper look at how System2 works, see the developer documentation. It covers the agent system, tools, database schema, knowledge persistence, skills, scheduling, and the real-time WebSocket protocol. For contributing, see CONTRIBUTING.md.

License

AGPL-3.0-only. See LICENSE.