📈 pi-analytics — Understand Pi Activity Without Sending Data Away

Measure local model, skill, tool, and provider reliability activity without storing conversation or tool content or sending analytics elsewhere.

✨ Features

• Collects content-free metrics automatically after installation.
• Counts settled response cycles, logical LLM calls, skill activations, tool calls, and observed provider errors.
• Reports tool failures and duration, model attribution, and per-response call distributions.
• Separates recovered provider errors from terminal failures.
• Provides Today, rolling 7-day, rolling 30-day, and all-time views through /analytics.
• Stores private, versioned JSONL locally, isolates concurrent writers, and does not start an analytics server.

📦 Install

Install persistently:

pi install npm:@narumitw/pi-analytics

Try the published package without installing:

pi -e npm:@narumitw/pi-analytics

Build and try a local checkout from the repository root:

npm --workspace @narumitw/pi-analytics run build
pi -e ./packages/pi-analytics

The package declares dist/index.ts, so an unbuilt local checkout must run the build before Pi loads the package directory.

The extension uses Node's built-in filesystem APIs and has no native database dependency. Pi extensions run with the Pi process's user permissions, so install only trusted packages.

🚀 Quick start

Complete at least one Pi response, then run:

/analytics

The default overview covers the last seven rolling days:

Analytics · Last 7 days

Response cycles                    83
LLM calls                         192
Calls per response        2.31 · P95 6
Tool calls                        414
Tool errors                         7
Skill activations                  31
Provider errors                     4
Recovered errors                    3

Use the menu to change the range or inspect Skills, Tools, Provider reliability, Response cycles, and Data & privacy. The dashboard includes finalized cycles and omits active work.

📐 Metric definitions

Response cycles and LLM calls

A response cycle starts when Pi begins agent work and normally ends at agent_settled. Retries, overflow-compaction recovery, tool follow-ups, and queued continuations before settlement remain in that cycle.

An LLM call is one logical provider generation confirmed by an assistant-message lifecycle. A provider can make several HTTP attempts within it, so 429 → 429 → 200 counts as one LLM call, three observed HTTP responses, two provider errors, and one recovered generation. Pi cache-warming requests emit provider hooks without an assistant lifecycle; they are excluded from ordinary LLM-call and reliability counts rather than being reported as interrupted generations.

Skills

An activation is User initiated when an observed interactive or RPC /skill:<name> input belongs to an active or subsequently started response cycle. This includes skill commands queued while Pi is streaming. It is Model initiated when Pi's built-in read tool successfully loads the exact canonical SKILL.md path Pi discovered. Each skill counts at most once per response cycle, with explicit user use taking precedence.

Pi does not expose a first-class skill-invocation event or post-chain acceptance event to input observers. The extension does not count unsuccessful reads, provider behavior hidden from Pi, or non-standard loading such as bash plus cat SKILL.md.

Tools

A tool call starts at Pi's tool_execution_start event and finishes at tool_execution_end. The extension stores the tool name, model attribution, timing, completion state, and final error flag. Pi does not expose enough information to distinguish a call blocked by another extension from other tool errors, so both count as errors.

Provider reliability

Pi exposes HTTP responses and final assistant failures, but not every provider-SDK transport retry. The dashboard therefore calls these values observed provider errors. It reports HTTP 429 and 5xx counts, conservative error categories, recovered errors, and terminal failures. Raw error messages are classified in memory and discarded.

💬 Commands

Run /analytics to inspect local usage, skills, tools, and provider reliability over a chosen time range. It accepts no arguments and supports TUI and RPC; print and JSON modes reject it before reading analytics data. Deleting analytics data requires confirmation, and cancellation leaves data unchanged.

🔒 Security and privacy

Current analytics live under:

<pi-agent-directory>/pi-analytics/
├── current
└── generations/
    └── <opaque-generation-id>/
        └── <opaque-writer-id>.jsonl

The opaque IDs are storage coordination identifiers generated by the extension; they are not Pi session IDs. On Unix, directories are restricted to mode 0700 and files to 0600. Linked storage roots, markers, and writer files are rejected.

Stored fields are limited to timestamps and durations; extension-generated record IDs; provider/model IDs and thinking level; tool and skill names; user/model skill source; counts, outcomes, and completion states; HTTP status codes; and classified provider-error categories. Provider-supplied tool-call IDs are replaced with local ordinals before publication. The extension does not store prompts, responses, thinking content, tool arguments or results, raw error messages, HTTP headers, cwd/project/file paths, session names or IDs, or credentials.

Each finalized response cycle is one versioned, newline-terminated frame. Frames larger than 1 MiB are dropped. Local writes receive a 5-second cancellation deadline. Node filesystem cancellation is best-effort, so an operating-system request that has begun may still finish. The extension reports the first failed or timed-out write and a later recovery without exposing filesystem errors.

/analytics streams and validates the active generation, checks cancellation between files and records, and periodically yields to the event loop. It ignores a crash-truncated final frame. Completed malformed frames and unsupported format versions fail closed without replacing existing files.

Clear analytics data

Choose Data & privacy → Clear analytics data… to publish a fresh active generation atomically. Other Pi processes observe it before their next write. A record racing with Clear can land immediately before or after the switch.

The extension then removes obsolete generations. If another process still uses an obsolete file, Clear reports incomplete physical cleanup but the logical clear remains complete. Stop other Pi processes and clear again to retry cleanup. File deletion does not guarantee secure erasure from the storage medium.

🧭 Legacy SQLite data

Versions that used Turso/SQLite stored data in:

<pi-agent-directory>/pi-analytics.db
<pi-agent-directory>/pi-analytics.db-wal

The JSONL version does not open, import, migrate, delete, or rewrite these files. New analytics start empty.

If legacy history matters, stop every old Pi process first and preserve both files together. If it does not matter, stop every old Pi process before deleting both files manually. Never copy or remove only the main DB while an old process may still own its WAL.

🚧 Limitations

• There are no retention settings; records remain until explicitly cleared.
• Analytics are best-effort derived metadata. A failed or interrupted local write may be omitted.
• Large all-time histories require scanning the active JSONL generation when the dashboard opens.
• Prometheus, JSON/CSV export, cloud sync, browser dashboards, token/cost reporting, and project attribution are not included.
• Statistics cover only events visible through Pi's public extension API.
• Dedicated cache-warming observations require an upstream request-kind and completion signal; persisted warming token and cost totals are not imported into analytics.

🗂️ Package layout

packages/pi-analytics/
├── src/                               # Authoritative implementation and helpers
│   ├── index.ts                       # Thin Pi entrypoint
│   └── analytics.ts                   # Collection lifecycle and dashboard command
├── dist/                              # Generated Jiti runtime
├── scripts/build-runtime.mjs          # Runtime builder
└── test/                              # Behavior and lifecycle coverage

The generated runtime is built from src/index.ts and does not import back into src.

🔎 Keywords

Pi extension, Pi coding agent, local analytics, agent skills, tool usage, model calls, provider reliability, JSON Lines, content-free metrics.

📄 License

MIT. See LICENSE.