sypi-plumber

Query a large JSON/CSV blob or captured tool response as SQL and return only the requested rows. The extension does not place 20k lines of nested JSON into context. It ingests the source once into a normalized relational view under ~/.sypi. The source never enters the repository or context. Only the queried result set returns. Each op re-reads and revalidates the file. The saved resource is context, not disk I/O.

One tool, plumber, with behaviors behind op::

• op: "load" - Ingest a source and make it the current table.
  • path (preferred): A file path. The blob stays on disk and never enters context. This path pairs with netpeek-captured responses.
  • data: Inline source text. The blob enters context, so use it for small or medium inputs.
  • format: json | csv | auto (default auto, sniffs from the text).
  • rootPath: dotted path to the array/object to tabularize, e.g. data.items. Omit it to auto-detect the largest array-of-objects (so a { data: { items: [...] } } envelope just works).
  • name: display name for the table (defaults to the filename).
• op: "schema" - List the current table's columns with inferred types and row count, so the model knows what to query without ever seeing the data.
• op: "query" - Run a sql SELECT against the current table. It returns the matching rows plus matched (total before LIMIT) and truncated.

Nested JSON objects flatten to dotted columns (user.addr.city); arrays and over-deep subtrees become JSON-text cells so they stay queryable. CSV cells are lightly coerced to number/boolean/null so WHERE age > 30 works on text.

Supported SQL

A small, strict, single-table subset (hand-rolled, no dependency):

SELECT ( * | col [AS alias] | FN(col|*) [AS alias] , ... )
[ FROM name ]                       -- accepted, ignored (one table per load)
[ WHERE expr ]                      -- = != < <= > >= LIKE IN IS [NOT] NULL, AND/OR/NOT, ( )
[ GROUP BY col , ... ]              -- with COUNT / SUM / AVG / MIN / MAX
[ ORDER BY col [ASC|DESC] , ... ]
[ LIMIT n [OFFSET m] ]

No JOIN, subquery, HAVING, or write statements; anything outside the grammar is a clean parse error, never a silent misparse. Everything is deterministic with zero LLM in the query path.

Bounds

Sized for an 8GB ceiling and enforced on load: input capped at 64 MiB (checked before the file is read), 200k rows, 512 columns, flatten depth 8. Query results default to 1,000 rows and are hard-capped at 10,000, so a bare SELECT * can never dump the whole table back into context.

Design notes

• Imports only the extension API (extensionStateDir, readAtomicStateObject, updateAtomicStateObject).
• Exactly one registered tool; all behavior via op:.
• The loaded table is cached at ~/.sypi/plumber/current.json, never the user's repo.
• Dependency-free. The JSON/CSV parsers, the SQL parser, and the query engine are all hand-rolled.