tson-mongo 🚀

Parses TSON (Token-Structured Object Notation) responses back into MongoDB documents, filter queries, update pipelines, and native BSON types.

Reduce your LLM output generation costs & latency by 30% to 50% when requesting structured outputs from models like Gemini, OpenAI GPT-4o, Claude, and Llama.

💡 Why tson-mongo?

When requesting structured data or database payloads from an LLM:

• ❌ High Output Costs: LLM output tokens cost 3x to 4x more than input tokens.
• ❌ Slower Generation Latency: Forcing models to generate verbose JSON ({"name": "...", "status": "..."}) slows down generation speed significantly.
• ❌ Formatting Bloat: JSON brackets, quotes, and repeated keys waste generation tokens.

tson-mongo allows your AI models to output compact TSON format, and instantly converts the AI response into ready-to-write MongoDB payloads (insertMany, find, updateOne).

📊 Output Token Savings & Speed Benchmark

| Task | Standard JSON Output | TSON Format Output | Output Tokens Saved | Speedup | | :--- | :--- | :--- | :--- | :--- | | Extract 10 Documents | ~1,200 tokens | ~610 tokens | 590 tokens | ~2x Faster ⚡ | | Extract 50 Documents | ~5,800 tokens | ~2,350 tokens | 3,450 tokens | ~2.2x Faster ⚡ | | Generate DB Filter | ~150 tokens | ~80 tokens | 70 tokens | ~1.8x Faster ⚡ |

📦 Installation

npm install tson-mongo
# or
yarn add tson-mongo
# or
pnpm add tson-mongo

⚡ Quick Start

1. Parse LLM Response directly into MongoDB Documents

import { tsonToMongo } from 'tson-mongo';
import { GoogleGenerativeAI } from '@google/generative-ai';

const ai = new GoogleGenerativeAI(process.env.GEMINI_API_KEY);
const model = ai.getGenerativeModel({ model: 'gemini-1.5-pro' });

// Ask LLM to output in compact TSON format
const prompt = `
Extract structured user profiles from the email text and respond strictly in TSON format with headers: _id, name, email, role, active, createdAt.
`;

const response = await model.generateContent(prompt);
const tsonText = response.response.text();

// Convert TSON response back to MongoDB documents with BSON type restoration
const documents = tsonToMongo(tsonText, {
  restoreDates: true,
  restoreObjectIds: true,
  trackSavings: true
});

// Insert straight into database!
await db.collection('users').insertMany(documents);

2. Generate MongoDB Query Filters from LLMs

import { tsonToMongoFilter } from 'tson-mongo';

const llmFilterResponse = `
status: active, pending
role: Lead Architect
`;

const filter = tsonToMongoFilter(llmFilterResponse);
console.log(filter);
// Output: { status: { $in: ['active', 'pending'] }, role: 'Lead Architect' }

// Use directly in MongoDB query!
const results = await db.collection('users').find(filter).toArray();

3. Generate MongoDB Update Pipelines

import { tsonToMongoUpdate } from 'tson-mongo';

const llmUpdateResponse = `
status: completed
paidAt: 2026-08-27T15:45:00.000Z
totalAmount: 1499.99
`;

const updatePipeline = tsonToMongoUpdate(llmUpdateResponse);
console.log(updatePipeline);
// Output: { $set: { status: 'completed', paidAt: Date(...), totalAmount: 1499.99 } }

await db.collection('orders').updateOne({ _id: orderId }, updatePipeline);

4. Live Terminal Test & Demo

Run the interactive CLI demonstration to test output parsing:

npm run demo

5. Persistent Token Savings Tracker (SAVINGS_TRACKER.json)

tson-mongo tracks cumulative output token savings automatically in SAVINGS_TRACKER.json when trackSavings: true is passed:

import { getGlobalSavingsTracker } from 'tson-mongo';

const tracker = getGlobalSavingsTracker();
console.log(tracker.getStats());

🛠️ API Reference

tsonToMongo(tsonString, options?)

Parses TSON tabular or compact strings back into MongoDB objects / document arrays.

Options (TsonToMongoOptions)

| Option | Type | Default | Description | | :--- | :--- | :--- | :--- | | restoreDates | boolean | true | Reconstructs ISO 8601 strings to native JS Date objects. | | restoreObjectIds | boolean | true | Reconstructs oid:64f... strings. | | castNumbers | boolean | true | Casts numeric strings back to numbers. | | castBooleans | boolean | true | Casts 'true' and 'false' to boolean primitives. | | trackSavings | boolean | false | Updates SAVINGS_TRACKER.json with cumulative metrics. |

tsonToMongoFilter(tsonString, options?)

Converts a TSON string from an LLM response into a MongoDB Query Filter object.

tsonToMongoUpdate(tsonString, options?)

Converts a TSON string into a MongoDB $set update pipeline object.

👤 Author & Contact

Developed with ❤️ by Abhi Asok.

For business inquiries, collaboration, or support:

• 📧 Email: [email protected]
• 📞 Phone / WhatsApp: +91 9142125724
• 💼 LinkedIn: linkedin.com/in/abhi-asok-09439788
• 🐙 GitHub: github.com/AbhiArvension

📄 License

MIT © Abhi Asok