CAG (Chunked Augmented Generation)

CAG is a TypeScript library specifically designed to overcome context window limitations in browser-based AI models, with particular optimization for Chrome's built-in Gemini Nano implementation. It enables efficient processing of large text inputs by intelligently managing chunk boundaries while maintaining semantic coherence.

Overview

CAG addresses the fundamental challenge of processing large inputs within browser-based AI models' restricted context windows. It employs sophisticated chunking strategies and browser-specific optimizations to extend the effective processing capacity while maintaining performance stability.

Practical Example: Document Summarization

Imagine you have a 100-page research paper in your browser that you need to summarize. Without CAG, you'd hit context window limits trying to process it all at once. Here's how CAG handles it:

import { CAG } from "cag-js";

// Initialize CAG with settings optimized for document summarization
const cag = new CAG({
  chunkSize: 24576,  // About 6,144 tokens, optimal for Gemini Nano
  chunkOverlap: 4096,  // Ensures context continuity between sections
  iteration_limit: 3,   // Number of refinement passes
  iteration_output_token_limit: 2048  // Target summary length
}, "Summarize the following text section, focusing on key findings: {text}");

// Your 100-page research paper as text
const researchPaper = `[Your long research paper content...]`;

// Generate a coherent summary
try {
  const summary = await cag.generate_recursive(researchPaper);
  console.log("Research Paper Summary:", summary);
} catch (error) {
  console.error("Error generating summary:", error);
}

What happens behind the scenes:

1. CAG splits your 100-page paper into overlapping chunks that fit Gemini Nano's context window
2. Each chunk is summarized while maintaining connections to surrounding content
3. These summaries are combined and refined over multiple passes
4. You get a coherent, well-structured summary of the entire paper, processed entirely in your browser

Key Features

• 🔄 Dual Processing Modes: Sequential and Recursive Generation
• 📏 Intelligent Chunk Management with Context Preservation
• ⚙️ Browser-Optimized Resource Management
• 🔍 Chrome Gemini Nano Integration
• 📊 Comprehensive Performance Metrics
• 🛡️ Privacy-Preserving Local Processing
• 📚 Full TypeScript Support

Installation

npm install cag-js

Try it Out: CAG-JS Playground

Want to see CAG in action before integrating it into your project? Check out the CAG-JS Playground - an interactive environment where you can experiment with CAG's capabilities using Chrome's built-in Gemini Nano model.

The playground provides:

• 🎮 Live examples of content processing
• 📝 Ready-to-use components
• 🔍 Interactive testing environment
• 💡 Implementation examples

To get started with the playground:

1. Install Chrome Canary

2. Enable Gemini Nano support in chrome://flags:

   • Enable "Prompt API for Gemini Nano"
   • Enable "Enables optimization guide on device"

3. Clone and run the playground:

   git clone https://github.com/vivekVells/cag-js-example
   cd cag-js-example
   npm install
   npm run dev

Usage

Basic Implementation

import { CAG } from "cag-js";

const cag = new CAG({
  chunkSize: 24576,  // Optimized for Chrome's Gemini Nano context window
  chunkOverlap: 4096,
  iteration_limit: 5,
  iteration_output_token_limit: 6144  // Gemini Nano token limit
}, "Process the following text: {text}");

// Sequential Processing
const result = await cag.generate_sequential(longText);

// Recursive Processing
const recursiveResult = await cag.generate_recursive(longText);

Configuration

interface CAGConfig {
  chunkSize: number;         // Characters per chunk (recommended: 24576 for Gemini Nano)
  chunkOverlap: number;      // Overlap between chunks
  iteration_limit?: number;  // Maximum recursive iterations
  iteration_output_token_limit?: number;  // Target output length in tokens
}

Processing Modes

Sequential Generation

• Processes chunks independently in sequence
• Maintains document structure
• Ideal for parallel processing
• Best for content where chunks can be processed independently

Recursive Generation

• Hierarchical chunk processing
• Progressive refinement of output
• Maintains semantic coherence across chunks
• Optimal for summarization and content transformation

Performance Considerations

CAG's performance has been extensively benchmarked across different content lengths:

• Small (CWQ ≤ 1): Optimal for content within single context window
• Medium (1 < CWQ ≤ 2): Efficient dual-window processing
• Large (2 < CWQ ≤ 3): Balanced performance with chunk coordination
• Extra Large (3 < CWQ ≤ 4): Sophisticated chunk management
• Humongous (CWQ > 4): Advanced resource optimization

Where CWQ (Context Window Quotient) = Content Length / (Base Token Window × Character-to-Token Ratio)

Browser Requirements

• Chrome with Gemini Nano support
• Minimum 4GB available memory
• WebAssembly support
• Hardware acceleration recommended

Use Cases

• Document Summarization
• Content Expansion
• Technical Documentation Processing
• Large-Scale Text Analysis
• Knowledge Base Processing
• Multi-Stage Content Refinement

Development

1. Clone the repository:

git clone https://github.com/yourusername/cag-js.git
cd cag-js

2. Install dependencies:

npm install

3. Build:

npm run build

4. Test:

npm test

Error Handling

CAG implements robust error handling for:

• Browser resource constraints
• Context window limitations
• Chunk processing failures
• Memory management issues

Contributing

1. Fork the repository
2. Create feature branch: git checkout -b feature/enhancement
3. Commit changes: git commit -m 'Add enhancement'
4. Push: git push origin feature/enhancement
5. Submit Pull Request

License

MIT License - see LICENSE file

Support

• GitHub Issues
• Documentation

Acknowledgments

Built with:

• Chrome's Gemini Nano Implementation
• LangChain Text Splitters
• TypeScript