RafayGen — Advanced AI Chat + CLI Coding Agent

A full-stack Next.js AI chat application with multi-provider streaming, plus a terminal-first CLI coding agent that can inspect and execute shell commands like a lightweight Codex-style workflow.

✨ Features

• Multi-provider AI — OpenAI, Groq, Anthropic (auto-detected from .env)
• Advanced streaming engine — TokenBatcher reduces SSE frames by ~80%
• Circuit breaker — auto-skips broken providers with exponential back-off
• Live throughput metrics — tokens/s, TTFT, latency in real time
• 3D Typing Indicator — animated orbs + wave bar while AI thinks
• Dev Signature — spinning hex badge shows streaming/done state
• Abort support — stop generation mid-stream
• Provider selector — switch between Auto / OpenAI / Groq / Anthropic
• Code rendering — fenced code blocks with syntax highlighting
• CLI coding agent — interactive terminal assistant with command approvals or --yolo
• Works on Android Termux — full deployment guide included

🚀 Quick Start

1. Clone / extract project

unzip rafaygen.zip -d ~/rafaygen
cd ~/rafaygen

2. Add API keys

cp .env.example .env
nano .env   # add at least one key

OPENAI_API_KEY=sk-...
GROQ_API_KEY=gsk_...      # free & very fast
ANTHROPIC_API_KEY=sk-ant-...

3. Install & run

npm install
npm run dev      # development (hot reload)
# OR
npm run build && npm start   # production

Open http://localhost:3000

💻 CLI Coding Agent

The copied project now includes a stronger Codex-style terminal agent at cli/index.mjs, with built-in file/search/patch tools, direct shell shortcuts, approvals, persistent memory, local skills, themes, browser login, device-code auth, a first-run setup wizard, and a publishable rafaygen binary.

cd /data/data/com.termux/files/home/aiapp/rafaygen-cli
npm run cli

If no local provider keys or saved service auth are present, rafaygen now opens a first-run setup flow like Codex and offers login, signup, API-key, and base-URL options automatically.

Install-style usage:

npm install -g rafaygen
rafaygen --help

# or
npx rafaygen --help

Publish steps are documented in PUBLISHING.md.

Service auth flows:

rafaygen login --base-url http://127.0.0.1:4242
rafaygen signup --base-url http://127.0.0.1:4242
rafaygen whoami --base-url http://127.0.0.1:4242
rafaygen auth verify rgk_live_variant_xxx --base-url http://127.0.0.1:4242
rafaygen auth use-key rgk_live_variant_xxx --base-url http://127.0.0.1:4242

Single prompt:

rafaygen --approval auto "inspect this repo and explain the build setup"

Useful flags:

rafaygen --provider openai --mode code --max-steps 10
rafaygen --cwd /some/project --approval auto "fix the failing build"
rafaygen --model gpt-4o-mini "review this codebase"
rafaygen --base-url https://your-host.example login

Inside the interactive CLI:

• /help shows commands
• /tools shows the AI toolset
• /models shows local and remote model paths
• /memory shows the persistent memory log
• /skills lists installed skills
• /skill create <name> creates a local skill scaffold
• /skill install <path> imports a local skill
• /theme <name> switches CLI theme
• /setup opens the auth/setup wizard
• /auth, /login, /signup, /logout manage service auth
• / shows the command palette
• !npm test runs a shell command directly
• !! reruns the last shell command
• /provider auto|openai|groq|anthropic switches provider order
• /model <name|auto> sets a model override
• /cwd <path> changes the shell working directory
• /approval auto|manual toggles shell approvals
• /ls, /read, and /find are built-in quick shortcuts
• /clear resets the session
• /exit quits

📁 Project Structure

src/
├── app/
│   ├── api/chat/stream/route.ts   ← SSE API (multi-provider)
│   ├── globals.css                ← All styles + animations
│   ├── layout.tsx
│   └── page.tsx
├── components/
│   ├── Chat.tsx                   ← Main chat UI + SSE client
│   ├── ChatMessage.tsx            ← Message bubble + code renderer
│   ├── ChatInput.tsx              ← Textarea + provider selector
│   └── ai/
│       ├── TypingIndicator3D.tsx  ← Animated orbs while waiting
│       ├── DevSignature.tsx       ← Streaming/done badge
│       └── DevComponents.tsx      ← Barrel export
└── lib/
    └── server/
        └── streamEngine.ts        ← TokenBatcher, CircuitBreaker, etc.

cli/
├── index.mjs                      ← Terminal coding agent entrypoint
└── lib/
    ├── env.mjs                    ← Env loading + provider chain
    ├── providers.mjs              ← OpenAI / Groq / Anthropic calls
    ├── serviceAuth.mjs            ← Browser login, device code, API key auth
    ├── state.mjs                  ← Config, memory, themes, skills, auth home
    └── tools.mjs                  ← Built-in file/search/patch/shell tools

📱 Termux (Android) Deployment

See TERMUX_DEPLOY.sh for full instructions. Quick version:

# In Termux:
pkg install nodejs git unzip
cp /sdcard/Download/rafaygen.zip ~/
unzip ~/rafaygen.zip -d ~/rafaygen
cd ~/rafaygen
cp .env.example .env && nano .env   # add API key
npm install --legacy-peer-deps
npm run build
npm start

Open http://localhost:3000 in your Android browser.

🔌 API Reference

POST /api/chat/stream

{
  "messages": [
    { "role": "user", "content": "Hello!" }
  ],
  "provider": "groq",   // optional: auto | openai | groq | anthropic
  "model": "llama-3.3-70b-versatile"  // optional: override default
}

SSE Events: | Event | Payload | Description | |-------|---------|-------------| | meta | { provider, model, ts } | Stream started | | token | { text } | Token chunk | | progress | { tokensPerSecond, totalTokens, ttftMs, elapsedMs } | Throughput (every 2s) | | done | { result, latencyMs, ttftMs, totalTokens } | Completed | | error | { error, retryable } | Failed |

GET /api/chat/stream

Health check — returns provider availability.

🔧 Environment Variables

| Variable | Required | Description | |----------|----------|-------------| | OPENAI_API_KEY | One of these | OpenAI API key | | GROQ_API_KEY | One of these | Groq API key (free tier available) | | ANTHROPIC_API_KEY | One of these | Anthropic Claude key | | NEXT_PUBLIC_APP_VERSION | No | Version shown in UI (default: 2.0.0) |

🤖 Claude AI Limits Analysis

From the streaming engine, the most heavily used limits:

| Limit | Usage | Notes | |-------|-------|-------| | Max tokens/request | 4096 | Set in streamEngine.ts per call | | SSE frame rate | ~80% reduced | TokenBatcher 16ms window | | Provider rate limits | Managed | StreamCircuitBreaker with exp back-off | | Streaming throughput | Monitored | StreamHealthMonitor every 2s | | Concurrent requests | 1 per session | AbortController enforces this |