⬡ prompt-debug

🔬 Chrome DevTools, but for AI Prompts — Debug, Analyze & Compare LLM prompts in real-time

🤔 The Problem

Every AI developer faces this:

Write prompt → Run → Output is wrong
       ↓
"Why did this happen?" — No idea
       ↓
Guess → Change → Run again → Still wrong
       ↓
Wasted time + Wasted money + Frustration 😤

There are no proper debugging tools for LLM prompts.

• You don't know how many tokens your prompt uses
• You don't know how much it costs per request
• You can't compare two prompt versions side by side
• You can't track your experiments over a session

prompt-debug solves all of this.

✨ Features

| Feature | Description | |---|---| | ⚡ Real-time Streaming | See responses word-by-word, just like ChatGPT | | 🔤 Token Counter | Exact input + output token counts per request | | 💸 Cost Tracker | Cost in both ₹ INR and $ USD per request | | ⏱ Latency Tracker | Response time in milliseconds | | ⇌ Prompt Compare | A/B test two prompts side by side | | 🔥 Prompt Heatmap | Visualize which words have the most impact | | 📋 Session History | All your runs saved with full details | | 🌍 Multi-Provider | Groq, OpenAI, Together AI — switch instantly | | 🔌 WebSocket | Real-time connection, no page refresh needed | | 🔐 Secure | API keys stored locally in browser, never on any server |

🚀 Quick Start

# Install globally
npm install -g prompt-debug

# Start the debugger
prompt-debug

Browser opens automatically at http://localhost:3000 🎉

That's it. No config files. No setup. Just run and debug.

📦 Installation

Global (Recommended)

npm install -g prompt-debug

Local (Project-specific)

npm install prompt-debug
npx prompt-debug

Requirements

• Node.js >= 18.0.0
• A free API key from any supported provider

🎯 Usage

# Start on default port 3000
prompt-debug

# Start on custom port
prompt-debug --port 8080

# Start without opening browser
prompt-debug --no-open

# Show version
prompt-debug --version

# Show help
prompt-debug --help

Use as a Node.js Module

import { startServer } from "prompt-debug";

startServer({
  port: 3000,
  openBrowser: true,
});

🔑 Supported Providers

| Provider | Free Tier | Get API Key | |---|---|---| | ⚡ Groq | ✅ Yes — Very generous | console.groq.com/keys | | ◎ OpenAI | ❌ Paid | platform.openai.com/api-keys | | ⬡ Together AI | ✅ Yes — $25 free credits | api.together.ai |

💡 Recommended for beginners: Start with Groq — it's completely free and blazing fast!

🤖 Supported Models

Groq (Free ⚡)

• llama-3.3-70b-versatile — Best quality
• llama-3.1-8b-instant — Fastest
• gemma2-9b-it — Google's Gemma
• mixtral-8x7b-32768 — Great for code

OpenAI

• gpt-4o — Most capable
• gpt-4o-mini — Fast & cheap
• gpt-3.5-turbo — Classic
• o1, o1-mini — Reasoning models

Together AI (Free tier)

• meta-llama/Llama-3-70b-chat-hf
• mistralai/Mixtral-8x7B-Instruct-v0.1
• 100+ open-source models

🔄 Models are auto-fetched from your account when you enter your API key — you always get the latest available models!

💰 Cost Tracking

prompt-debug shows you the exact cost of every request:

Prompt: "Explain React hooks in simple terms"
Model:  llama-3.3-70b-versatile (Groq)

Input tokens:  12      → ₹0.0000
Output tokens: 284     → ₹0.0019
Total cost:    ₹0.0019 ($0.000023)
Latency:       342ms

Monthly estimate example:

10 users × 50 requests/day × 30 days = 15,000 requests
Average cost per request: ₹0.002
Monthly total: ₹30 (~$0.36)

Now you can budget your AI features properly! 💡

🏗️ Architecture

Browser (React UI)
      ↕  WebSocket (real-time)
Node.js Backend (Express)
      ↕  HTTPS
AI Provider (Groq / OpenAI / Together)

Why WebSocket?

| Regular HTTP | WebSocket | |---|---| | Wait for full response | Stream word by word | | One request, one response | Persistent connection | | Slow feel | Real-time feel ✅ |

📁 Project Structure

prompt-debug/
├── bin/
│   └── cli.js          ← CLI entry point (prompt-debug command)
├── index.js            ← Server + WebSocket + All providers
├── package.json
└── README.md

🛠️ Development

# Clone the repo
git clone https://github.com/yourusername/prompt-debug
cd prompt-debug

# Install dependencies
npm install

# Start in dev mode (auto-restart on changes)
npm run dev

# Build frontend
cd frontend && npm run build

🤝 Contributing

Contributions are welcome! Here's how:

1. Fork the repository
2. Create a feature branch — git checkout -b feature/amazing-feature
3. Commit your changes — git commit -m "Add amazing feature"
4. Push to the branch — git push origin feature/amazing-feature
5. Open a Pull Request

Ideas for Contributions

• [ ] Anthropic Claude support
• [ ] Cohere support
• [ ] Export history as CSV/JSON
• [ ] Real token heatmap using logprobs
• [ ] Dark/Light theme toggle
• [ ] Prompt templates library
• [ ] Monthly cost dashboard with charts
• [ ] Docker support

🐛 Bug Reports

Found a bug? Open an issue with:

• Your Node.js version (node --version)
• Your OS (Windows/Mac/Linux)
• Steps to reproduce
• Expected vs actual behavior

📊 Comparison with Other Tools

| Feature | prompt-debug | LangSmith | OpenAI Playground | Postman | |---|---|---|---|---| | Free | ✅ 100% Free | ❌ Paid | ❌ Limited | ❌ Limited | | Open Source | ✅ | ❌ | ❌ | ❌ | | Groq Support | ✅ | ❌ | ❌ | ✅ | | INR Cost | ✅ | ❌ | ❌ | ❌ | | Streaming | ✅ | ✅ | ✅ | ❌ | | Prompt Compare | ✅ | ✅ | ❌ | ❌ | | Local Install | ✅ | ❌ | ❌ | ✅ | | Setup Time | 30 sec | 10 min | Instant | 5 min |

📜 Changelog

v1.0.0 (2024)

• 🎉 Initial release
• ✅ Groq, OpenAI, Together AI support
• ✅ Real-time WebSocket streaming
• ✅ Token counter + Cost tracker (₹ + $)
• ✅ Prompt A/B comparison
• ✅ Session history
• ✅ Prompt heatmap

📄 License

MIT © Ankur Ojha

Free to use, modify, and distribute. See LICENSE for details.

⭐ Support

If this project helped you, please consider:

• ⭐ Starring the repo on GitHub
• 🐦 Sharing on Twitter/LinkedIn
• 🐛 Reporting bugs and issues
• 🤝 Contributing new features

👨‍💻 Author

Built with ❤️ by Ankur Ojha

"Built an open-source Prompt Debugger for LLMs with real-time token analysis, INR cost tracking, WebSocket streaming, and multi-provider support (Groq/OpenAI/Together AI)"