SOGA - Smart Orchestration Glasses Agent

Voice-controlled AI coding assistant for AR smart glasses

Website • Documentation • Discord

🏆 MongoDB.local 2026 Hackathon - Top 6 Finalist

SOGA was built for the MongoDB.local 2026 hackathon in San Francisco and selected as a Top 6 Finalist. It demonstrates the power of combining:

• MongoDB for real-time conversation persistence and settings storage
• Voice AI for hands-free coding interactions
• AR Smart Glasses for ambient computing
• Multi-agent orchestration for parallel task execution

What is SOGA?

SOGA stands for Smart Orchestration Glasses Agent.

SOGA is a Mentra MiniApp built on top of MentraOS. It consists of a desktop daemon (CLI) that runs on your machine, connecting to the SOGA MiniApp on your smart glasses. Speak a command through your glasses, and AI agents execute coding tasks on your local machine in real-time.

The Vision

You: "Hey SOGA, create 5 personalized landing pages for our VIP judges"

→ 5 AI agents spawn in parallel
→ Each creates a project and starts a dev server
→ You see live URLs appear in your glasses
→ Hot reload as agents work
→ SOGA proactively messages: "🟢 All 5 sites are live!"

Installation

Using Bun (Required)

bunx @mentra/soga

Note: Bun is required because SOGA uses Bun-specific APIs for process spawning. Install Bun at bun.sh if you haven't already.

Global Installation

bun add -g @mentra/soga
soga

Quick Start

1. Start the Daemon

soga

On first run, you'll be guided through setup:

• Enter your email (your Mentra account email)
• Configure server URL (defaults to SOGA cloud)
• Set your workspace directory

2. Connect Your Glasses

Open the MentraOS app on your smart glasses and launch the SOGA MiniApp. Sign in with the same Mentra account email.

3. Start Coding with Voice

"Hey SOGA, add a dark mode toggle to the settings page"
"Hey SOGA, refactor the auth module to use JWT"
"Hey SOGA, create a new React component for user profiles"

Commands

soga                        # Start daemon (interactive setup if needed)
soga status                 # Check connection status
soga config                 # Show current configuration
soga config reset           # Reset configuration
soga run "<goal>"           # Run a one-off agent task
soga help                   # Show help

# Options
soga --server <url>         # Connect to custom server
soga --email <email>        # Set user email
soga --provider <name>      # CLI provider: opencode (default) or claude
soga --model <model>        # AI model to use

Configuration

Environment Variables

# Server connection
SOGA_SERVER_URL=https://your-server.com

# AI Provider (optional - interactive setup available)
GEMINI_API_KEY=your-gemini-key        # For OpenCode with Gemini
ANTHROPIC_API_KEY=your-anthropic-key  # For Claude CLI

Config File

Configuration is stored in ~/.soga/config.json:

{
  "email": "[email protected]",
  "serverUrl": "https://api.soga.mentra.glass",
  "daemonName": "My MacBook",
  "primaryWorkspace": "/Users/you/Projects"
}

Note: The email should match your Mentra account email to link the daemon to your glasses.

CLI Providers

SOGA supports multiple AI coding CLI tools:

OpenCode with Gemini (Default)

soga --provider opencode --model google/gemini-3-flash-preview

Fast, cost-effective, great for most tasks.

Claude CLI

soga --provider claude

Requires claude CLI installed and ANTHROPIC_API_KEY set.

How It Works

┌─────────────────┐     ┌─────────────────┐     ┌───────────────────────────────┐
│   AR Glasses    │────▶│  MentraOS App   │────▶│        SOGA Daemon            │
│  (Voice Input)  │     │  (SOGA MiniApp) │     │       (Your Machine)          │
└─────────────────┘     └─────────────────┘     │                               │
                                                │  ┌─────────────────────────┐  │
                                                │  │      AI Agents          │  │
                                                │  │  (OpenCode / Claude)    │  │
                                                │  └─────────────────────────┘  │
                                                └───────────────────────────────┘

1. Voice Command - Speak to your AR glasses running the MentraOS app
2. SOGA MiniApp - Receives transcription and sends task to your daemon
3. Agent Execution - Daemon spawns AI coding agents (OpenCode/Claude) on your machine
4. Real-time Updates - Progress streams back to glasses and web UI
5. Proactive Messaging - Agent notifies you when tasks complete

Features

🎙️ Voice-First Interface

Speak naturally to code. Wake words, end phrases, and continuous listening.

🤖 Multi-Agent Orchestration

Spawn multiple AI agents in parallel for complex tasks.

📊 Real-Time Progress

See what agents are doing: reading files, writing code, running commands.

🔄 Proactive Messaging

Agents message YOU when things happen - completions, errors, questions.

🌐 Web Dashboard

Full web UI for chat history, settings, and detailed agent logs.

⚙️ Customizable Settings

• Custom wake words (for multi-user demos)
• End phrases ("over", "send it") for instant submission
• Always-speak mode for audio feedback on any glasses

Requirements

• Runtime: Bun 1.0+ (required - install here)
• OS: macOS, Linux, or Windows (WSL)
• AI CLI: OpenCode (included) or Claude CLI (optional)
• API Key: Gemini API key (free tier available) or Anthropic API key

Development

Running Locally

# Clone the repo
git clone https://github.com/mentra-app/soga.git
cd soga/daemon

# Install dependencies
bun install

# Run in dev mode (hot reload)
bun run dev

# Or run directly
bun run start

Project Structure

daemon/
├── src/
│   ├── cli.ts              # CLI entry point
│   ├── index.ts            # Main daemon logic
│   ├── daemon-client.ts    # WebSocket client
│   ├── agent-pool.ts       # Agent management
│   ├── terminal-agent.ts   # AI CLI wrapper
│   ├── cli-provider.ts     # Provider abstraction
│   ├── observer.ts         # LLM observer for status
│   ├── config.ts           # Configuration management
│   ├── secrets.ts          # API key management
│   └── types.ts            # TypeScript types
├── package.json
├── tsconfig.json
└── README.md

Troubleshooting

"Connection refused" error

Make sure the SOGA server is running and accessible:

soga status

"No API key configured"

Set up your AI provider:

# Interactive setup
soga

# Or set environment variable
export GEMINI_API_KEY=your-key

Agent not responding

Check that the workspace directory exists and is accessible:

soga config

Contributing

We welcome contributions! Please see our Contributing Guide for details.

License

MIT License - see LICENSE for details.

Links

• 🌐 Mentra Website
• 📖 Documentation
• 💬 Discord Community
• 🐙 GitHub
• 🐦 Twitter

Built with ❤️ by Mentra

Making the future of computing wearable