friday-runtime

Self-contained agent runtime for Friday AI. Orchestrates Claude conversations, manages MCP servers, handles permissions, skills, and multi-modal AI providers.

This package can be published and consumed independently — no dependency on the Electron app.

Directory Structure

packages/runtime/
├── index.js                      # Public API exports
├── friday-server.js              # Stdio entry point
├── server.js                     # HTTP/WebSocket server
├── .mcp.json                     # MCP server definitions & auth schemas
├── package.json
│
├── src/
│   ├── runtime/
│   │   ├── AgentRuntime.js       # Core agent orchestrator
│   │   └── RoleBasedAgentRuntime.js
│   ├── mcp/
│   │   └── McpCredentials.js     # Secure credential storage (keytar/file fallback)
│   ├── config.js                 # Config loader, template variable substitution
│   ├── skills/
│   │   ├── SkillManager.js       # Two-tier skill loading system
│   │   └── global/               # 27+ expertise markdown files
│   ├── agents/                   # Agent routing & configuration
│   ├── scheduled-agents/         # Background automation (cron-based)
│   ├── sessions/                 # Session persistence & history
│   ├── oauth/                    # OAuth flow handlers
│   └── sandbox/                  # Process sandboxing
│
├── providers/                    # Multi-modal AI providers
│   ├── ProviderRegistry.js       # Central provider management & MediaContext
│   ├── openai.js                 # OpenAI: GPT-5.2, gpt-image-1.5, Sora 2, TTS/STT
│   ├── google.js                 # Google: Gemini 3, Imagen 4, Veo 3.1, Cloud TTS/STT
│   └── elevenlabs.js             # ElevenLabs: Eleven v3, Flash v2.5, Turbo v2.5
│
├── mcp-servers/
│   ├── media-server.js           # MCP server: image/video/audio generation tools
│   ├── terminal-server.js        # MCP server: shell command execution
│   └── resend/                   # MCP server: email via Resend
│
├── config/
│   └── GlobalConfig.js           # Persistent user configuration
│
├── rules/
│   └── rules.json                # Agent behavior rules
│
└── docs/
    └── multi-modal-providers-plan.md

Quick Start

import { AgentRuntime, loadBackendConfig } from 'friday-runtime';

const config = await loadBackendConfig();
const runtime = new AgentRuntime({
  workspacePath: config.workspacePath,
  rules: config.rules,
  mcpServers: config.mcpServers,
  sessionsPath: config.sessionsPath,
});

runtime.on('message', (msg) => console.log(msg));
await runtime.handleQuery('Hello, Friday');

Agent Loop

User Query → AgentRuntime.handleQuery()
  ├── 1. Build system prompt (with skills)
  ├── 2. Prepare MCP servers from .mcp.json
  ├── 3. Call Claude SDK query() with:
  │       ├── model, mcpServers, systemPrompt, canUseTool
  ├── 4. SDK spawns MCP servers, discovers tools via listTools()
  ├── 5. Claude decides which tools to use
  ├── 6. Permission gate checks each tool call
  ├── 7. Tool executes → result back to Claude
  └── 8. Stream response chunks to consumer

MCP Servers

| Server | Purpose | |--------|---------| | filesystem | File read/write via @modelcontextprotocol/server-filesystem | | terminal | Shell command execution (custom server) | | github | GitHub API via @modelcontextprotocol/server-github | | friday-media | Image gen, video gen, TTS, STT, multi-model queries | | firecrawl | Web scraping | | figma | Design file access | | resend | Email sending | | discord/reddit/twitter | Social platform APIs | | gmail/google-drive | Google Workspace | | supabase | Database operations |

Multi-Modal Providers

The provider system enables image generation, video generation, TTS, STT, and multi-model chat:

| Capability | OpenAI | Google | ElevenLabs | |------------|--------|--------|------------| | Image gen | gpt-image-1.5 | Imagen 4 Ultra/Standard/Fast | — | | Video gen | Sora 2 / Sora 2 Pro | Veo 3.1 / Veo 3.1 Fast | — | | TTS | gpt-4o-mini-tts | Google Cloud TTS | Eleven v3, Flash v2.5 | | STT | Whisper | Google Cloud STT | — | | Chat | GPT-5.2 | Gemini 3 Pro/Flash | — |

Auto-selects best available provider based on API key availability and user preferences.

Skills System

Two-tier expertise injection into system prompts:

• Tier 1 — Expert Skills (max 2): User-selected via @mention tags
• Tier 2 — Internal Skills (max 2): Agent-selected via [REQUEST_SKILLS: ...]

Dependencies

| Package | Purpose | |---------|---------| | @anthropic-ai/claude-agent-sdk | Core agent SDK | | @anthropic-ai/sdk | Anthropic API client | | @modelcontextprotocol/sdk | MCP protocol | | openai | OpenAI API (image, video, TTS, STT, chat) | | @google/genai | Google Gemini, Imagen, Veo API | | @elevenlabs/elevenlabs-js | ElevenLabs TTS API | | ws | WebSocket support | | pino | Structured logging | | zod | Schema validation |

Detailed Plans

• Multi-Modal Providers Plan — Full architecture for image/video/audio capabilities