GUI Chat Protocol

TypeScript type definitions for the GUI Chat Protocol plus lightweight Vue and React adapters. These primitives let you build plugins that expose GUI-capable tool calls to any chat application.

Need the narrative overview, motivation, and roadmap? Read the GUI_CHAT_PROTOCOL.md spec.

Documentation Map

• README.md (this file): package usage, exports, and integration guidance
• spec/API_REFERENCE.md: canonical type signatures for core, Vue, and React APIs
• spec/CREATING_A_PLUGIN.md: step-by-step plugin authoring guide
• spec/GUI_CHAT_PROTOCOL.md: high-level protocol design, motivation, and long-form examples
• AGENTS.md: contributor workflow, scripts, and review requirements

Repository Layout

| Path | Purpose | |------|---------| | src/types.ts, src/schema.ts, src/inputHandlers.ts | Core framework-agnostic contracts | | src/vue.ts, src/react.ts | Adapter typings for Vue 3 and React 18/19 | | spec/ | Protocol specification (GUI_CHAT_PROTOCOL.md), API reference, and plugin guides | | dist/ | Build artifacts generated by yarn build | | tsconfig.json, vite.config.ts | Build + declaration settings |

Pair this README with the spec docs referenced above: use spec/GUI_CHAT_PROTOCOL.md for motivation, spec/API_REFERENCE.md for exact signatures, and spec/CREATING_A_PLUGIN.md for the scaffold walkthrough.

This Package

This npm package (gui-chat-protocol) provides the TypeScript type definitions for implementing the GUI Chat Protocol:

• Framework-agnostic core types - For plugin logic without UI dependencies
• Vue adapter - Vue 3 component types
• React adapter - React 18/19 component types

By defining a standard protocol, plugins become portable across different chat applications, and applications can easily integrate plugins without framework-specific dependencies.

Architecture

Tool Plugin Architecture

┌─────────────────────────────────────────────────────────────┐
│                     Chat Application                         │
│  ┌─────────────┐  ┌─────────────┐  ┌─────────────────────┐  │
│  │    LLM      │  │   Plugin    │  │      UI Layer       │  │
│  │  Interface  │◄─┤   Manager   │◄─┤  (Vue/React/etc.)   │  │
│  └─────────────┘  └──────┬──────┘  └─────────────────────┘  │
└──────────────────────────┼──────────────────────────────────┘
                           │
         ┌─────────────────┼─────────────────┐
         │                 │                 │
         ▼                 ▼                 ▼
   ┌───────────┐    ┌───────────┐    ┌───────────┐
   │  Plugin A │    │  Plugin B │    │  Plugin C │
   │  (Quiz)   │    │  (Image)  │    │  (Form)   │
   └───────────┘    └───────────┘    └───────────┘

Key Types

| Type | Description | |------|-------------| | ToolPluginCore | Framework-agnostic plugin definition | | ToolPlugin | Vue-specific plugin with Vue components | | ToolPluginReact | React-specific plugin with React components | | ToolResult | Standardized result from plugin execution | | ToolContext | Context passed to plugin execute function | | ToolDefinition | OpenAI-compatible function definition |

Installation

npm install gui-chat-protocol
# or
yarn add gui-chat-protocol

Package Exports

// Core types (framework-agnostic)
import { ToolPluginCore, ToolResult, ToolContext } from "gui-chat-protocol";

// Vue-specific types
import { ToolPlugin } from "gui-chat-protocol/vue";

// React-specific types
import { ToolPluginReact } from "gui-chat-protocol/react";

API Reference

Detailed type signatures now live in spec/API_REFERENCE.md. Keep this README focused on architecture, tooling, and integration guidance.

Creating a Plugin

A complete walkthrough (types → core plugin → Vue/React wrappers) now lives in spec/CREATING_A_PLUGIN.md.

Example Implementations

Quiz Plugin

An interactive quiz plugin that presents multiple-choice questions:

• Repository: MulmoChatPluginQuiz
• Features:
  • Multiple questions with choices
  • Answer tracking with viewState persistence
  • Both Vue and React implementations

// Usage
import QuizPlugin from "@mulmochat-plugin/quiz/vue";
// or
import QuizPlugin from "@mulmochat-plugin/quiz/react";

GenerateImage Plugin

An image generation plugin that creates images from text prompts:

• Repository: MulmoChatPluginGenerateImage
• Features:
  • Text-to-image generation
  • File upload and clipboard paste support
  • Backend abstraction for different providers

// Usage
import GenerateImagePlugin from "@mulmochat-plugin/generate-image/vue";

Compatible Applications

MulmoChat

A multi-modal voice chat application with comprehensive plugin support:

• Repository: MulmoChat
• Features:
  • OpenAI Realtime API integration
  • Voice and text input
  • Multiple backend providers
  • Full plugin ecosystem

Building Your Own Application

Any application can implement the GUI Chat Protocol by:

1. Loading plugins that export { plugin } default export
2. Passing tool definitions to the LLM
3. Executing plugins via plugin.execute(context, args)
4. Rendering plugin components (viewComponent/previewComponent)
5. Handling ToolResult for UI updates and LLM responses

// Example: Loading a plugin
import Plugin from "@mulmochat-plugin/quiz/vue";

// Get tool definition for LLM
const tools = [Plugin.plugin.toolDefinition];

// Execute when LLM calls the tool
const result = await Plugin.plugin.execute(context, args);

// Render the view component
<component :is="Plugin.plugin.viewComponent" :selected-result="result" />

Framework Support

| Framework | Import Path | Plugin Type | |-----------|-------------|-------------| | Core (no UI) | gui-chat-protocol | ToolPluginCore | | Vue 3 | gui-chat-protocol/vue | ToolPlugin | | React 18/19 | gui-chat-protocol/react | ToolPluginReact |

Both Vue and React are optional peer dependencies - only install what you need.

License

MIT