Max/MSP AI Agent Harness & MCP Bridge 🎛️🤖

A Model Context Protocol (MCP) bridge that turns external AI agents into a dynamic canvas control harness for generating, modifying, and interacting with local Max/MSP and Max for Live patches via thispatcher and give them super powers as Max dev assitants.

This project establishes a direct pipeline to link LLMs straight to your Max canvas, featuring a Human-in-the-Loop staging environment so you can visually review layout commands before executing them.

This project is very much a work in progress

🎯 Scope & Philosophy

What This Is:

• A makeshift agent harness for Max MSP and Max for Live. It is a structured control pipeline designed for developers, patchers, and sound designers who want a precise, machine-interpretable framework to manipulate the Max layout workspace canvas programmatically and actually useful max dev Ai Assistants

What This Isn't:

• Vibe coding for Max. This is not a "magic button" that writes complete DSP patches based on abstract human feelings. It requires strict context formatting, explicit coordinate layouts, and architectural guidance. You are the architect; the agent is your automated layout technician.

What To Expect

I'm not sure if there will ever be a full vibe coding experience for Max. The main issue is that Max is visual and Ai just can't see the patch. Copy and pasting patches into LLM's does work but the JSON that is generated is thousands of lines for even small patches and wastes tokens and clouds context. Agents built from this will work best as conversation first development. If you need ideas, suggestions, or are fixing bugs and just need the agent to provide guidance and basic patch building, this is for you. Do not expect complex patches with sub patchers from these agents but, jumping off points, templates and guidance they excel at. Anything code related they can do well obv. If you need Javascript or even coeebox code they do really well.

🏗️ Architecture Layout

This project operates as a unified monorepo, keeping your execution layers, layout assets, and AI logic synchronized:

mcp-max-harness/
├── package.json            # NPM publication settings (Tracks the bridge script)
├── max-bridge.mjs          # The host-level native MCP Bridge
├── README.md               # Setup and usage guide
│
├── max-project/              # The Max/MSP runtime environment
│   ├── agent-harness.maxpat # Drop-in UI abstraction object
│   └── max-bridge.js  # Script run inside Max's [node.script] object
    └── agent-harness/      # This is the max project which contains examples for implementing this indside of max.
│
└── ai-assets/              # Agent behavior definitions
    ├── system-prompt.txt   # Instructions defining the LLM's persona
    └── knowledge-base/     # Reference documentation for layout injection
        ├── _index.md.      # an index of the knowledge base 
        └── ABLETON         # all ableton related objects including abl.dsp and abl.device
        └── GEN             # All Gen related objects
        └── GUIDE           # Tips, tricks and standards for Max for live dev
        └── JIT             # All Jit related objects
        └── MAX             # All Max objects
        └── MC              # All MC objects
        └── MSP             # All MSP Objects

🔌 Cross-Agent Integration Guide

Because this tool is built entirely on the open Model Context Protocol (MCP) standard, you are not locked into a single application. Below is how to hook up your Max harness across different AI engines:

1. Osaurus Configuration

1. Open Osaurus Settings and navigate to Remote MCP Providers / Forms.
2. Create a new custom provider configuration with these parameters:
   • Name: max-harness-bridge
   • Transport Mechanism: stdio
   • Execution Scope: Host (Critical: Bypasses the container sandbox)
   • Command: node
   • Arguments: /absolute/path/to/your/repo/max-bridge.mjs
   • Auto-Connect: Enabled
3. Save the provider configuration. Click Test / Probe to verify.

2. Claude Desktop App Configuration

To integrate this harness directly into the official Anthropic Claude Desktop interface:

1. Open your Mac Terminal and open the Claude configuration file: nano ~/Library/Application\ Support/Claude/claude_desktop_config.json
2. Add your Max bridge under the mcpServers block:

   {
     "mcpServers": {
       "max-msp-harness": {
         "command": "node",
         "args": ["/absolute/path/to/your/repo/max-bridge.mjs"]
       }
     }
   }

3. Restart Claude Desktop. A small plug/hammer icon will appear in the input chat box indicating Claude can now see your patch.

3. Open WebUI (or LibreChat / AnythingLLM)

If you are running a local chat system over Docker or an internal web runner:

1. Navigate to your admin settings console panel under Admin Settings > Connections > MCP Servers.
2. Add a new standard stdio hook connection entry.
3. If Open WebUI runs in a Docker sandbox, ensure you call Node on the host or bind the directory mount paths properly so the subprocess can open port 3500 over your system's network loopback layer.

⚡ Abstraction Preparation (Inside Max)

1. Open the /max-patch/ directory.
2. Open agent-harness.maxpat or drag it as an abstraction object directly onto your target project canvas.
3. Click the "1. Install Dependencies" action button in the Presentation interface. This securely maps your local environment and fires script npm install asynchronously through [node.script] without blocking or freezing the real-time audio threads.
4. Click "2. Start Server" (script start). The console should print: listening on 127.0.0.1:3500.

🛡️ The "Human-In-The-Loop" Safety Staging Gate

To prevent an LLM from accidentally miscalculating coordinate placement or executing invalid structural patterns that could crash the Max framework runtime environment, this tool utilizes a safe sandbox staging gate:

1. The AI calculates patch layout changes and pipes them to the server via the custom MCP tool.
2. The incoming message parsing loop maps the parameters into a Message Box object inside the abstraction layout instead of executing them instantly.
3. You retain total control: Review the generated layout instruction lines inside the log window. When you are satisfied, click the manual Bang button to dump the code payload directly into thispatcher.

🧠 Optimizing the AI Knowledge Base

Standard LLMs are frequently untrained on the specific nuances of Max MSP and Max For Live. To ensure reliable performance, attach the contents of the /ai-assets/ folder to your workspace:

1. System Prompt (system-prompt.txt): Paste this into your Agent profile configuration. It instructs the LLM how to format single-action lists without invalid prefix tags like script script.
2. Knowledge Base (/knowledge-base/): Upload these markdown max refference and syntax directly to your AI Workspace file attachment layer (or Claude Project Knowledge Base). This gives the transformer a rich context database indexing valid structural connection paths, argument spacing syntax, and positional object coordinates.