Pi Provider - MLX Local

Note: This is an unofficial, community-built provider. It runs local AI models on Apple Silicon using Apple's MLX framework.

A Pi coding agent provider extension for local AI inference using Apple's MLX framework.

Features

• Local AI Inference: Run AI models entirely on your Mac (Apple Silicon)
• No API Costs: Free inference using your local GPU/memory
• Privacy-First: Your data never leaves your machine
• Apple Silicon Optimized: Leverages MLX for efficient inference on M1/M2/M3/M4 chips
• Vision Support: Supports vision-language models for image understanding
• OpenAI-Compatible: Uses MLX-VLM server with OpenAI-compatible API

Requirements

• macOS: Monterey 12.3 or later
• Apple Silicon: M1, M2, M3, or M4 chip
• Python: 3.10+ with pip
• MLX-VLM: Installed automatically via /mlx_init

Installation

Option 1: Using pi install (npm)

pi install npm:@sinamtz/pi-mlx-provider

Option 2: Using pi install (git)

pi install git:https://github.com/sinamtz/pi-mlx-provider

Option 3: Install globally via npm

npm install -g @sinamtz/pi-mlx-provider

Option 4: Clone locally

git clone https://github.com/sinamtz/pi-mlx-provider.git
cd pi-mlx-provider
npm install

Configuration

Enable Local AI

1. Edit ~/.pi/settings.json to add the MLX provider settings:

{
  "mlx-provider": {
    "local_intelligence": true,
    "local_model": true,
    "mlx_model": "mlx-community/gemma-4-e2b-it-4bit",
    "mlx_port": 8080,
    "mlx_host": "127.0.0.1"
  }
}

| Setting | Type | Default | Description | |---------|------|---------|-------------| | local_intelligence | boolean | false | Enable local AI processing | | local_model | boolean | false | Use local model (requires local_intelligence=true) | | mlx_model | string | mlx-community/gemma-4-e2b-it-4bit | MLX model to use | | mlx_port | number | 8080 | Server port | | mlx_host | string | 127.0.0.1 | Server host |

Note: Both local_intelligence AND local_model must be true to enable local mode.

Auto-Start Behavior

The extension automatically starts the MLX server on load when:

1. Local mode is enabled in settings (local_intelligence and local_model are both true)
2. A state file exists at ~/.pi/agent/extensions/mlx-settings.json (created by /mlx_init)

This means after running /mlx_init once, the server will automatically start on future loads without needing to re-run /mlx_init.

Usage

Initialize MLX Server (First Time)

pi -e ./pi-mlx-provider
/mlx_init

This will:

1. Install/upgrade mlx-vlm package
2. Download the MLX model (first time only)
3. Start the MLX-VLM server
4. Save state to ~/.pi/agent/extensions/mlx-settings.json
5. Register the mlx-local provider

Auto-Start on Future Loads

Once initialized, the server will automatically start on future loads when local mode is enabled. No need to run /mlx_init each time.

Check Server Status

/mlx_status

Stop Server

/mlx_stop

This stops the server and removes the state file, preventing auto-start on future loads.

Select Model

In pi's interactive mode:

/model mlx-local

Or use the model selector UI.

Supported Models

| Model Type | Example | Input | |------------|---------|-------| | Text-only LLM | mlx-community/Llama-3.2-3B-Instruct-4bit | text | | Vision-Language | mlx-community/Qwen2-VL-2B-Instruct-4bit | text, image | | Gemma | mlx-community/gemma-4-e2b-it-4bit | text |

Recommended Models

• gemma-4-e2b-it-4bit: Good balance of quality and speed (default)
• Llama-3.2-3B-Instruct-4bit: Fast, good for simple tasks
• Qwen2-VL-2B-Instruct-4bit: Vision support for image understanding

Troubleshooting

"MLX server is not running"

• Run /mlx_init to start the server

"Failed to install mlx-vlm"

• Ensure Python 3.10+ is installed
• Try: pip3 install --upgrade mlx-vlm

"Model download failed"

• Check internet connection
• Ensure sufficient disk space for model files

Port already in use

• Change mlx_port in settings.json to a different port (e.g., 8081)

Out of memory

• Use a smaller/quantized model (4bit variants)
• Close other memory-intensive applications

How It Works

pi agent
    └── pi-mlx-provider extension
            ├── Reads settings from ~/.pi/settings.json
            ├── If local_intelligence + local_model enabled AND state file exists:
            │   ├── Auto-starts mlx-vlm server (or reconnects if already running)
            │   ├── Registers "mlx-local" provider
            │   └── Routes requests to localhost:PORT
            ├── If enabled but no state file:
            │   └── Waits for /mlx_init command
            └── If disabled: uses default cloud provider

State Persistence

The extension maintains state in ~/.pi/agent/extensions/mlx-settings.json:

• Created by /mlx_init after successful server start
• Read on extension load to determine if auto-start should occur
• Removed by /mlx_stop to prevent auto-start on future loads

License

MIT

Contributing

Contributions welcome! Please open an issue or PR on GitHub.