AI-Harness

A layered LLM harness for limitless memory, emotional depth & workflow automation

Table of Contents

• AI-Harness
  • About
    • Features
    • Built With
  • Setup
    • Model Recommendations
  • Usage
    • Configuration
    • Agents
    • Commands
    • MCP
    • Skills
    • Tools
  • Road Map
  • License

About

AI-Harness is a sophisticated harness designed to leverage Large Language Models (LLMs) for comprehensive knowledge management and workflow automation. This system combines cutting-edge features to create an efficient, adaptable platform that integrates seamlessly with various tools and technologies.

Features

• Layered Architecture: The best experience at the lowest cost by delegating work smartly
  • Efficient "Conscious" model handles normal requests, responses and orchestrates tasks
  • Lightweight + Fast "Sub-Conscious" model handles emotional intelligence, memories, and context summarization
  • Heavy Duty "Deep Thought" models are used for high-stakes tasks in one-off requests
• Advanced Memory Management: Document Knowledge + RAG + K-D Trees + Edge discovery = Limitless instant traversable knowledge
  • When context fills, "Sub-Conscious" summarizes oldest messages and frees context space creating endless windows
  • Then the "Sub-Conscious" model extracts facts and organizes knowledge into markdown files with wikilinks between them (Obsidian compatible)
  • RAG embeddings are created and cached in K-D trees to instantly feed relevant memories into the "Conscious" model ahead of answering
  • "Conscious" model uses WikiLinks to navigate edges and traverse related knowledge as needed
• Emotional Depth: Make your LLM unique
  • Create a custom avatar for your LLM that emotes with messages based on the "Sub-Conscious"
  • "Sub-Conscious" model sets tone of the "Conscious" dynamically based on personality
  • "Sub-Conscious" model performs self-reflection and modifies behavior to better suit the user
• Multi-Environment: For every environment, professional and personal
  • CLI: Talk to your LLM with one off questions or in an interactive terminal
  • WebUI: Jump into the browser at any time for a richer experience
  • PWA: Install the client on any device for quick access and remote work
• Virtu-World: Inspired by Mega Man Battle Network
  • Give your LLM an avatar and world to navigate and customize
  • Battle friends and bust viruses to keep your network safe
  • Let your LLM socialize and trade battle chips, agents, skills, tools, and commands
• All the harness goodness:
  • Agents: Define subagents your "Conscious" model can delegate work to using different models, tools, and instructions
  • Commands: Create /commands to cut out the repetitive prompting
  • Knowledge: Use offline zim archives (like Wikipedia) to backfill small models with endless knowledge
  • MCP: Connect your LLM to 3rd party services with MCP servers
  • Skills: Have the LLM create its own skill files or train it yourself on how to do certain things
  • Tools: Create custom JavaScript tools to allow your LLM to automate tasks
• Multi-Model Support: Use any combination
  • Local LLM (via OpenAI protocol)
  • Anthropic / Claude
  • OpenAI / ChatGPT

Built With

Setup

Prerequisites

• Node.js
• Optional: llama.cpp

Instructions

1. Set root: $AI_HOME="$HOME/AI"
2. Clone the project: npm --global install @ztimson/ai-harness
3. Open CLI or send message: ai [message]

Prerequisites

• Node.js
• Optional: llama.cpp

Instructions

1. Set root: $AI_HOME="$HOME/AI"
2. Clone the project: cd $AI_HOME && git clone https://git.zakscode.com/ztimson/ai-harness.git .
3. Install (automatically links to path): npm i
4. Start using the CLI/WebUI: ai --help

Don't just use all the models from one column, the goal is to combine models to maximize all VRAM. Stick to Q4 models (numbers below) — or Q3 if desperate.

| Role | Budget | Local Only | Perfect Balance | |------------------------------------|--------------------------------------------------------------------|-----------------------------|-------------------------------------| | Sub-Conscious / Memory / Quick Agent | Nemotron 3 Nano 4B (~4GB)Gemma 4 E4B (~6GB) | Gemma 4 E4B (~6GB) | Gemma 4 E4B (~6GB) | | Conscious / Primary / Orchestrator | Model above (no extra cost)ChatGPT ($2-5 / $12-30 1M Tokens) | Gemma 4 12B (~8GB) | Gemma 4 12B (~8GB) | | Deep Thought / Expert Agent | Claude ($2-10 / $12-50 1M Tokens) | Qwen 3.6 27b/32b (~18-30GB) | Claude ($2-10 / $12-50 1M Tokens) |

Recommended Context Settings:

• Max Length (local only): 100,000 Tokens (~200 letter pages)
• Compression Trigger: 75,000 Tokens (~150 letter pages)
• Compression Target: 50,000 Tokens (~100 letter pages)

Usage

Folder Structure

This harness is highly moddable via the $AI_HOME path (defaults to: $HOME/AI)

$AI_HOME
   ├── AGENTS.md (System prompt / Assistant's Rules)
   ├── agents
   │   └── researcher.md
   ├── bin (AI Harness Code)
   ├── commands (Markdown prompt templates)
   ├── data
   │   ├── settings.json (AI settings)
   │   ├── history.json (Chat history - delete to clear)
   │   ├── mcp.json (MCP definitions)
   │   └── schedule (Scheduled automated events)
   ├── models (Model images)
   ├── notes (User knowledge base (Obsidian Vault) and AI memory)
   │   ├── Personal (Information about user)
   │   │   └── Info.md
   │   ├── Journal
   │   │   └── 2026-01-01.md (Weekly notes)
   │   ├── Collection (Created as needed)
   │   │   └── Subject.md
   │   └── People
   ├── resources (zim archives for offline resources)
   │   └── archive_en_2026_01.zim
   ├── skills
   │   └── some-tool-api.md
   └── tools
       ├── fetch.js
       └── search.js

Config File

{
  "name": "AI Assistant",      // Your assistant's name
  "modelDir": "models",        // Override models path
  "port": 3000,                // Server port
  "conscious": "chatgpt-4o",   // Primary "Conscious" Model
  "primary": "Gemma4-E4B",     // Background "Sub-Conscious" Model
  "temperature": 1,            // Primary model temperature
  "tokens": {
    "min": 50000,              // Compress oldest messages until smaller than
    "compress": 80000,         // Trigger compressing history once exceeds
    "max": 100000              // Model max to stay under
  },
  "models": {
    "Gemma4-E4B": {
      "proto": "openai",
      "host": "http://127.0.0.1:1234/v1"
    },
    "chatgpt-4o": {
      "proto": "openai",
      "token": "..."
    },
    "claude-sonnet-5": {
      "proto": "anthropic",
      "token": "..."
    }
  }
}

---
name: Think Hard
description: Instructions on when to invoke subagent
model: claude-sonnet-5
temperature: 1
delegate: true // true = Agent responds directly, false = orchestrator responds on agent's behalf
mcp:           // null = All
skills: []     // empty array = None
tools:         // Fetch tool only
  - fetch
---

System prompt...

---
name: command
description: Help description
defaults:
  1: main
  2: no context provided
  USER: no prior user message
  AGENT: no prior agent message
  MESSAGE: no conversation yet
  ARGUMENTS: no arguments
---

First word: $1
Word 2 and on: $2+
Prior user message: $USER[1]
Last round of agent/tool work: $AGENT[1+]
Last 2 full turns: $MESSAGE[2+]
Everything typed: $ARGUMENTS

Variables:

• $1, $2, ... — single positional argument (whitespace-split)
• $2+ — argument 2 through the end of input, unsplit
• $ARGUMENTS — entire raw argument string (alias for $1+)
• $USER[N] — turn N's user message only
• $AGENT[N] — turn N's agent + tool output only (no user text)
• $MESSAGE[N] — turn N's full merged content (user + agent + tool)
• $USER[N+] / $AGENT[N+] / $MESSAGE[N+] — turns 1-N

[
  {"name": "Tools", "host": "https://...", "token": "secret"}
]

---
name: Example Skill
description: Instructions on when to read skill
---

How to behave...

(function(Config) {
    return {
        name: '...',
        description: 'When to call tool & how it works',
        args: {
            url: {type: 'string', description: 'URL to fetch', required: true},
            method: {
                type: 'string',
                description: 'HTTP method to use',
                enum: ['GET', 'POST', 'PUT', 'DELETE'],
                default: 'GET'
            },
            body: {type: 'object', description: 'HTTP body to send', default: {}},
            retry: {type: 'number', description: 'Retry count', default: 1, min: 1, max: 3},
            withCreds: {type: 'boolean', description: 'Send credentials', default: false}
        },
        fn: async ({url, method, body, retry, withCreds}, stream, ai) => {
            const {lib} = await import('...'); // Async Imports

            Config.meta(key); // Access config file properties
            Config.agents; // List of loaded agents: {name: '', description: '', content: ''}[]
            Config.mcp; // List of loaded MCPs: {name: '', host: '', token: ''}[]
            Config.skills; // List of skills: {name: '', description: '', content: ''}[]
            Config.tools; // List of tools: {name: '', description: '', args: {...}, fn: (args, stream, ai) => {...}}[]

            return 'success' || {error: 'failed'}; // Must be JSON serializable
        }
    }
})

Road Map

• Features
  • [ ] Collapsible side nav + ctrl click
  • [ ] Display server errors in client
  • [ ] Configure home page URL
  • [ ] Webhooks for triggering AI on event
  • [ ] File converter (images, audio, video, pdfs, etc...)
• New Tools
  • [ ] Notifications
  • [ ] Puppeteer/browser
  • [ ] Email (read inbox / send emails)
  • [ ] SMS/Phone
  • [ ] Map route
  • [ ] Image generator
  • [ ] Discord/Slack
  • [ ] Database
• NAVI/PET Integration
  • [ ] Voice cloning and TTS
  • [ ] Voice mode (ASR -> LLM -> TTS)
  • [ ] Avatar + animations + emoting
  • [ ] Game World
  • [ ] Mod support (standardize)

License

Copyright © 2023 Zakary Timson | Available under MIT Licensing

See the license for more information.