TMLPD — Parallel Multi-LLM Execution Module

Part of the A3M Router ecosystem.

Parallel multi-LLM execution with confidence-weighted ensemble merging. Runs providers simultaneously, scores each result, and returns the best answer with transparent reasoning.

What's New in v1.3.0

Token Optimization — 6 patterns from arXiv:2608.17188 for 40-60% token reduction:

| Pattern | Description | |:--------|:------------| | Semantic Cache | Embedding-based similarity caching (cosine > 0.85 threshold) | | Context Stratification |分层 context levels (LOW: 512 tokens, MEDIUM: 2048, HIGH: 8192) | | Token-Aware Fallback | Route to cheap/medium/expensive models by token count | | Schema Contraction | Inject schema reference vs full description | | Fetch-Once/Process-Local | One expensive fetch, extract with cheap model | | Inter-Agent Compression | Compress messages between agents |

Core Features

| Feature | Description | |:--------|:------------| | Parallel execution | Run N providers simultaneously, not sequentially | | Ensemble scoring | Score results on specificity, structure, and relevance | | Token Optimization | 6 patterns for 40-60% token reduction | | Query-type presets | Auto-configure provider + temp per task type | | Cost tracking | Per-query cost display with provider breakdown | | Persistent memory | Cross-session .memory.json with keyword indexing | | Prefix caching | RadixAttention-style caching for repeated prefixes | | Speculative decoding | Medusa/EAGLE-style multi-token prediction | | Token compression | ISON encoding for ~40% token reduction |

Usage

import { executeEnsemble, createPresetRouter, TokenOptimizer } from "tmlpd-pi";

// Parallel ensemble: run all providers simultaneously, pick best
const result = await executeEnsemble(
  "Explain vector databases",
  systemPrompt,
  context,
  { nvidia: callNvidia, groq: callGroq }
);
console.log(`Winner: ${result.winner} (score: ${result.scores[result.winner]})`);

// Token Optimization - 40-60% token reduction
const optimizer = new TokenOptimizer();
const optimized = await optimizer.optimizeQuery(
  "Explain quantum computing",
  history
);
console.log(`Context level: ${optimized.contextLevel}`);
console.log(`Recommended model: ${optimized.recommendedModel}`);
console.log(`Cache hit: ${optimized.cacheHit}`);

// Query-type presets: auto-configure per task
const router = createPresetRouter();
const preset = router.classify("Write a Python sort function"); // → 'code'

Token Optimization Patterns

import { SemanticCache, ContextStratifier, TokenAwareFallback } from "tmlpd-pi";

// Semantic caching - cache by embedding similarity
const cache = new SemanticCache({ similarityThreshold: 0.85 });
const cached = await cache.get("Explain quantum entanglement");
if (cached) console.log(`Cache hit! Similarity: ${cached.similarity}`);

// Context stratification - match context depth to query complexity
const stratifier = new ContextStratifier();
const result = stratifier.classify("What is 2+2?", history);
// → LOW context (512 tokens, no history)

// Token-aware fallback - route by token count
const fallback = new TokenAwareFallback();
const decision = fallback.selectModel(estimatedTokens);
// → cheap model for <500 tokens

Exports

• createTMLPD, TMLPDTools — Core parallel execution
• executeEnsemble, mergeComplementary, recordFeedback — P0 Ensemble voting
• createPresetRouter, getPresetForQuery, DEFAULT_PRESETS — P1 Query presets
• TokenOptimizer, SemanticCache, ContextStratifier, TokenAwareFallback — NEW Token Optimization
• EpisodicMemoryStore — P3 Persistent memory with auto-save
• CostTracker, BudgetEnforcer — P2 Cost tracking
• ResponseCache, PrefixCache — Caching layers
• HALOOrchestrator, MCTSWorkflowOptimizer — Advanced orchestration

Research Backing

• Token Optimization (arXiv:2608.17188) — NEW 6 patterns for 40-60% token reduction
• RouteLLM (arXiv:2404.06035) — Learned cost-quality routing
• RadixAttention (arXiv:2312.07104) — 5-10x speedup via prefix caching
• Medusa (arXiv:2401.10774) — 2-3x faster generation
• A-Mem (arXiv:2502.12110) — Episodic memory patterns

Part of the A3M Router ecosystem. "Nobody does parallel multi-LLM execution with result merging. Everyone does sequential fallback."