Flex-MD (v4.0) — Markdown Output Contract with Smart Token Estimation

Flex-MD is a TypeScript library for building and enforcing Markdown Output Contracts with LLMs. It treats Markdown as a semi-structured data format, allowing you to define required sections, list types, and tables while maintaining 100% standard Markdown compatibility.

What's New in v4.0:

• 🎯 Automatic Token Estimation: Calculate max_tokens directly from your spec
• 📏 System Parts Protocol: Standardized size hints that guide LLMs AND enable token prediction
• 🧠 Smart Toolbox: Cognitive cost analysis, confidence scoring, and improvement detection
• 🔧 Auto-Fix: Automatically improve specs with one command

Key Features

Core (v3.0)

• Standard Markdown: No proprietary tags. Pure headings, lists, and tables.
• Strictness Levels (L0–L3): From loose guidance to rigid structural enforcement.
• Deterministic Repair: Auto-fixes misformatted LLM output (merged fences, missing headings, format conversion).
• Instructions Output Format Guidance: Generate formal "Instructions Blocks" for LLM prompts directly from spec objects.
• Issues Envelope: A structured failure format for when repairs fail, allowing safe fallbacks.

Smart Features (v4.0)

• Token Estimation: Automatically calculate max_tokens for API calls based on your spec
• System Parts: Structured instruction patterns (Length: 2-3 paragraphs, Items: 3-5) that guide LLMs and enable estimation
• Compliance Checking: Validate specs meet quality standards (L0-L3 compliance levels)
• Cognitive Cost Analysis: Measure how much effort your spec requires to write/maintain
• Confidence Scoring: Know how accurate your token estimates will be
• Improvement Detection: Find issues and get actionable suggestions
• Auto-Fix: Apply improvements automatically

Installation

npm install flex-md

Quick Start

1. Define your Output Format Spec (OFS) with System Parts

import { parseOutputFormatSpec, getMaxTokens } from 'flex-md';

const spec = parseOutputFormatSpec(`
## Output format
- Short answer — text (required)
  Length: 1-2 sentences. Be concise and direct.

- Reasoning — ordered list (required)
  Items: 3-5. Explain your logic step by step.

- Assumptions — list (optional)
  Items: at least 2. List any key assumptions made.

empty sections:
- If a section is empty, write \`None\`.
`);

// Automatically estimate max_tokens needed
const maxTokens = getMaxTokens(spec);
console.log(`Estimated max_tokens: ${maxTokens}`); // ~650

2. Use in Your LLM API Call

const response = await fetch('https://api.anthropic.com/v1/messages', {
  method: 'POST',
  headers: { 
    'Content-Type': 'application/json',
    'x-api-key': API_KEY 
  },
  body: JSON.stringify({
    model: 'claude-sonnet-4-20250514',
    max_tokens: maxTokens,  // Automatically calculated!
    messages: [{
      role: 'user',
      content: yourPrompt + '\n\n' + buildMarkdownGuidance(spec)
    }]
  })
});

3. Enforce the Contract

import { enforceFlexMd } from 'flex-md';

const llmResponse = await response.json();
const result = enforceFlexMd(llmResponse.content[0].text, spec, { level: 2 });

if (result.ok) {
  console.log(result.extracted.sectionsByName["Short answer"].md);
  console.log(result.extracted.sectionsByName["Reasoning"].md);
} else {
  console.log(result.outputText); // Issues Envelope
}

System Parts Protocol

System Parts are structured prefixes in section instructions that serve dual purposes:

1. Guide the LLM on expected output size
2. Enable token estimation for max_tokens calculation

Syntax

[SYSTEM_PART]. [OPTIONAL_GUIDANCE]

Examples:

// Text sections
"Length: 2-3 paragraphs. Provide detailed analysis."
"Length: brief. Keep it short."

// Lists
"Items: 3-5. Focus on key insights."
"Items: at least 3. Be comprehensive."

// Tables
"Rows: 5-7, Columns: 3. Include metrics."

// Code
"Lines: 20-30. Include error handling."
"Lines: ~50. Provide complete example."

Allowed Values

| Section Type | System Part Pattern | Examples | |--------------|-------------------|----------| | text | Length: <value> | brief, moderate, detailed, extensive, 1-2 sentences, 2-3 paragraphs | | list | Items: <value> | 3, 3-5, at least 3 | | table | Rows: <value>, Columns: <value> | Rows: 5, Columns: 3, Rows: 3-5, Columns: 4 | | code | Lines: <value> | 20, 15-25, ~50 |

See System Parts Guide for complete reference.

Compliance Levels (for Spec Authors)

Compliance levels measure how much detail you provide in system parts:

| Level | Detail | Cognitive Load | Token Estimation Accuracy | |-------|--------|---------------|--------------------------| | L0 | No system parts | None | Fallback (~±40%) | | L1 | Simple values | Minimal | Basic (~±30%) | | L2 | Ranges allowed | Low | Good (~±20%) | | L3 | Full spec with "at least", "~" | Medium | Precise (~±10%) |

L2 is recommended for most use cases - good balance of effort and accuracy.

Examples by Level

// L0 - No system parts (fallback estimation)
"Just provide a summary."

// L1 - Simple values
"Length: brief. Provide a summary."
"Items: 3. List the main points."

// L2 - Ranges
"Length: 2-3 paragraphs. Provide detailed analysis."
"Items: 3-5. List key insights."

// L3 - Full specification
"Items: at least 5. Include all relevant factors."
"Lines: ~50. Provide a complete working example."

Smart Toolbox

Token Estimation

import { getMaxTokens, estimateSpecTokens } from 'flex-md';

// Quick estimate
const maxTokens = getMaxTokens(spec);

// Detailed estimate with options
const estimate = estimateSpecTokens(spec, {
  includeOptional: true,
  safetyMultiplier: 1.3,
  strategy: 'average'  // 'conservative' | 'average' | 'generous'
});

console.log(estimate);
// {
//   total: { estimated: 650, min: 520, max: 780, confidence: 'high' },
//   bySectionName: { ... },
//   overhead: 60
// }

Compliance Checking

import { checkCompliance, formatComplianceReport } from 'flex-md';

const report = checkCompliance(spec, 2); // Check if meets L2

console.log(formatComplianceReport(report));
// Shows which sections need improvement to meet target level

Confidence Scoring

import { calculateConfidence } from 'flex-md';

const confidence = calculateConfidence(spec);

console.log(`Confidence: ${confidence.grade} (${confidence.overall}%)`);
console.log('Recommendations:', confidence.recommendations);
// Grade: B (82%)
// Recommendations: ["Good confidence, but can be improved", ...]

Cognitive Cost Analysis

import { calculateCognitiveCost } from 'flex-md';

const cost = calculateCognitiveCost(spec);

console.log(`Cost: ${cost.totalCost}/100`);
console.log(`Assessment: ${cost.recommendation}`);
// Cost: 28/100
// Assessment: "Moderate cognitive load - reasonable effort required"

Improvement Detection

import { detectImprovements, formatImprovementReport, autoFix } from 'flex-md';

// Detect issues and opportunities
const analysis = detectImprovements(spec, 2);
console.log(formatImprovementReport(analysis));

// Auto-fix quick wins
const fixResult = autoFix(spec, analysis.improvements, {
  applyQuickWinsOnly: true
});

console.log(fixResult.summary);
// "Applied 4 fixes, skipped 1"

Complete Smart Analysis

import { analyzeSpec, formatSmartReport } from 'flex-md';

const analysis = analyzeSpec(spec, 2);
console.log(formatSmartReport(analysis));

Output:

╔═══════════════════════════════════════════════════╗
║         FLEX-MD SMART ANALYSIS REPORT            ║
╚═══════════════════════════════════════════════════╝

📊 SUMMARY DASHBOARD
──────────────────────────────────────────────────
Compliance:      ✓ L2 PASS
Confidence:      B (82%)
Cognitive Cost:  28/100
Token Estimate:  650 tokens

💡 RECOMMENDATIONS
──────────────────────────────────────────────────
🟢 Low Priority:
   • Good confidence, but can be improved
     → Upgrade a few sections to L3 for better precision
...

Strictness Levels (for LLM Output Enforcement)

| Level | Goal | Guidance | Enforcement | | :--- | :--- | :--- | :--- | | L0 | Plain Markdown | "Reply in Markdown." | None. Accept as-is. | | L1 | Sectioned MD | "Include these headings..." | Headings must exist. | | L2 | Fenced Container | "Return inside a single block..." | Exactly one fenced block. | | L3 | Typed Structure | "Reasoning is an ordered list..." | Enforce list/table kinds. |

Note: These are different from Compliance Levels (which measure spec quality) - Strictness Levels control how strictly Flex-MD enforces the contract on LLM output.

The Repair Pipeline

Flex-MD doesn't just validate; it repairs. Our deterministic 9-step plan handles:

1. Container Normalization: Wrapping or merging multiple fenced blocks.
2. Heading Standardization: Case-insensitive matching and naming cleanup.
3. Missing Headings: Adding required sections as None.
4. Stray Content: Moving text outside headings into a default section.
5. Format Conversion: Transforming bullets to numbered lists (and vice-versa) based on spec.

Real-World Example

import { 
  parseOutputFormatSpec, 
  getMaxTokens, 
  analyzeSpec,
  enforceFlexMd 
} from 'flex-md';

// 1. Define spec with system parts
const spec = parseOutputFormatSpec(`
## Output format
- Executive Summary — text (required)
  Length: 2-3 paragraphs. Summarize findings and recommendations.

- Key Metrics — table (required)
  Rows: 5-7, Columns: 3. Include: Metric, Current, Target.

- Action Items — ordered list (required)
  Items: 5-10. Prioritize by impact.

- Technical Details — code (optional)
  Lines: 20-30. Include implementation examples.
`);

// 2. Analyze spec quality
const analysis = analyzeSpec(spec, 2);
console.log(`Confidence: ${analysis.confidence.grade}`);
console.log(`Max tokens: ${analysis.tokenEstimate.total.estimated}`);

// 3. Use in API call
const response = await anthropic.messages.create({
  model: 'claude-sonnet-4-20250514',
  max_tokens: getMaxTokens(spec, { safetyMultiplier: 1.3 }),
  messages: [{
    role: 'user',
    content: `Analyze Q4 performance.\n\n${buildMarkdownGuidance(spec)}`
  }]
});

// 4. Enforce and extract
const result = enforceFlexMd(response.content[0].text, spec, { level: 2 });

if (result.ok) {
  const summary = result.extracted.sectionsByName["Executive Summary"].md;
  const metrics = result.extracted.sectionsByName["Key Metrics"].md;
  // Use structured output...
}

Advanced Usage

Custom Token Estimation

const estimate = estimateSpecTokens(spec, {
  includeOptional: false,      // Skip optional sections
  safetyMultiplier: 1.5,       // Extra headroom
  strategy: 'conservative'     // Use minimum estimates
});

CI/CD Integration

// validate-specs.ts
import { analyzeSpec } from 'flex-md';

const analysis = analyzeSpec(spec, 2);

const highPriorityIssues = analysis.recommendations
  .filter(r => r.priority === 'high');

if (highPriorityIssues.length > 0) {
  console.error('High priority issues found');
  process.exit(1);
}

Progressive Enhancement

Start simple and upgrade as needed:

// Version 1: No system parts (works, but fallback estimation)
const v1 = `
## Output format
- Summary — text (required)
  Write a summary.
`;

// Version 2: Add L1 system parts (better)
const v2 = `
## Output format
- Summary — text (required)
  Length: brief. Write a summary.
`;

// Version 3: Upgrade to L2 (best balance)
const v3 = `
## Output format
- Summary — text (required)
  Length: 2-3 sentences. Write a summary.
`;

API Reference

Core Functions

• parseOutputFormatSpec(markdown) - Parse spec from markdown
• stringifyOutputFormatSpec(spec) - Convert spec to markdown
• buildMarkdownGuidance(spec, options) - Generate LLM instructions
• enforceFlexMd(text, spec, options) - Validate and repair LLM output

Token Estimation (v4.0)

• getMaxTokens(spec, options?) - Get estimated max_tokens
• estimateSpecTokens(spec, options?) - Detailed token estimate
• parseSystemPart(instruction, kind) - Parse system part from instruction
• estimateTokens(systemPart) - Estimate tokens for system part

Smart Toolbox (v4.0)

• checkCompliance(spec, level) - Validate compliance level
• calculateConfidence(spec) - Score estimation confidence
• calculateCognitiveCost(spec) - Measure spec complexity
• detectImprovements(spec, level?) - Find issues and suggestions
• autoFix(spec, improvements, options?) - Apply automatic fixes
• analyzeSpec(spec, level?) - Complete smart analysis

Reporting (v4.0)

• formatComplianceReport(report) - Format compliance check
• formatImprovementReport(analysis) - Format improvements
• formatSmartReport(analysis) - Format complete analysis

Documentation

Detailed guides can be found in the docs folder:

• System Parts Guide - Complete protocol reference
• Token Estimation Guide - How estimation works
• Smart Toolbox Guide - Using analysis features
• MDFlex Compliance Spec - Output enforcement
• OFS Syntax Guide - Output Format Spec syntax

Migration from v3.0

v4.0 is 100% backwards compatible with v3.0. All existing code continues to work.

To adopt v4.0 features:

1. Add system parts to your spec instructions:

   - Summary — text (required)
   -   Provide a brief overview.
   + Summary — text (required)
   +   Length: 2-3 sentences. Provide a brief overview.

2. Use token estimation:

   const maxTokens = getMaxTokens(spec);

3. Analyze and improve your specs:

   const analysis = analyzeSpec(spec);
   console.log(formatSmartReport(analysis));

Why Flex-MD v4.0?

Before v4.0

// Guessing max_tokens
const response = await api.create({
  max_tokens: 2000,  // 🤷 Is this enough? Too much?
  ...
});

After v4.0

// Precise estimation
const maxTokens = getMaxTokens(spec);  // ✓ 650 tokens (±20%)
const response = await api.create({
  max_tokens,  // 🎯 Right-sized
  ...
});

Benefits

• ⚡ Faster responses: Right-sized tokens mean lower latency
• 💰 Lower costs: Don't overpay for unused tokens
• 🎯 Better accuracy: Clear size expectations guide LLMs
• 🔍 Quality insights: Know your spec's strengths/weaknesses
• 🛠️ Easy maintenance: Auto-detect and fix issues

License

MIT

Flex-MD v4.0 - Smart Markdown contracts for production LLM applications.