llmail-test

Test documentation and state tracking system that works as a plugin of llmail.

Overview

llmail-test helps maintain a history of test states by:

• Assigning unique IDs to tests
• Creating documentation for each test
• Tracking test state changes
• Capturing context when tests pass or fail

Installation

1. Install llmail globally:

npm install -g llmail

2. Install llmail-test in your project:

npm install --save-dev llmail-test

3. Add the plugin to your llmail configuration:

# llmail.yaml
plugins:
  llmail-test:
    testPattern: "**/*.test.{js,ts}"  # Optional: default pattern for test files
    autoSync: true                     # Optional: auto-sync test documentation
    maxRetries: 3                      # Optional: max retries for failed tests

Setup

Initialize test documentation:

# Initialize test documentation
llmail test setup

# Show what would happen without making changes
llmail test setup --dry-run

This will:

1. Find all test files in your project
2. Add IDs to tests that don't have them
3. Run your test suite
4. Create initial documentation

Test Documentation

Each test gets a unique 4-character ID that's added to its name:

// Before
test("should render emoji", () => {
  // test code
});

// After
test("should render emoji (#a62f)", () => {
  // test code
});

Documentation is maintained in your issues directory:

inbox/          # New failures needing triage 
issues/
  ├── _pass/          # Currently passing tests
  │   └── b83c-issues-passed.md
  └── fail/          
     └── a62f-issues-failed.md

Commands

# Initial setup
llmail test setup [--dry-run] [--interactive]

# Run tests and update documentation
llmail test run

# Get current status
llmail test status

# Create/update docs for all tests
llmail test sync

Git Integration

llmail-test automatically tracks git context for your tests:

// Test updates include git context
<Pass time="2024-02-04T12:00:00Z" commit="abc123" branch="main">
Test passed successfully
</Pass>

// Test metadata tracks last passing state
{
  "state": "pass",
  "lastPassingCommit": {
    "hash": "abc123",
    "date": "2024-02-04T12:00:00Z",
    "branch": "main"
  }
}

This helps you:

• Track when tests started failing
• Identify problematic commits
• Maintain test history across branches
• Support investigation and debugging

Requirements

• Node.js 18+
• npm or yarn
• Git repository
• llmail (installed globally)
• Jest or Vitest test runner

License

MIT