spectacle

Given the amount of code generated by agents, the human review is the bottleneck. The idea is to reduce the amount of code to review by moving the review to higher abstraction level of specs. Current implementations of LLMs are not deterministic yet for reliable translation from spec -> code.

As a workaround this is an experiment to do a reverse deterministic projection to higher level of abstraction. Currently going from tests -> specs, which can be reviewed more easily.

spectacle parses test files (using tree-sitter), extracts BDD style comments into Cucumber-compatible markdown-with-Gherkin spec document.

Currently implemented parsers are for .lua, .dart, .ts, .tsx.

Example

Given a test file stack_spec.ts with BDD-style comments:

import { describe, it, expect } from "vitest";

describe("Stack", () => {
  it("pops the most recently pushed value", () => {
    // Given an empty stack
    const stack = new Stack<number>();
    // When 1 and then 2 are pushed
    stack.push(1);
    stack.push(2);
    // Then popping returns 2
    expect(stack.pop()).toBe(2);
    // And popping again returns 1
    expect(stack.pop()).toBe(1);
  });

  it("throws when popping an empty stack", () => {
    // Given an empty stack
    const stack = new Stack<number>();
    // When pop is called
    // Then an error is raised
    expect(() => stack.pop()).toThrow();
  });
});

Running spectacle stack_spec.ts extracts the comments into a markdown-with-Gherkin spec:

# Feature: Stack

### Scenario: pops the most recently pushed value

- **Given** an empty stack
- **When** 1 and then 2 are pushed
- **Then** popping returns 2
- **And** popping again returns 1

### Scenario: throws when popping an empty stack

- **Given** an empty stack
- **When** pop is called
- **Then** an error is raised

TodoMVC Demo

See examples/todomvc/ demo of the technique applied to the TodoMVC app.

• app spec - standard TodoMVC behavior spec used as the checklist of behaviors to cover.
• specs - feature files based on app spec, projected from tests by spectable.
• tests - vitest E2E tests carrying Gherkin comments.

Install

npm install -g @dundalek/spectacle

Usage

# one file → stdout
spectacle test/e2e/behavior_spec.lua

# many files → docs/specs/
spectacle --out-dir docs/specs test/e2e/behavior_spec.lua test/commands_spec.lua

# stdin → stdout
cat my_spec.lua | spectacle

Develop

git clone https://github.com/dundalek/spectacle.git
cd spectacle
npm install

Run tests:

npm test

Run straight from source:

node src/spectacle.ts my_test.ts