Clerestory

Clerestory is a procedural text generation tool based on the concept of context-free grammars heavily inspired by Kate Compton's Tracery and Ian Holmes's Bracery.

The aim is to provide a simple and expressive syntax which can work well in the context of a larger project where some state is managed from outside the text-generation grammar -- what the authors of Bronco call a yielding grammar.

This is still WIP and API/expression syntax may change significantly.

Why use this

The purpose of this project is to provide a lightweight, efficient, productive tool for ad hoc generation of procedural text, ranging from person or place names to item descriptions to entire dialogues or stories. The API provides ways to guide what text is generated, such as conditional rules and different distributions.

Key concepts

Architecture overview

The main elements of Clerestory are Expressions, Symbols, and Grammars. An Expression is a single unit of parseable text that always evaluates to a string. Expressions have syntax that allows for referencing other Symbols on the same grammar, generating randomized alternations, and applying text transformations (called modifiers). A Symbol is a set of rules consisting of one or more Expressions, with some config about which rules to choose. When evaluated it also resolves to a string, after selecting a rule and evaluating the Expression it contains. Once evaluated, a Symbol's value is fixed (unless you change it). Finally, a Grammar is simply a stateful set of Symbols. You can access (or change!) individual Symbols on Grammar.state or expand() the Grammar from its defined origin to produce a single string output.

Grammar

The main artefact you will interact with is a Grammar, which is simply a set of Symbols. Expanding a Symbol on a Grammar will produce text by evaluating that Symbol's rules, parsing the resulting Expression, and following references to any other Symbols recursively until a final text is produced.

import { Grammar } from 'clerestory';

const animalGrammar = new Grammar({
    // `pet` and `story` are Symbols
    pet: [ 'cat', 'dog', 'fish'],
    story: 'Yesterday I bought a {pet}.'
});
console.log(animalGrammar.expand('story'));
// > "Yesterday I bought a fish."
console.log(animalGrammar.state.pet);
// > "fish"
// now that the "pet" symbol has been expanded it will keep this value
// until you change it
animalGrammar.state.pet = 'mongoose';
console.log(animalGrammar.expand('story'));
// > "Yesterday I bought a mongoose."

Symbols

A Symbol can be thought of as simply a set of rules for producing a string of text. The simplest possible Symbol just returns a single string, but Symbols have a lot of other powerful features for guiding how text gets generated. You can easily add or change Symbols on a Grammar by accessing its state. Continuing the example above:

// add a new symbol
animalGrammar.state.benefit = ['loyalty', 'personality', 'smell'];
// change an existing symbol
animalGrammar.state.story = 'I love my {pet} for its {benefit}.';
animalGrammar.expand('story');
// > "I love my mongoose for its personality."

Note an important difference with Tracery: in Clerestory, Symbols are "flattened" as soon as they are accessed, whether through a reference in another symbol or by outside code. Tracery differentiates between static variables and dynamic symbols that share the same namespace. Unlike Tracery, Clerestory does not have syntax for setting variables via an expression, but does allow referencing a symbol without flattening it like {~symbol}.

const schrodinger = new Grammar({ cat: ['alive','dead']});
console.log(schrodinger.state.cat);
// > "dead"
schrodinger.state.box = 'I looked in the box and the cat was {cat}.';
console.log(schrodinger.expand('box'));
// > "I looked in the box and the cat was dead."

The expand method on a Grammar or Symbol will always reevaluate the symbol (though any other Symbols referenced will keep their preexisting values).

console.log(schrodinger.expand('cat'));
// > "alive"

You can also access the Symbols on a Grammar directly to modify or reset them:

schrodinger.symbols.cat.reset();
console.log(schrodinger.state.cat);
// > "dead"

Rules

The possible expressions a symbol can produce are called rules. The cat symbol above has two rules: "alive" and "dead"; expanding it will choose one at random. The real power of a context-free grammar comes from the expression syntax, described below, which allows you to reference other symbols, modify symbol outputs, and evaluate conditional statements within these expressions.

Distributions

You can influence which rules a symbol will return by setting a different distribution on the symbol.

random is the default and will pick a rule at random each time. pop will select a rule at random and remove it from the stack. Note that if you run out of rules, the symbol will return an empty string. weighted allows you to set a weight on each rule to determine its relative chance of being picked. You will need to pass rules in object syntax like [{ text: 'cat', weight: 5 },{ text: 'ferret', weight: 1 }]. Weight should be a positive integer. popWeighted works by creating a number of copies of each rule equal to its weight, which are removed as they are used. shuffle acts like a "deck of cards". Rules are shuffled and one is drawn on each evaluation, and when they run out the used rules are reshuffled into a new "deck".

To specify a distribution, you need to use object-style symbol defintion:

const petGrammar = new Grammar({
    pet: { rules: ['cat','dog','fish'], distribution: 'shuffle' }
});

Functional rules

A powerful feature of Clerestory is that it supports passing a function as the "rules" for a symbol. This allows you to completely bypass the built-in rule selection functionality and generate abritrary expressions in pretty much any way you wish.

const daysOfWeek = ['Sunday', 'Monday', 'Tuesday', 'Wednesday', 'Thursday', 'Friday', 'Saturday'];
const dayGrammar = new Grammar({
    today: () => daysOfWeek[new Date().getDay()]
});

Conditional rules

Another powerful feature that Clerestory adds is conditional rules. When defining a rule in object syntax, you can add a condition key, which is simply an expression. The rule will only be chosen if it evaluates to a truthy value.

const hungryGrammar = new Grammar({
    comment: [
        {
            text: 'I am starving!',
            condition: '{isHungry}'
        },
        {
            text: 'I couldn\'t eat a thing!'
        }
    ]
});

You can also set conditionValue to check the condition expression against a defined value (which is also treated as an expression):

const nameGrammar = new Grammar({
    firstName: [
        {
            text: '{firstNameFeminine}',
            condition: '{gender}',
            conditionValue: 'feminine'
        },
        {
            text: '{firstNameMasculine}',
            condition: '{gender}',
            conditionValue: 'masculine'
        }
    ]
});

Expressions

Expressions are the individual bits of text produced by symbol rules and parsed by Clerestory. Expression syntax is very simple, largely inspired by Tracery.

In this expression:

Why hello there, {traveler}!
// Why hello there, Gandalf!

{traveler} is a reference to another symbol in the same grammar. Once referenced in this way, the symbol traveler will persist its generated value. If you want to reference a symbol without saving the output, use {~traveler} which will always expand the symbol without flattening it.

Alternations

Bracery-style alternations are also supported:

[Hi|Hello|Yo], {traveler}!
// Hello, Gandalf!

You can even nest symbol references inside alternations:

[Hi {traveler}!|{traveler}, what's up?|Howdy!] How fares your quest?
// Gandalf, what's up? How fares your quest?

Clerestory also supports Tracery-style modifiers. These are generally simple text transformations which can be chained. A few useful defaults are included and you can add your own.

{animal.s.uppercase}
// CATS

Digits and letters

You can generate random digits inside alternations. The syntax: [#] will generate a random digit between 0 and 9, so [###] would be anything from 000 to 999. For more control, you can use n-n syntax to generate a digit in a specific range. [2-50] will generate a digit between 2 and 5 followed by a 0, so 20, 30, 40, or 50. @ works similarly and will generate a random letter.