learn-content-engine

Framework-agnostic TypeScript engine that parses and validates lesson content from pluggable sources into a canonical lesson object.

It takes raw content (a single-JSON lesson plus a manifest.yaml) and a set context, and produces canonical lesson / set-entry objects. It contains no network, storage, or UI code - you supply the bytes and keep fetch + persistence. The bundled, strict JSON-Schema makes it a self-contained format reference: you can author and validate lessons without the application the format originated in (Adaptive Learner). Beyond validity it checks a lesson against the shared quality minimums (validateLessonQuality), turns a parametric exercise into a concrete instance (resolveExerciseVariables) and maps a lesson run's percent to the verdict of its set's evaluation block (gradeFor), so every consumer gets the same answer. Tracks the lesson schema, currently v1.19.

Install

npm install learn-content-engine

ESM, ships TypeScript declarations, Node >= 18.

As a git dependency

For development loops against an unreleased revision, or when consuming a fork, install straight from GitHub, pinned to a commit or tag for reproducibility:

// package.json of the host app
{
  "dependencies": {
    "learn-content-engine": "github:astrapi69/learn-content-engine#<commit-or-tag>"
  }
}

dist/ is not committed, so on install npm runs the package's prepare script (npm run build) to compile dist/ (JS + .d.ts) from source in the checkout. No extra step is needed in the host - a plain npm install builds the engine.

Quick example

import { parseLesson, validateLesson, type LessonSetContext } from "learn-content-engine";

const context: LessonSetContext = {
  language: "fr", target_language: "fr", source_language: "en", domain: "language",
};
const raw = `{ "id": "01", "title": "Greetings", "steps": [
  { "id": "s1", "type": "exercise",
    "exercise": { "id": "e1", "type": "free_text", "prompt": "Say hello.", "accept": ["bonjour"] } }
] }`;

const lesson = parseLesson(raw, context);        // canonical ContentLesson (set context injected)
const result = validateLesson(JSON.parse(raw));  // explicit, opt-in validation
if (!result.valid) console.error(result.errors); // [{ path, message, id, severity, docAnchor, params? }, …]

Documentation

• Getting started - install, the 5-minute pipeline example.
• Concepts - the pipeline, context inheritance, the legacy alias, schema policy.
• Lesson format reference - every field and exercise type, with tested examples.
• Schema diagrams - four pictures of the format; the two schema-derived ones are generated and drift-gated.
• Authoring patterns - expressing common exercise ideas (true/false, conjugation, synonyms, collocations, word order, sung chunks) with the existing types.
• Validation - the strict schema, the semantic rules, the quality minimums, the error model, and the /rules entry for browser consumers.
• Extensions - opt-in ext: exercise types, the portability contract, the registry.
• QTI interop - the optional QTI import/export adapter (2.x and 3.0 dialects) and its qti import / qti export command, mapping table, fidelity limits.
• Architecture - the engine boundary, consumer parity, roadmap.
• Comparative analysis - exercise types benchmarked against Moodle, H5P, QTI 3.0, Canvas and Duolingo; the core / extension / adoption tiers, why not QTI or H5P as the native format, what is still open.
• API reference - generated TypeDoc for the core, /rules and /qti entry points.
• Contributing - TDD workflow, release gate, adding an exercise type.
• Security policy - supported versions, private vulnerability reports.
• Code of conduct - Contributor Covenant 2.1.

Reference extensions

The core enum has six exercise types. Twelve more are worked out as reference extensions under src/examples/ext-ref-*, each with an engine half (payload validation), a consumer half (minimal render and grade), tests, and a doc-gated example lesson in Extensions. They are excluded from the published package; a consumer adopts one under its own vendor namespace (adaptive-learner runs ext:ref-categorization as ext:al-categorization). Before concluding a type is missing, look here:

| Extension | Works out | |---|---| | ext:ref-ordering | put these items in the correct order | | ext:ref-categorization | sort items into their buckets (1:n) | | ext:ref-error-correction | one token in the sentence is wrong: mark it, correct it | | ext:ref-reading-comprehension | a passage bound to N sub-questions (stimulus with questions) | | ext:ref-graded-quiz | scored question set with points, partial credit, pass threshold | | ext:ref-dictation | hear a clip, type what you heard | | ext:ref-image-description | look at the picture, answer in free text | | ext:ref-audio-choice | gapped sentence, pick the audio clip that fills it | | ext:ref-audio-tiles | hear a sentence, build its translation from word tiles | | ext:ref-speak-and-record | hear, reveal, record yourself (ungraded) | | ext:ref-hotspot | click the correct region of an image (rect and circle zones) | | ext:ref-parsons | arrange scrambled code lines by order and indentation |

Parametric exercises (sampled variables, computed answers) are not an extension but a core field since schema 1.14: variables on any exercise (engine#151, see Variables). The gap analysis behind this list is docs/comparative-analysis.md.

Public API

| Export | Kind | Purpose | |---|---|---| | parseLesson | fn | raw source + context → canonical ContentLesson (via an adapter) | | singleJsonLessonAdapter | fn | the built-in single-JSON source adapter | | parseManifest | fn | raw manifest.yaml text → ParsedManifest | | asContentSetEntry | fn | raw parsed set → canonical ContentSetEntry | | inheritFromRootEntry | fn | root-manifest entry + set-file entry → one ParsedSet; a field the set file leaves silent inherits the root value (engine#246) | | resolveSetEntry | fn | root-manifest entry + set-file entry → the one ParsedSet to project; the root owns ROOT_OWNED_SET_FIELDS, a silent field falls back to the other file (engine#246) | | ROOT_OWNED_SET_FIELDS | const | the set fields the root entry owns: title, description, visibility, review_status | | resolveLanguagePair | fn | language-pair resolution (legacy alias + en default) | | setBasePath | fn | repo-relative base dir for a set | | asContentSetBook | fn | project a manifest book block → ContentSetBook \| null | | validateLesson | fn | validate a lesson against the bundled schema + semantic rules → ValidationResult | | validateManifest | fn | validate a manifest against the bundled schema (legacy alias normalized) | | validateManifestPair | fn | compare a set manifest with its root-manifest entry; E-MANIFEST-ENTRY-MISMATCH per differing field (engine#246), W-MANIFEST-DESCRIPTION-COPY when the set manifest carries the root-owned description (engine#268) | | validateLessonQuality | fn | check a shape-valid lesson against the quality minimums, keyed to its purpose (practice, bridge, quiz) → ValidationResult of E-QUALITY-* shortfalls; a publication threshold, not validity (Quality minimums) | | resolveExerciseVariables | fn | resolve a parametric exercise into a concrete instance: sample, evaluate in declaration order, round, substitute every {{name}}; returns the exercise, the values (to persist and replay) and the tolerances of pure-reference accepted answers (Variables) | | evaluateExpression | fn | evaluate a computed variable's expression with values for its names, on the parser the validator uses | | gradeFor | fn | a lesson run's percent under its set's evaluation block: passed against pass_percent (inclusive) and, under grades, the row with the highest min_percent the run reaches; the consumer computes the unrounded percent (evaluation) | | QUALITY_MINIMUMS | const | the numbers of schema/quality-rules.json that validateLessonQuality applies | | RENAMED_RULE_IDS | const | every retired rule id mapped to the id its rule carries now; a rule id's prefix carries its tier, so a raise or lowering renames it (Rule catalog) | | currentRuleId | fn | the id a rule carries now: id itself, or for a retired id the one it was renamed to; for a mirror or an accepted-warnings list keyed on full ids | | collectStableIds | fn | set-wide stable_id view over several lessons: total count + duplicates with locations (the cross-lesson half the schema cannot see) | | buildStableIdInventory, compareStableIdInventories, formatStabilityResult | fn | the pure core of the shipped check-stable-ids gate: published-state vs head, violations V1-V6 (V5/V6 read each tree's declared retired_ids, engine#131; V3 accepts a change the head declares in type_migrations, engine#254) | | computeStableIdCoverage, gateStableIdCoverage, formatCoverageResult | fn | the pure core of the shipped check-stable-id-coverage gate: how many listed sets are fully minted, judged against the repo-local baseline | | isBaseCredible | fn | whether a comparison base is a plausible predecessor: an empty history is a broken run, not a clean one (the floor under the stability gate) | | stableIdDeclarations | fn | a set manifest's retired_ids and type_migrations, tagged with the set, as the stability gate reads them (engine#254) | | TYPE_MIGRATIONS | const | the type changes a set may declare under a kept stable_id: cloze -> multiple_choice, cloze -> free_text (engine#254) | | isAllowedTypeMigration | fn | whether from -> to is one of TYPE_MIGRATIONS | | lessonIdOrderingIssues | fn | set-level ordering check over a set's lesson ids: mixed NN- prefixes, inconsistent widths, lexicographic-vs-numeric divergence (validateManifest runs it over metadata.lessons) | | KNOWN_CONTENT_DOMAINS | const | the canonical domain vocabulary of the known-values-plus-other contract (engine#127); consumers group their subject facet on it instead of keeping a copy | | CEFR_LEVELS | const | the CEFR proficiency bands (A1..C2) a language set declares as its level | | LEVEL_NONE | const | the explicit no-level sentinel ("none") a deliberately level-less non-language set declares (engine#127) | | isKnownContentDomain | fn | whether a domain value is in the canonical vocabulary (case-insensitive; absent counts as the language default) - the W-DOMAIN-UNKNOWN lint's predicate | | isKnownLevel | fn | whether a level is a CEFR band or, for a non-language set, the none sentinel - the W-LEVEL-UNKNOWN lint's predicate | | ContentLesson, ContentSetEntry, ContentSetBook, ContentSetSource, … | types | the canonical internal format | | ContentLessonInlineExample | type | one inline worked example (schema v1.5) on a theory step or exercise | | ContentLessonCard, ContentLessonExercise, ContentLessonStep, ContentLessonResource, ContentLessonClozeBlank, ContentLessonCardTokenRole, ContentLessonCardTokenRoleName | types | the rest of the ContentLesson field family (cards, exercises, steps, cloze blanks, token roles) | | ContentCardMediaType, ContentExerciseDirection, SetStatus, SetVisibility | types | enum-like fields on the canonical types above | | SetReviewStatus, SetAttribution | types | the set entry's review standing (schema v1.9) and attribution block | | StableIdReport, StableIdDuplicate | types | the collectStableIds return shape | | StabilityResult, StabilityViolation, StableIdElement, StableIdInventory | types | the buildStableIdInventory/compareStableIdInventories return + input shapes | | TypeMigration | type | one allowed { from, to } pair of TYPE_MIGRATIONS | | StableIdCoverage, CoverageVerdict, CoverageSet, CoverageFailure | types | the computeStableIdCoverage return shape | | ResolvedExerciseVariables, ResolveExerciseVariablesOptions | types | the resolveExerciseVariables return shape and its options (random, values) | | EvaluationOutcome | type | the gradeFor return shape: the scheme applied, passed when the block has a pass mark, grade under grades | | ValidationResult, ValidationIssue, ValidationSeverity, ValidationParams, ValidationParamValue | types | the validate* return shape ({ valid, errors[], warnings[] }), its issue-severity enum, and the optional params an issue carries when its message names a value | | ExerciseExtension, ExtensionRegistry | types | the ext: extension registry contract (schema 1.7, see Extensions) | | LessonSetContext, LessonSourceAdapter, ParsedManifest, ParsedSet, ParsedSetAsset, ParsedSetBook | types | adapter + manifest surface | | Card, CardTokenRole, ClozeBlank, Direction, Exercise, ExerciseType, InlineExample, Lesson, LessonPurpose, LessonResource, LessonStep, MediaType, Pair, PictureImage, StepType, TokenRole | types | the underlying generated schema element types (one per schema/lesson.schema.json $defs entry ContentLesson* wraps) |

The bundled JSON-Schema ships too, so a content repo can mirror against it directly: import schema from "learn-content-engine/schema/lesson.schema.json". The same holds for quality-rules.json (the numbers validateLessonQuality applies) and grading-presets.json (the grading presets catalog).

Two subpath entries sit next to the package root:

| Entry | Exports | Purpose | |---|---|---| | learn-content-engine/rules | validateLessonRules, validateManifestRules, validateLessonQuality, QUALITY_MINIMUMS, RENAMED_RULE_IDS, currentRuleId, gradeFor (with its EvaluationOutcome), isSlugId, SLUG_ID_PATTERN, SLUG_ID_MAX_LENGTH, plus two helpers the structural layer composes: unusedCardIds (the detection core of W-CARD-UNUSED) and normalizeManifestAliases (maps the legacy language alias to target_language before validation) | the semantic rules and author lints without the structural layer: no ajv, no node:*, for a browser consumer that has already shape-checked its input (Validation) | | learn-content-engine/qti | importQti, exportQti, qtiLessonAdapter, ... | the optional QTI adapter and its XML parser (QTI interop) |

Scope

By design, this package contains only parse / transform / validate / types + the single-JSON source adapter - no fetch, storage, or UI; those stay in the consumer. See architecture.md. The adaptive-learner app consumes this library (pinned in its frontend/package.json) as the reference consumer, so parse/validate/types live here once. As of v0.6.0 this engine is the canonical source of the lesson schema (schema authority moved here, roadmap stage 4); consumers - adaptive-learner and the content repos - mirror it. See Schema authority.

What this is NOT

This is a language-learning-shaped lesson engine: the format is built around cards, drill-style exercise types, and a target/source language pair (see concepts.md). The shape carries more than languages, though - a domain field (language, programming, psychology, ...; known values plus other, see content domains) lets the same format hold knowledge-domain sets (tech courses, driving-test prep, dog training in the dedicated domain repos below); there, target_language is simply the language the content is written in. It is deliberately not:

• a general assessment standard - the CORE schema covers the exercise types its consumers render and grows additively when a consumer needs a new core type. A consumer needing a bespoke type can add one via the opt-in extension tier (ext: types) without touching the core enum; the core stays the portable authority.
• a runtime - no rendering, answer checking, scoring of a run, scheduling/SRS, persistence, or networking; consumers own all of that. What it computes, it computes as pure functions of what the format defines: resolveExerciseVariables (since 0.34.0) turns a parametric exercise into one instance from the random source the consumer passes in, and gradeFor (since 0.41.0) maps the percent a consumer computed for a run to the verdict of the set's evaluation block. The grading presets are a catalog of such blocks, not grading code.
• a content repository - it ships the format, the validator, and the author tooling, not lessons.

Example repositories

The engine is used in the wild - these repos show the full consumer setup (pinned engine version, byte-mirrored schema artifacts, make lint running the same validator locally that CI enforces):

• adaptive-learner-content-template - fork-and-go scaffold with one example set, the validator wiring, CI gates and docs; the fastest way to see the engine used end to end.
• adaptive-learner-content-test - the test/starter repo and a conformance target of the engine's make conformance-real, plus the full author-tooling setup.
• adaptive-learner-content - the production content repo: the language sets plus the app tutorial (28 sets and counting).
• the alc-* domain repos - one repo per knowledge domain, all created from the template and registered in the app's repo registry: psychology, programming, technology, ai, traffic-knowledge, dog-training, die-waehrung-des-geistes and books (one set per book).

Schema authority

The canonical source of the lesson schema is this engine's schema/lesson.schema.json (+ content-manifest.schema.json, quality-rules.json, grading-presets.json). It is an authored artifact here; consumers mirror the schema shipped in each pinned engine release:

• adaptive-learner (the reference consumer) keeps its Pydantic models as an editorial tool for its own runtime types, but its generated schema must be byte-identical to the engine's - its parity gate treats the engine as the reference (the consumer conforms to the engine, not the reverse).
• Content repos mirror the schema from the pinned engine release via their drift tool (schema/engine-version.txt).

The lesson schema's $id is engine-owned: https://astrapi69.github.io/learn-content-engine/schema/lesson.schema.json.

To evolve the schema, edit the artifact here (the frozen byte baseline in src/schema-baseline.test.ts guards against accidental content drift), run make sync-types to regenerate src/types/lesson-schema.generated.ts from it, mirror any new cross-field rule in src/rules.ts, extend the fixtures + rule catalog, and bump the version; consumers then re-pin. The TypeScript types are generated here (in-engine, scripts/generate-lesson-types.mjs), so they cannot drift from the schema; the drift gate runs in release-check + CI.

Changelog

See CHANGELOG.md - one dated section per release, from the current release back to 0.1.0.

License

MIT © Asterios Raptis