@origints/markdown

Markdown parsing and manipulation for Origins with full lineage tracking.

Features

• Parse Markdown with GFM (GitHub Flavored Markdown) support
• YAML frontmatter extraction
• Source position tracking for all nodes
• Type-safe navigation and extraction
• Convert to HTML
• Support for tables, task lists, and footnotes
• Integrates with Origins transform registry

Installation

npm install @origints/markdown @origints/core

Usage with Planner

Extract content from a Markdown file

import { Planner, loadFile, run } from '@origints/core'
import { parseMarkdown } from '@origints/markdown'

const plan = new Planner()
  .in(loadFile('README.md'))
  .mapIn(parseMarkdown())
  .emit((out, $) => out.add('title', $.select('heading').text()))
  .compile()

const result = await run(plan, { readFile, registry })
// result.value: { title: 'My Project' }

Extract collections with selectAll

Use selectAll() to extract data from all matching nodes as an array:

// Extract all heading texts from a document
const plan = new Planner()
  .in(loadFile('README.md'))
  .mapIn(parseMarkdown())
  .emit((out, $) =>
    out.add(
      'headings',
      $.selectAll('heading', node => node.text())
    )
  )
  .compile()

const result = await run(plan, { readFile, registry })
// result.value: { headings: ['Introduction', 'Getting Started', 'API'] }

Extract structured data from repeated nodes

// Extract all link URLs and labels
const plan = new Planner()
  .in(loadFile('README.md'))
  .mapIn(parseMarkdown())
  .emit((out, $) =>
    out.add(
      'links',
      $.selectAll('link', node => node.text())
    )
  )
  .compile()

Extract top-level children

Use children() to extract each direct child of a node:

const plan = new Planner()
  .in(loadFile('README.md'))
  .mapIn(parseMarkdown())
  .emit((out, $) =>
    out.add(
      'blocks',
      $.children(node => node.text())
    )
  )
  .compile()

Extract frontmatter fields

// doc.md:
// ---
// title: My Post
// date: 2024-01-15
// tags:
//   - typescript
//   - origins
// ---
// # Content here

const plan = new Planner()
  .in(loadFile('doc.md'))
  .mapIn(parseMarkdown())
  .emit((out, $) => out.add('title', $.select('yaml').text()))
  .compile()

Combine Markdown with other sources

const plan = new Planner()
  .in(loadFile('README.md'))
  .mapIn(parseMarkdown())
  .emit((out, $) => out.add('title', $.select('heading').text()))
  .in(loadFile('package.json'))
  .mapIn(parseJson())
  .emit((out, $) =>
    out
      .add('version', $.get('version').string())
      .add('name', $.get('name').string())
  )
  .compile()

Standalone usage (without Planner)

For direct Markdown navigation:

import { parseMarkdownImpl, MarkdownNode } from '@origints/markdown'

const node = parseMarkdownImpl.execute(markdownString) as MarkdownNode

// Select nodes using CSS-like selectors
const headingResult = node.select('heading')
if (headingResult.ok) {
  console.log(headingResult.value.text())
}

// Select by attribute
const h1Result = node.select('heading[depth=1]')
const codeResult = node.select('code[lang="typescript"]')

// Nested selectors
const listItems = node.selectAll('list > listItem')

// Get all text content
console.log(node.text())

Typed node extraction

const headingResult = node.select('heading')
if (headingResult.ok) {
  const data = headingResult.value.asHeading()
  if (data.ok) {
    console.log(data.value.depth) // 1, 2, 3, etc.
  }
}

const linkResult = node.select('link')
if (linkResult.ok) {
  const data = linkResult.value.asLink()
  if (data.ok) {
    console.log(data.value.url)
  }
}

Converting to HTML

import { parseMarkdownImpl, toHtml } from '@origints/markdown'

const node = parseMarkdownImpl.execute('# Hello\n\nWorld') as MarkdownNode
const html = toHtml(node)
// <h1>Hello</h1>\n<p>World</p>

Frontmatter extraction (standalone)

import { parseMarkdownImpl, extractFrontmatter } from '@origints/markdown'

const node = parseMarkdownImpl.execute(markdownWithFrontmatter) as MarkdownNode
const frontmatter = extractFrontmatter(node)
if (frontmatter) {
  console.log(frontmatter.title)
}

API

| Export | Description | | -------------------------------------- | ----------------------------------------------------- | | parseMarkdown(options?) | Create a transform AST for use with Planner.mapIn() | | parseMarkdownImpl | Sync transform implementation (string input) | | parseMarkdownAsyncImpl | Async transform implementation (string or stream) | | registerMarkdownTransforms(registry) | Register all Markdown transforms with a registry | | MarkdownNode | Navigable wrapper with selector support | | toHtml(node) | Convert Markdown to HTML | | toJson(node, options?) | Convert MarkdownNode to JSON | | extractFrontmatter(node) | Extract YAML frontmatter |

License

MIT