@uniweb/projections

Artifacts derived from a Uniweb site's content alone — no foundation, no runtime, nothing rendered.

A Uniweb site is content; a foundation is code; the runtime orchestrates them. Producing HTML needs all three. The artifacts in this package need only the first, which is what makes them identical in every context — browser SPA, prerender, edge SSR, desktop, unipress — and stable when the foundation or the runtime changes.

npm install @uniweb/projections

Most projects never import this directly. uniweb build and uniweb dev emit these artifacts for you. Reach for the package when you are building a tool that consumes a site's content outside the normal build — a custom publisher, an analysis pass, an alternate host.

What it produces

| Artifact | For | Built from | |---|---|---| | llms.txt | agents — discovery | the page graph + each page's first paragraph | | /{route}.md | agents — retrieval | section.content via @uniweb/content-writer | | search index | site search | the same page graph | | sitemap.xml | crawlers — every page, in every language | the page graph, each language's content and its record pages | | robots.txt | crawlers — the site's rules | seo.robots and the site's address |

The index

Follows the llms.txt convention — an H1 title, a blockquote summary, and ## groups of annotated links:

# Uniweb

> Component content architecture — a framework where content authors and
> component developers can't break each other's work.

## Getting Started

- [Quickstart](https://example.com/docs/getting-started/quickstart.md): Create your first Uniweb site in 5 minutes.

## Authoring

- [Working with Collections](https://example.com/docs/authoring/collections.md): Collections let you manage repeating content — blog posts, team members, products — as a set of files.

Two properties are what make this worth having over a sitemap:

• Links point at the .md projection, not the HTML page, so following an entry is one hop rather than fetch-then-strip.
• Descriptions never require an author to have written anything. The chain is page.yml::description → seo.ogDescription → the first paragraph of the first section that has prose, as clean plain text. Grouping is taken from the page graph: content-less container folders become the ## headings.

The per-page markdown

Section bodies in page order, serialized from the stored ProseMirror document. No frontmatter, no section type, no params — type names which component renders a section, which is a rendering assignment and foundation-specific. Emitting it would break the property this package exists for: the output is a projection of the site, identical under a swapped foundation.

The output carries Uniweb dialect for icons (![](lu-house)). That is what the author wrote, and it is more informative to an agent working on a Uniweb site than a lossy translation to plain CommonMark would be.

An inset — ![A diagram](@NetworkDiagram) — contributes its caption as plain text, never its component: a component name is a rendering assignment, like a section's type. The search index reads insets the same way.

sitemap.xml

Every page a site serves, at the URL its own <head> states as canonical, with its URL in each of the site's languages as hreflang alternates. Made in two steps, so each language's content can be read on its own:

import { sitemapEntries, renderSitemap } from '@uniweb/projections'

const entries = languages.flatMap(({ locale, content, records }) =>
  sitemapEntries(content, { locale, records })
)
const sitemap = renderSitemap(entries) // null when there is nothing to list

records are the record pages served in that language, since a parametric page (/books/:slug) is one template in the content. Pass the record entries of a search index (generateRecordSearchIndex(...).entries), and each is listed after the site's own pages by its template's rules.

A page is listed where its URL answers with it. A redirect, a page with no content of its own (it sends visitors to its first page), a rewrite, a draft (hidden), a knowledge: page, a page kept out of search (seo.noindex, its own or the site's) and one naming another URL as its canonical are left out. With no address in the content (config.url) there are no entries, since a sitemap's URLs must be absolute.

robots.txt

renderRobotsTxt(siteContent, { sitemap }) makes the file from seo.robots in site.yml — allow, disallow, crawlDelay, additionalSitemaps, and contentSignals, which say what the content may be used for once fetched — with a Sitemap: line on the site's address when sitemap says the site serves one. A signal the site does not declare is left out: unstated is not no.

It returns null when there is no file to serve: for a site served under a base path, since crawlers read robots.txt only at a host's root, and for a site that declares no rules and has no sitemap to name.

Configuration

Sites configure this under agents: in site.yml. Free, and on by default:

agents:
  index: true          # emit llms.txt              (default true)
  markdown: true       # emit per-page .md          (default true)
  exclude: [/internal] # additional route exclusions

agents: false turns the whole capability off in one word.

Pages are excluded when they set seo.noindex or hidden, when a route segment starts with _, when the route is a dynamic template (/blog/:slug), or when an exclude prefix covers them. noindex/hidden on a content-less container hides its whole branch, since a container is pure structure and suppressing only the heading would orphan its children.

These exclusions are load-bearing rather than tidy-up. An index is more revealing than a sitemap because it describes pages rather than listing them: an unlinked page becomes discoverable and summarized. Weakening them turns the on-by-default into a leak.

Usage

import { renderSiteIndex, renderPageMarkdown } from '@uniweb/projections'

const index = renderSiteIndex(siteContent) // links on the address the site states, `config.url`
const markdown = renderPageMarkdown(siteContent.pages[0])

API

Rendering

| Export | Signature | |---|---| | renderSiteIndex | (siteContent, options?) → string | | renderPageMarkdown | (page, { includeChildren = true, record }?) → string | | resolvePageDescription | (page, { maxChars = 200 }?) → string | | sitemapEntries | (siteContent, { locale, records }?) → entries | | renderSitemap | (entries) → string \| null | | renderRobotsTxt | (siteContent, { sitemap = false }?) → string \| null |

renderSiteIndex options: address, basePath, locale, defaultLocale, exclude, title, description, rootGroupTitle, maxDescriptionChars, records. Links are absolute on the site's address — address, by default the one the content states (config.url) — and root-relative under basePath when it is unknown: a root-relative link still resolves for an agent that arrived via the index.

Record pages. A parametric page (/blog/:slug) is never listed itself. Pass the record entries of a search index as records and each record's page is listed in its place. A record's page projects as renderPageMarkdown(parametricPage, { record }): the page's markdown, then the record's fields as JSON, then each rich-text field as markdown, found through its sections (body.content) but not in a list. A rich-text field markdown cannot carry whole stays in the JSON.

Page graph — selectIndexablePages, groupPagesForIndex, buildPageUrl, applyRouteTranslation, isContainer, isDynamicTemplate.

Config and filenames — resolveAgentsConfig, pageMarkdownFilename, markdownFilenameRoute (its inverse), INDEX_FILENAME.

Answering a markdown request — resolveMarkdownRequest(content, path, context?), for a server that serves the .md links renderSiteIndex makes. It returns what resolveRoute (@uniweb/core/resolve-route) returns for the page the path names: /fr/index.md is the French root, and a redirect goes to the markdown of the page it goes to.

The internal vocabulary is projections. llms.txt is one emitter's filename — a third-party convention that may not survive — so it appears exactly once, in config.js, at the edge. A change of convention costs a constant.

Search — also available from the @uniweb/projections/search subpath: generateSearchIndex, extractSearchContent, generateRecordSearchIndex, mergeSearchIndexes, getSearchConfig, getSearchIndexFilename.

Ranking and results — from @uniweb/projections/search:

• buildSearchStructure and rankSearchEntries index and rank the entries. A ranking says which terms it scored with.
• searchResult(entry, { terms }) turns an entry into the result every search provider returns. Its snippet marks those terms where the engine's own fold finds them, so cafe marks "café". matchRanges gives the same ranges on their own.
• emptyResult, buildSnippet and highlightMatches are its parts. They are also at /search/result and /search/snippet, which load no engine.

Renamed in 0.5.0 — generateCollectionIndex is now generateRecordSearchIndex, and a search entry carries type: 'record' / group / id: "record:<group>:<slug>" where it carried type: 'collection' / collection / id: "collection:…". "Collection" is a build-side concept — a named set compiled to one file — and this function is called with records that may never have been one.

import { generateSearchIndex, isSearchEnabled } from '@uniweb/projections/search'

if (isSearchEnabled(siteContent)) {
  const index = generateSearchIndex(siteContent, { locale: 'en' })
}

Search extraction lives here rather than in @uniweb/build so that one implementation serves every publisher. @uniweb/build re-exports this module, so existing call sites are unchanged.

Localization

The default locale's artifacts sit at the output root; other locales go under /{locale}/, mirroring the search index's established pattern. Pass locale and defaultLocale to renderSiteIndex and it builds locale-correct URLs, including translated route segments from config.i18n.routeTranslations — the same rule the sitemap uses, so the two can never disagree about where a localized page lives.

Environment contract

Runs anywhere JS runs. No node:*, no bundler, no DOM, no filesystem — enforced by tests/environment.test.js, which walks the real import graph.

This is the package's reason to exist rather than a nicety. The same site's artifacts are made in more than one place — by the static build, and by a host that serves the site from its content and derives them on the request path, in a Worker. An artifact derived from site content has to come out identical whichever made it, or a site's artifacts change with whoever served them — and an index that exists on one host and vanishes on another is worse than none, because agents are told to rely on it. One implementation, imported by each.

The same contract is why search/generate.js imports the leaf subpath @uniweb/core/locale-config rather than the bare @uniweb/core entry, whose package root pulls in heavier dependencies. The environment test forbids the bare entry, so the trap cannot be re-entered.

Development

pnpm test    # vitest

License

Apache-2.0