@growth-labs/cms
v0.8.8
Published
The Fronts-proven publishing/admin engine, packaged for reuse. `@growth-labs/cms` is a versioned library each site installs and parameterizes with its own `D1Database`, R2 media bucket, authz, and theme tokens — **not** a shared runtime engine. See [`cms-
Readme
@growth-labs/cms
The Fronts-proven publishing/admin engine, packaged for reuse. @growth-labs/cms
is a versioned library each site installs and parameterizes with its own
D1Database, R2 media bucket, authz, and theme tokens — not a shared runtime
engine. See cms-reuse-decision.md
for the package-not-engine ruling.
Status: schema contract + publishing engine + API routes. This package ships the D1 schema contract and its TypeScript types (the WS7-04/06 foundation), the publishing engine — the content data-access core (WS7-05) — and the API route handlers (
@growth-labs/cms/routes, WS7-07): the/api/v1/publisher/*content/authors/media/cron surface as framework-agnostic mountable factories with auth/authz injected. The admin integration mounts multipart podcast uploads at/admin/api/media/podcast/{init,part,complete,abort}.
What's here
- The D1 contract — the 11-table content/revision/media/authors/foundry-callback
schema, verified against prod
fronts-data2026-05-29:content_items,article_content,video_content,podcast_content,content_revisions,content_tags,content_tag_links,content_relations,media_assets,authors,foundry_callback_events.content_items.typeacceptsarticle,video,podcast,newsletter, andpage; article, newsletter, and page body rows are stored inarticle_content. - Row types — a TypeScript interface per table, with the CHECK domains encoded as union types.
- Migration helpers — the SQL as a wrangler-applicable file plus an inlined constant for tests/tooling.
- The publishing engine (WS7-05) — the content data-access core, exported
from the package root and the
@growth-labs/cms/enginesubpath: content CRUD- lifecycle (
createContent,updateContentItem,scheduleContent,publishContent,unpublishContent,duplicateContentItem), revisions (createRevision,getContentSnapshot), tags/relations,ensureUniqueSlug,countWords/estimateReadTime, the publish-validation guard, the scheduled-publish sweep (runScheduledPublish), the article-body validator, and the content-integrity sanitize passes. Every function takes an injectedD1Database(the site supplies its own binding) — no Fronts coupling.
- lifecycle (
The contract excludes identity/entitlement tables (users, subscription*,
*_entitlements, gl_identity_links, CRM, mailer) and the analytics-owned
gl_content_progress. Those belong to other workstreams — pulling them in is the
monolith-creep this boundary guards against.
Intentional schema divergences (do not normalize)
media_assets.created_at/updated_atare TEXTdatetime('now'), while the content tables use INTEGERunixepoch(). This matches prod and the engine code; normalizing it breaks parity.video_contentusesvideo_id(renamed fromstream_uidin migration 0018). Do not reintroducestream_uid.content_items.typeCHECK includesnewsletter(added in prod 0024) andpage. The full domain isarticle,video,podcast,newsletter, andpage.
Apply the schema
The supported adoption path is wrangler, against the package's migrations/:
wrangler d1 migrations apply <YOUR_D1_BINDING>For tests and provisioning tooling without disk access, the same statements are available programmatically:
import { applyMigrations, CMS_TABLES } from '@growth-labs/cms/schema'
await applyMigrations(env.SITE_DB) // tests / provisioning only — never request-timeUse the types
import type { ContentItemRow, MediaAssetRow } from '@growth-labs/cms'Use the engine
The engine functions take the site's own D1Database as the first argument:
import { createContent, publishContent, runScheduledPublish } from '@growth-labs/cms'
const { id } = await createContent(env.SITE_DB, {
type: 'article',
slug: 'hello-world',
title: 'Hello, world',
content: { bodyMarkdown: '# Hello\n\nFirst post.' },
})
await publishContent(env.SITE_DB, id, Math.floor(Date.now() / 1000))
// In a scheduled handler:
await runScheduledPublish(env.SITE_DB)primaryCategory (stored in content_items.channel) is site-supplied taxonomy;
it defaults to 'analysis' (the Fronts default) when omitted.
Masthead consumers can expose named category controls by passing
primaryCategories: [{ slug, label }] to the Astro integration. The editor
persists the selected slug through primaryCategory, and the Library can filter
the same canonical content_items.channel value.
Consumers can require a bounded primary-topic selection for chosen content
types. The editor persists the selected slug through primaryTopic on the
initial create request and every later update; it does not derive or alter the
content slug:
primaryTopic: {
label: 'Region',
contentTypes: ['article', 'video', 'podcast'],
options: [
{ slug: 'middle-east', label: 'Middle East' },
{ slug: 'global', label: 'Global' },
],
}When this control applies to a content type, Masthead will not create the item
until an editor makes a selection. Content types outside contentTypes keep
the package's existing topic behavior.
Sites can also expose bounded metadata selects without forking Masthead:
contentMetadataSelects: [{
key: 'format',
label: 'Format',
contentTypes: ['video'],
options: [
{ value: 'briefing', label: 'Briefing' },
{ value: 'series', label: 'Series' },
],
defaultValue: 'briefing',
inferenceMarkerKey: 'format_inferred',
}]Values live in the existing bounded content_items.metadata_json object and
participate in normal revisions. Masthead preserves undeclared keys; a human
selection clears only the configured inference marker.
The Masthead rich-text editor keeps link and underline marks off leading and trailing whitespace for both toolbar selections and pasted formatted content. Underline is trimmed independently from the link mark: boundary whitespace is layout, while underline on the actual linked text is preserved. Markdown serialization applies the same normalization as a final save-time safeguard.
Lossless tags
Content tags store canonical identity and declared display labels separately.
content_tags.slug is the global canonical slug identity; content_tag_links.label
is the per-content display label declared by that item. This preserves cases where
multiple labels canonicalize to the same slug across the fleet, for example
kim-jong-un with declared labels kim jong-un, kim jong un, and
kim-jong-un.
Every tag write/read boundary uses the shared taxonomy validator. Labels must be
trimmed, NFC-normalized, nonblank strings that slugify to a canonical slug, with
no duplicate canonical slugs within one content item. The measured fleet evidence
in docs/cms-taxonomy-fleet-evidence.{json,md} set the package bounds:
MAX_ITEM_TAG_COUNT = 32MAX_TAG_LABEL_BYTES = 128MAX_ITEM_TAGS_TOTAL_BYTES = 2048
Revision payloads carry parallel tags and tagLabels arrays. Older revisions
that only have tags remain readable and restore deterministically by treating
the canonical tag as its legacy display label.
The duplicateContentItem column-alignment gotcha
duplicateContentItem does four INSERT…SELECT copies whose explicit column
lists are hand-maintained and must stay positionally aligned to the schema.
created_at/updated_at are deliberately omitted so the copy gets fresh
DEFAULT (unixepoch()) timestamps; the duplicate is forced to status='draft',
featured=0, null publish timestamps/revision, and a video copy is forced to
processing_state='ready' (NOT re-queued to Foundry). Any future
content_items column addition requires a matching edit in
duplicateContentItem or the duplicate silently drops the new field.
Portable Text body (schema v1)
article_content.body_portable_text (migration 0024, nullable) holds the
structured body as a versioned envelope: { "version": 1, "content": [...] }
where content is a Portable Text array. Portable Text is the durable
store; body_markdown is a derived view written from the same editor doc on
every save — legacy markdown consumers keep working unchanged, and a NULL
envelope means "no structured body, render the markdown".
src/schema/portable-text.ts— the envelope contract:PORTABLE_TEXT_VERSION,wrapPortableText,parsePortableTextEnvelope(zod-validated; returnsnullfor anything malformed so consumers fall back to markdown). Exported via@growth-labs/cms/schema.src/ui/editor/portable-text.ts— pure serializersdocToPortableText/portableTextToDocbetween the Tiptap doc and the Portable Text array.- Vocabulary v1: standard
block(decoratorsstrong/em/code/strike-through,linkmarkDefs, flat lists vialistItem+level), plus custom typesimage,code,tweetEmbed,horizontalRule, andtable. Tables follow the@portabletext/markdowncanonical shape —rows[] → cells[] → value: block[]— so cells hold full Portable Text and in-cell links are ordinary annotations. Cell extension fields:header,colspan,rowspan,colwidth,align(these have no markdown form; the derived GFM pipe table drops them). - Staleness rule: a
bodyMarkdownwrite that does not carry a freshbodyPortableTextNULLs the stored envelope (same pattern asbody_html), so an external markdown writer can never leave a stale structured body for renderers to prefer. - Golden round-trip tests:
__tests__/ui/portable-text.test.ts. - Typed editorial blocks (Phase 1):
callout({ tone, content }, markdown form = GFM alert> [!NOTE]),pullQuote({ attribution, content }, markdown form =:::pullquotedirective),generatedFaq({ markdown }, the machine-owned<!-- generated-faq -->region as an opaque atom that re-serializes byte-identically), and thecitationannotation (markDef { _type: 'citation', href }, markdown form[[label]](href)). Editor node specs live insrc/ui/editor/blocks.ts. - Editor UX: a
/slash menu at line start inserts any block (src/ui/editor/slash-menu.ts— the item list + filter are pure and tested), a drag handle reorders blocks (@tiptap/extension-drag-handle; its collaboration peer set is a hard static-import requirement at the pinned tiptap version and ships in the admin bundle only), and toolbar buttons cover callout / pull quote / citation. - Site tooling imports the serializer quartet from
@growth-labs/cms/content(backfills, HTML-diff gates, importers) — never from deepui/editorpaths.
Page layouts (@growth-labs/cms/puck)
article_content.layout_json (migration 0025, nullable) stores a Puck page
layout as a versioned envelope { "version": 1, "data": <Puck Data> } for
builder pages; NULL means "classic page, render the body". Unlike
body_portable_text, the layout is not derived from the editor doc: on
update, provided → written (explicit null clears), absent → preserved, with
no coupling to bodyMarkdown staleness. bodies:false published reads strip
it with the body columns.
The @growth-labs/cms/puck subpath ships the builder library:
createPuckConfig({ renderRichTextHtml })— the shared PuckConfig(components v1:Section(slot container),Heading,RichText,Image,CTAButton,Spacer). Render components are server-safe (no hooks, no DOM): the public site renders them on Cloudflare Workers with zero client JS via<Render>from@puckeditor/core/rsc(always import/rsc, never the root entry, which drags the full editor bundle).RichTextstores a Portable Text envelope in its props and delegates HTML projection to the consumer-injectedrenderRichTextHtml— Puck's built-inrichtextfield is deliberately unused (its server path requires a DOM and breaks on workerd).layoutToMarkdown/layoutJsonToMarkdown— lossy projection of a layout to classic markdown (keepsbody_markdownpopulated for search/SEO surfaces when a page is layout-driven).- The layout envelope contract re-exported from
src/schema/layout.ts:wrapLayout,parseLayoutEnvelope(returnsnullfor malformed input so renderers fall back to the classic body path).
@puckeditor/core is an optional peer dependency — only consumers using
this subpath (or the admin Builder mode) need it; outside the admin editor
bundle it is a type-only import.
CMS pins its copy of Puck 0.23.0's complete 20-package Tiptap family, including
@tiptap/core, @tiptap/pm, and @tiptap/react, to the same 3.30.2 editor
line. That cannot constrain a consumer's separate @puckeditor/core peer:
pnpm only applies overrides from the consuming project's root, and Puck's
caret ranges otherwise admit a newer, incompatible Tiptap graph.
Consumers must keep root pnpm.overrides matching the published
tiptapConsumerOverrides map in this package manifest. The map covers Puck's
20 direct editor dependencies plus React's transitive bubble- and
floating-menu packages. Keep the bridge until Puck publishes dependency
constraints that independently produce one editor family; a CMS version or
lockfile refresh alone is not sufficient. @tiptap/y-tiptap follows its own
3.0.x release line and is intentionally outside this override map.
Builder mode (pages)
Page documents get a Classic body / Builder toggle in the Masthead
editor. Builder mounts the Puck canvas (iframe disabled, no-external.css
— no external font fetch) over the same component config the public renderer
uses; RichText blocks embed the Portable Text editor as a custom field. Every
canvas change persists the triple: layoutJson (the wrapped envelope), a
derived body_markdown via layoutToMarkdown (falling back to the previous
body, then the title, so publish validation holds), and bodyPortableText:
null (the derived body is markdown-only — the stale-envelope rule applies).
Saves from the page editor always state layoutJson explicitly — a value
writes it, "Remove layout" clears it with an explicit null; article and
newsletter saves never send the key, so the engine's preserve semantics keep
them untouched.
How to add a new block type
Every block exists in four places; a change is complete only when all four round-trip in the golden tests:
- Editor node — a pure Tiptap
Node.create/Mark.createspec insrc/ui/editor/blocks.ts(no Editor/EditorView usage), registered insrc/ui/editor/extensions.ts, plus minimal styling insrc/ui/styles/broadsheet.css. - Portable Text — a
_typemapping in BOTH directions insrc/ui/editor/portable-text.ts(serializeBlockNodeandparseObject; annotations additionally touch the mark handling inbuildTextBlock/parseSpan). Follow the @portabletext/markdown canonical shape when one exists so ecosystem tooling understands the type. Document the shape insrc/schema/portable-text.ts. - Markdown dialect — a serialization in
serializeBlockand a parse inparseBlocks/parseInlineinsrc/ui/editor/serialize.ts. The derived markdown is what every legacybody_markdownconsumer sees; prefer ecosystem syntax (GFM) over invented directives when possible. - Site renderer — a component for the
_typein each consuming site (fronts:src/lib/article-portable-text.ts). Until the site ships it, the renderer's unknown-type guard falls the article back to the markdown path — safe, but the block renders in its markdown form.
Add golden tests in __tests__/ui/portable-text.test.ts covering: doc → PT →
doc identity, markdown → doc → markdown byte-stability, and (for annotations)
markDef structure. A block whose markdown form is lossy (like table cell
spans) is fine — Portable Text is the durable store — but the loss must be
deliberate and documented in the dialect header of serialize.ts.
Use the routes
The route handlers are framework-agnostic factories at the
@growth-labs/cms/routes subpath. Each handler is a pure
(ctx: RouteContext) => Promise<Response>; the consuming site supplies a thin
adapter that maps its request + bindings + its own authz implementation onto
a RouteContext. The package never decides who is an admin/publisher — it asks
the injected authz guard, so no site's context.locals.user / ADMIN_EMAILS
is baked in.
import { createCmsRoutes } from '@growth-labs/cms/routes'
const routes = createCmsRoutes({
authz: {
requireAdmin: (ctx) => (siteSaysAdmin(ctx) ? null : unauthorized()),
requirePublisher: (ctx) => (siteSaysPublisher(ctx) ? null : unauthorized()),
},
// theme tokens default to the Fronts values when omitted:
// mediaPublicDomain: 'media.fronts.co', mediaR2Binding: 'PUBLIC_MEDIA',
// mediaSiteId: 'fronts', publishTimezone: 'Europe/Paris', …
})
// In an Astro route (the site's adapter builds `ctx` from its APIContext):
export const POST = (c) => routes.content.create(toRouteCtx(c))The handler groups: routes.content (list/create/get/update/action — the
action endpoint covers schedule/publish/regenerate-takeaways/unpublish/archive/
unschedule/duplicate), routes.authors (list/create/get/update),
routes.media (uploadImage/listLibrary/libraryAction/uploadPodcast plus
init/part/complete/abort multipart flows for both podcast audio and video
source uploads), and routes.cron (publish — gated by requireAdmin).
Insight Engine v2 lanes and dismissal rationale
The signed contentInsights.ingest route strictly validates and preserves the
bounded Insight Engine v2 JSON wire contract before writing its existing
evidence and brief TEXT columns. Rows may carry action lanes, coverage,
score-breakdown, judge, and semantic evidence; briefs may carry a format and
content outline. These fields are optional for backward-compatible payloads,
but are never silently stripped when supplied by Foundry. No CMS migration is
needed because the data remains inside the existing JSON blobs.
Masthead presents those rows once in the five canonical action lanes —
commission, refresh, optimize CTR, consolidate, and monitor — with a
Legacy / unclassified fallback for older rows. Cards show only the bounded v2
evidence that a row provides, preserving legacy metrics and briefs without
placeholder values.
POST contentInsights.dismiss accepts exactly { intentKey, reasonCode, note? }.
reasonCode is one of off_topic, already_covered, cant_win,
wrong_format, not_now, or other. The optional note is trimmed once,
stored as null when blank, and limited to 500 characters; other requires at
least three non-whitespace characters. Migration
0022_content_insight_dismissal_reasons adds nullable legacy-compatible
reason_code and note columns, and applies the same enum and length bounds in
D1. The Masthead card action opens an internal editor and removes a card only
after this request succeeds; it never publishes content.
routes.subscriptions keeps paid members, exports, KPIs, plan mix, and paid
gift-order tracking separate from complimentary access administration. The
complimentary surface is explicit: list, grant, revoke, and retry-email
operations require manage_complimentary_access, validate browser-shaped
request and provider response bodies with Zod, and receive the trusted actor as
a server-derived argument. Browser payloads must not include actor/provenance
fields.
The admin UI includes Social Share. Content selection fills destination URLs from the browser origin so URL inputs receive absolute links, and Author Social requires the selected author but does not require a social handle when the operator is minting a link for that author to post themselves. Author Social does not render or submit Campaign; the analytics server owns it. Owned Social keeps Campaign editable and required.
Sites with a canonical author roster can disable ad hoc Social Share authors:
growthLabsCms({
workspaces: [{ id: 'fronts', name: 'Fronts' }],
socialSharing: { allowInlineAuthorCreation: false },
})With that option disabled, both share surfaces accept only synchronized
canonical authors and render a blocking error when canonical attribution is
missing. Canonical author writes that a consumer D1 trigger rejects preserve
the stable recovery class and affected id/slug through the API and UI:
AUTHOR_SYNC_IDENTITY_CONFLICT, AUTHOR_SYNC_NON_CANONICAL,
AUTHOR_SYNC_IMMUTABLE_ID, or AUTHOR_SHARE_CAMPAIGN_INVARIANT.
Survey response reporting
Sites can expose read-only survey results in Masthead without giving the CMS package direct access to the survey database. Inject a storage-neutral survey provider when constructing the CMS routes, and configure the one survey the admin shell should display:
const surveyRoutes = createSurveyRoutes({
authz,
providers: {
surveys: {
getResults: ({ surveyId }) => surveyService.getResults(surveyId),
getCsv: ({ surveyId }) => surveyService.getCsv(surveyId),
},
},
})
growthLabsCms({
workspaces: [{ id: 'fronts', name: 'Fronts' }],
surveyResults: { surveyId: 'survey_fronts_reader_2026_08' },
})The Astro integration's generated admin endpoints read the same provider from
Astro.locals.cmsProviders; a host middleware should inject the provider there.
Framework-neutral consumers can mount surveyRoutes.results and
surveyRoutes.csv directly.
SurveyProvider responses are strictly validated before serialization. The
JSON and CSV routes require view_survey_responses, available only to
owner, senior_editor, and editor. If no provider is injected, the routes
return an explicit 503 and the UI renders an unavailable state. The package
does not own survey storage, signing credentials, or reader-facing survey
routes.
The article-takeaways publish dispatch (Foundry/Hermes) is an OPTIONAL
injected dispatchTakeaways hook on the content config — the package carries no
Foundry coupling. Omit it and publish simply skips queuing takeaways.
Display-subheading generation uses the injected hooks.llm provider. The
editor's “Suggest subheading” action writes content_items.excerpt only when it
is not protected by ai_locked_fields, using up to six deterministic recent
published title/dek pairs from the same database as house-voice exemplars. A
manual publish synchronously generates a missing subheading; if generation is
unavailable, publication continues and Masthead creates an unread
ai_dek_failed notification so the omission is visible and actionable.
Published copy edits require the explicit edit_published capability, granted
to owner, senior editor, and editor roles.
Media transcription dispatch is also injected. When the host supplies the
Foundry media hook, dispatch-to-foundry works for video and podcast content,
and podcast create/update/publish automatically queues processing when
podcast_content.audio_r2_key is present and the row has neither a
processing_trigger_token nor a transcript. Foundry callbacks matched by
processing_trigger_token update video processing state or, for podcasts,
persist transcript, duration, description, and excerpt fields when the callback
payload includes them. Only an accepted callback can emit a terminal host
webhook; stale-token and duplicate callbacks return a successful no-op without
mutating state or replaying terminal effects. Every processing dispatch
requires an authoritative, positive, finite duration_seconds value on the
owning video or podcast row.
The engine passes that exact value as FoundryJobSpec.durationSeconds and fails
before invoking the host hook when the value is missing or invalid.
Hosts may inject an optional CmsProviders.mediaSource provider to normalize
remote video or podcast sources and measure their duration before a create or
update is persisted. The package passes { kind, sourceUrl,
knownDurationSeconds }; knownDurationSeconds is present only when the exact
submitted URL already matches stored source identity. The provider returns the
exact requestedUrl, its
stable resolvedUrl, the storage sourceKind, an authoritative whole-second
duration when available, and explicit readiness blockers. The package owns no
site hostname or portal rules. A failed duration probe does not lose the draft:
the source is saved with a null duration and the response includes
mediaSourceReadiness.ready = false. Foundry dispatch still refuses that row
until a valid duration exists. An unchanged persisted source reuses its stored
duration without another provider call. Clearing a source also clears its kind
and duration. An async result whose
requestedUrl no longer matches the submitted source is discarded as
stale_source_measurement.
The admin measures a selected local video or audio file from browser metadata before upload and rounds any fractional runtime up, so the integer contract never understates the source. Multipart init records that duration together with the exact object key and expected byte size in R2 custom metadata. Completion accepts the duration only after R2 confirms the same key, size, and metadata; a mismatch removes both the object and its staging row. A manually pasted URL or R2 key therefore requires an explicit authoritative whole-second duration. On content create and update, the engine resolves an exact matching package-owned media asset and persists its verified duration; a different supplied duration is rejected before any content mutation. Changing a video source URL/kind or podcast audio key without verified or explicit replacement duration clears the stored duration. Replacing podcast audio also invalidates the old Foundry callback token and clears the old transcript unless a replacement transcript is explicitly supplied.
Sites with publish prerequisites can inject publishReadiness on the content
config. The hook receives { ctx, contentId, type, status } before the generic
publish lifecycle runs. Returning { ready: false, blockers, message } blocks the
manual publish with a 409 and leaves the content status unchanged; throwing from
the hook fails closed with a 503. This keeps media-specific readiness checks in
the host site while preventing a configured site from bypassing them.
Consumer D1 migrations
Wrangler accepts one migrations_dir, while a site commonly already owns
numbered analytics, SEO, or product migrations. Vendor the exact SQL from the
installed CMS package into that existing directory, review it, and commit it:
pnpm exec growth-labs-cms-migrations sync --dir migrations --start 10
pnpm exec growth-labs-cms-migrations verify --dir migrations--start is the first unused site sequence and is mandatory. sync reserves a
contiguous range, copies every package migration without rewriting it, and writes
migrations/.growth-labs-cms-migrations.json with the package version and SHA-256
receipt. It fails before writing if any site SQL already owns a reserved sequence.
Existing managed SQL is never overwritten: consumer drift, upstream mutation of
an already-vendored migration, a changed starting sequence, or a symlink target
fails closed.
The default receipt path is the only manifest allowed inside migrations/.
When --manifest is supplied, it must name a .json file outside the migrations
directory whose parent is an existing non-symlink directory; it can never
collide with or become Wrangler SQL. Sync stages the full SQL set and receipt
before linking targets. A later link, verification, or receipt failure removes
every SQL file created by that attempt. An interrupted process may leave only
exact-hash linked SQL and/or *.tmp-<uuid> staging files, but no valid completed
receipt; verify therefore fails closed until a repeated sync reconciles the
batch. Wrangler ignores the temp files, which are safe to delete after the
repeated sync and verify succeed. Never run the managed D1 workflow without the
required verify step.
Both sync and verify acquire an exclusive adjacent <receipt>.lock file,
so concurrent invocations fail closed instead of racing the manifest update.
An abruptly terminated process can leave a stale lock; first confirm no vendor
or verify process is running, then remove only that lock and repeat sync
followed by verify.
Run sync deliberately after a package upgrade, inspect and commit the new SQL
and receipt, then require verify in pull-request CI before the managed D1
migration workflow. The command only reads and writes local files; it never opens
a network connection or applies a database migration. Do not call schema setup
from a request handler.
Bounded publication kernel
publishOne is the build-independent publication path for a single content item.
It creates a new immutable content_revisions row, asks each injected surface to
prepare revision-addressed non-public artifacts, moves
content_items.published_revision_id with a compare-and-swap, then commits live
surfaces. A new publish writes its immutable revision and durable
content_publication_attempts outbox row in one D1 batch, so a crash cannot
leave an orphan revision with no recovery provenance. The pointer move and the attempt's
transition to committing execute in one D1 batch, so a process crash cannot
leave a moved pointer with no discoverable recovery provenance. A losing pointer
CAS aborts prepared surfaces and never commits live sitemap/feed/search/archive
artifacts for the losing revision. A skipped or failed prepare fails closed and
leaves the canonical pointer unchanged. Every state write carries an optimistic
version guard, so a stale Worker cannot regress a committed or otherwise
terminal attempt. When D1 omits affected-row metadata, the reread row must
exactly match the intended state, version, surface sets, receipts, and diagnostic
or the write is treated as a conflict.
The body validator runs against the exact in-memory snapshot inserted into the
immutable revision. A concurrent draft edit therefore cannot replace the
validated body between validation and revision creation.
The surfaces property is required. Sites that intentionally have no live
surfaces must pass surfaces: [] and allowNoSurfaces: true; omitted surfaces
are treated as a wiring error. Surface names are canonical identifiers and
surrounding whitespace is rejected before any revision or outbox row is written.
import {
listPendingPublicationAttempts,
publishOne,
retryPublicationSurfaces,
rollbackPublication,
} from '@growth-labs/cms'
const receipt = await publishOne(env.SITE_DB, {
contentId,
createdBy: user.email,
surfaces: [
sitemapReconciler,
feedReconciler,
searchReconciler,
],
packageVersion: '0.5.3',
})
await rollbackPublication(env.SITE_DB, {
contentId,
targetRevisionId: priorRevisionId,
surfaces: [sitemapReconciler, feedReconciler, searchReconciler],
packageVersion: '0.5.3',
})Reconciler implementations are site-owned and injected. This package does not
ship a central fleet runtime and does not perform live customer writes by itself.
Every supplied surface must implement both prepare and commit; an inert
object that merely carries a required surface name fails before a revision or
attempt is created. Every hook receives an AbortSignal and is bounded to 10
seconds by default. Callers may set surfaceHookTimeoutMs to a positive integer
up to 30 seconds on publish, rollback, or retry. A timeout aborts the signal,
persists only the fixed surface_hook_timed_out diagnostic, and follows the
same durable failed-prepare, pending-commit, or incomplete-abort recovery path
as the corresponding hook failure. Surface hooks are idempotent and must stop
work promptly when their signal aborts:
prepare(input)may write only revision-addressed staged data that readers do not discover directly.commit(input)runs after the canonical pointer moved and may expose live sitemap/feed/search/archive artifacts for that canonical revision. It must be safe to repeat when a process crashes after the external commit but before its receipt is persisted.abort(input)cleans up staged artifacts after a failed prepare or losing CAS. Abort failures are stored in the attempt and included in the thrown error.
Hook receipts are runtime-normalized before persistence. Unknown statuses,
mismatched surface names, and malformed receipt fields become failed receipts
rather than corrupting the recovery record. Hook-supplied messages and thrown
exception text are never persisted in receipts_json or last_error; durable
diagnostics are fixed allowlisted values capped at 64 characters. Recovery also
redacts any unrecognized stored diagnostic before returning an attempt, keeping
signed URLs, credentials, request data, and customer content out of enumerable
recovery state.
Only the latest receipt for each surface and phase is retained, so repeated
commit or abort retries cannot grow an attempt row without bound.
Prepared surface names, surface receipts, and the remaining commit set are
persisted after every transition. Recovery skips already-prepared surfaces and
still aborts all durably prepared surfaces if a later prepare fails. The
surface returning a failed, skipped, thrown, or malformed prepare result is
also treated as possibly staged and receives idempotent cleanup before the
attempt can become aborted.
listPendingPublicationAttempts() returns at most 100 recoverable attempts by
default (configurable from 1 to 500) and accepts an { updatedAt, id } keyset
cursor for the next page. It enumerates live-surface attempts only for the
current canonical revision; a direct retry of a stale live-surface attempt marks
it superseded. A stale preparing or prepared attempt first aborts only its
durably prepared surfaces and never stages additional work after another
revision wins. Enumeration isolates malformed or oversized durable JSON as a redacted
recoveryError: 'invalid_durable_state' record so one corrupt attempt cannot
wedge recovery of healthy rows; retrying that row still fails strict decoding.
retryPublicationSurfaces() accepts only the durable
attemptId plus the currently pending reconcilers; operation, content, target
revision, previous revision, and package version are loaded from D1 rather than
trusted from a retry caller. Mid-abort and abort_failed attempts remain
enumerable with their incomplete prepared-surface set, so the same retry entry
point can finish idempotent cleanup and transition them to aborted.
Rollback targets must have durable attempt provenance proving that the revision previously became canonical. Revisions left behind by a failed prepare or a losing pointer CAS cannot be exposed through rollback.
let cursor
do {
const pending = await listPendingPublicationAttempts(env.SITE_DB, { cursor })
for (const attempt of pending) {
await retryPublicationSurfaces(env.SITE_DB, {
attemptId: attempt.id,
surfaces: attempt.pendingSurfaceNames.map(resolveSiteSurface),
})
}
const last = pending.at(-1)
cursor = last ? { updatedAt: last.updatedAt, id: last.id } : undefined
if (pending.length < 100) break
} while (cursor)importContentArchive imports archive fixtures or site-owned export bundles for
article, newsletter, and page bodies as drafts. Imported items are not published
and do not move the canonical publication pointer. Imports are bounded and
resumable: pass batchSize and the returned nextCursor to continue. Existing
namespaced sourceId mappings are checked before slugs and stored under a
unique index, making a retried or renamed-source batch idempotent. The importer
preserves rendered HTML, SEO, visibility, authorship, taxonomy, canonical/media
references, historical dates/timezone, and bounded declared JSON metadata. The
complete selected batch is validated before its first write.
maxArchiveItems, maxBatchItems, maxBodyBytes, and the package metadata
limit provide explicit DoS bounds.
An archive cursor greater than the bundle's item count is rejected as
archive_cursor_out_of_range; a cursor exactly at the item count is a valid
completed checkpoint.
Migrations 0020_content_metadata_read_model and
0021_published_revision_slug_index also enable explicit historical
publication times, indexed canonical-slug lookup, and the package-owned public read model. Pass publishedAt
to publishOne only for a staged archive item; it must be a non-negative epoch
second no later than now and is retained in the durable attempt across
retries. Omit it for normal publishing.
import {
getPublishedContentBySlug,
listPublishedContent,
publishOne,
} from '@growth-labs/cms'
await publishOne(env.SITE_DB, {
contentId: importedId,
now,
publishedAt: archivePublishedAt,
surfaces: [sitemapReconciler, feedReconciler, searchReconciler],
})
const article = await getPublishedContentBySlug(env.SITE_DB, slug)
const page = await listPublishedContent(env.SITE_DB, {
type: 'article',
channel: 'history',
limit: 50,
cursor,
})These APIs never render the mutable content_items draft as public content.
They resolve the immutable content_revisions payload selected by
published_revision_id, validate its ownership and shape, and return null
for an absent/non-live pointer. Corrupt, oversized, cross-owned, or ambiguous
state throws PublishedContentReadError and therefore fails closed. Listing is
keyset-only and capped at 100 rows, 4 MiB of immutable payloads, and 4 MiB per
payload. The package reads bounded payload outlines first, then fetches only the
selected revision pointers, so a caller cannot materialize 100 maximum-size
articles in one Worker request. Slug allocation also reserves slugs in
currently pointer-selected revisions, so an unpublished draft rename cannot
free a live URL for another item. Every call still receives the consumer site's
own injected D1 binding; the package creates no shared runtime or cross-site
read path.
A publication rollback changes the immutable revision pointer but deliberately
retains the current canonical publishedAt. A rollback is recovery, not a new
publication event, so it must not reorder feeds or keyset pages. Revision-owned
payload dates still roll back with the selected immutable payload.
The storage decision evidence for packages#229 lives in
benchmarks/PUBLICATION_STORAGE_PROTOCOL.md; raw benchmark distributions are
retained under benchmarks/results/ after the required runs. The benchmark uses
representative body/taxonomy distributions and records local Miniflare evidence
as local CPU/memory only; live D1/R2 latency remains a separate controlled
Worker measurement.
The source-only Worker harness for that separate controlled measurement lives in
benchmarks/LIVE_STORAGE_WORKER_PROTOCOL.md and
benchmarks/live-storage/. It is a disposable fixture only: no runtime package
export, no central fleet engine, no live Cloudflare resource creation, and no
customer content.
Reader state and privacy
Apply migration 0019_reader_state before adopting the reader-state API. The
consumer supplies a scope only after verifying both the realm user and site
token. identityUserId is the raw issuer UUID/ULID; prefixed subjects, email or
local IDs, and 64-hex analytics IDs are rejected. Content is always addressed by
its stable CMS ID plus content type, never by slug.
import {
clearReaderHistory,
exportReaderStatePage,
recordReaderOpen,
recordReaderProgress,
setManualReadState,
setSavedState,
} from '@growth-labs/cms'
const scope = { identityUserId: verifiedRawUserId, siteId: verifiedSiteId }
const content = { contentType: 'article', contentId: stableContentId }
await recordReaderOpen(env.SITE_DB, scope, content, {
behavioralConsent: consent.readerHistory ? 'granted' : 'denied',
})
await recordReaderProgress(env.SITE_DB, scope, content, {
behavioralConsent: consent.readerHistory ? 'granted' : 'denied',
progressBasisPoints: 9_000,
})
await setManualReadState(env.SITE_DB, scope, content, {
state: 'unread',
occurredAt: eventTime,
mutationId: idempotencyKey,
})
await setSavedState(env.SITE_DB, scope, content, {
saved: true,
occurredAt: eventTime,
mutationId: idempotencyKey,
})
const page = await exportReaderStatePage(env.SITE_DB, scope, { limit: 100 })
await clearReaderHistory(env.SITE_DB, scope)Progress is monotonic and reaches automatic completion at 90 percent. A manual
read/unread value overrides automatic completion until explicitly cleared.
Explicit mutations use event time plus mutationId for deterministic ordering
and may supply expectedVersion for compare-and-swap. Batch/list/export reads
are limited to 100 rows and remain inside one verified user/site scope. List
cursors order by last update; export cursors are a distinct opaque type ordered
by immutable row creation time so concurrent activity cannot move an existing
row across an export page boundary. Consent
withdrawal clears behavioral history while retaining explicit manual, saved,
and stated-preference choices; deleteReaderState erases the entire current
user/site scope. Network-wide erasure is intentionally orchestrated by invoking
that site-scoped operation under each separately verified site authority.
Rollback is additive and data-preserving: pin @growth-labs/[email protected] and stop
calling the reader-state surface, but leave cms_reader_state and migration
0019 in place. Re-adopting 0.3.0 resumes from the preserved rows; rollback
must never drop or reinterpret them through analytics identity.
Coming in later waves
- Broader admin UI polish and per-site extension points beyond the shipped Masthead screens.
