@dogsbay/format-fluidtopics-rest

Importer for a live Fluid Topics portal via the public khub REST API (Option B of the FT→Dogsbay migration). It reverse-engineers each topic's DITA-OT HTML5 — where the DITA element vocabulary survives in class names (conbody/taskbody/refbody, shortdesc, section, CALS tables, span.ph, …) — into Dogsbay's universal TreeNode model, producing ExportPage[] + NavItem[].

Works on any public FT portal. Gated portals (RBAC) throw KhubAuthError.

The sibling @dogsbay/format-fluidtopics-markdown does the same job from a Git Markdown export (Option A).

Usage

import { importFluidTopicsRest } from "@dogsbay/format-fluidtopics-rest";

const { pages, nav } = await importFluidTopicsRest({
  portalBase: "https://www.servicenow.com/docs", // or .../bas, docs.fluidtopics.com
  titleFilter: "Customer Relationship Management", // or maps: ["<mapId>"]
  hrefPrefix: "/docs",
});

Or the low-level converter / client:

import { htmlToTree, KhubClient } from "@dogsbay/format-fluidtopics-rest";
const { tree, title, topicType } = htmlToTree(htmlString); // topicType: concept|task|reference|topic

CLI: dogsbay migrate-fluidtopics-rest <portal-url> [--map <id>] [--title-filter <s>] [--single-page] [-o <dir>].

Single-page (whole-book) mode

Pass --single-page / singlePage: true to roll each map into one page (the whole book) instead of one page per topic — for readers who prefer a single long page over "bitty" topics, and to match FT portals that serve a book as one page. TOC nodes become nested headings (level = TOC depth), internal xrefs become in-page anchors, and the page lands at the book's common path prefix (e.g. …/metasys-ui-and-jct-technical-bulletin/16.0).

What it does

• khub harvest — maps → maps/{id}/toc → maps/{id}/topics/{cid}/content; path-scoped bases (…/bas/api/khub); polite throttling; 401/403 → KhubAuthError.
• HTML5 → TreeNode — topic type from the body class; DITA sections flattened to headings; CALS tables → structured table nodes; span.ph/keyword/uicontrol unwrapped, codeph → inline code; FT chrome (zDocs*, ai-summary, sentence, css-…, toggle/summary/tooltip) stripped.
• Frontmatter — topic_type → type, plus lang/version from the map metadata.
• Nav — from each map's TOC (the authoritative DITA map), with route slugs derived from prettyUrl.

Rendering fidelity

Tuned against the live FT portals; each item is regression-tested. Full story in docs-dev/fluidtopics-rest-importer.md.

• Internal xrefs — FT renders them as hrefless <span data-tocid data-mapid> (resolved by its client at runtime, zero hrefs in the static HTML). The importer builds a tocId → slug index from the selected maps' TOCs and resolves them to real routes, falling back to plain text (not a dead link) when out of set.
• Figures / inline images — block <img> survive; base64 data: images are extracted to real content/_assets/… files (filename from FT metadata).
• Lists & inline runs — consecutive inline content is coalesced into one paragraph, so xrefs inside list items resolve and prose doesn't fragment.
• Tables — CALS → structured tables; cells with block content (lists, paragraphs) are preserved (HTML tables per the Dogsbay-MD convention).
• Definition lists — <dl>/<dt>/<dd> → a real deflist (dt.inline term + dd block children), not literal : text.

Verified (live, 2026-06)

ServiceNow www.servicenow.com/docs (CRM, 24 topics), Johnson Controls docs.johnsoncontrols.com/bas (Metasys bulletin, 55), Fluid Topics docs.fluidtopics.com (13). Notably, REST succeeds on ServiceNow's CRM publication where the Markdown export (Option A) failed (its index.md TOC was stale: 1/24 targets existed) — the live khub TOC+content is authoritative.

Known gaps

• Nav-card landing grids come through as a generic table (Option A special-cases them into :::cards) — a nav-card-detection pass is future work.
• Cross-map xrefs resolve only when the target map is in the import set (single---map runs leave them as text).
• External image URLs are kept as-is (not downloaded); only inline data: images are localized to _assets.
• conref/keyref/profiling were already resolved/flattened upstream by FT's DITA-OT run (not recoverable, and not needed for delivery).