LinkLoom

Web scraping, Brave search, content extraction, and markdown conversion — in TypeScript. Pass a URL, get clean markdown. Render JS-heavy pages. Search the web with Brave. Extract links, tables, and embeddings.

Runs on both Bun (primary) and Node.js ≥ 20.

Requirements

• Bun ≥ 1 or Node.js ≥ 20 — pick either runtime
• Headless rendering additionally requires Camoufox (fetches its own browser binary)

Features

• URL to Markdown — pass any URL, get clean markdown back. Auto-detects HTML pages vs PDF files
• HTML to Markdown — extract readable content from raw HTML using Readability + Turndown
• PDF to Markdown — parse PDF buffers into structured markdown
• Headless browser rendering — render JavaScript-heavy pages via Camoufox (stealth Firefox/Playwright)
• iframe support — extract content from nested frames with configurable wait strategies
• Link extraction — pull and classify URLs from text or HTML (PDF vs Page)
• Web search — query Brave search results and return structured links
• Table extraction — scrape HTML tables from pages and convert to markdown tables
• Text embeddings — generate vectors via OpenAI or Gemini (optional, requires API key)
• CLI — use every feature from the command line
• MCP Server — expose all tools to Claude Desktop, Cursor, and other MCP clients

Tech Stack

Bun or Node.js, Camoufox, JSDOM + Readability + Turndown, pdf.js-extract, LangChain (optional embeddings). Tests run on Vitest under either runtime.

Install

bun add @boris.barac/linkloom
# or
npm install @boris.barac/linkloom

Quick start

CLI (no install)

bunx @boris.barac/linkloom scrape https://example.com        # Bun
npx @boris.barac/linkloom scrape https://example.com         # Node.js

Library

import { convertLinkToMarkdown } from "@boris.barac/linkloom";

const markdown = await convertLinkToMarkdown("https://example.com");
console.log(markdown);

MCP server

bunx @boris.barac/linkloom mcp        # Bun
npx @boris.barac/linkloom mcp         # Node.js

See the MCP Server docs for Claude Desktop / Cursor client config.

Runtime support

The package ships pre-built ESM (dist/) that runs on both Bun and Node.js ≥ 20 — no bundler or TypeScript compiler required at install time.

From source, Bun executes .ts directly; Node uses tsx (installed as a dev dependency):

bun run src/cli.ts scrape https://example.com        # Bun (direct)
node --import tsx src/cli.ts scrape https://example.com  # Node

Camoufox Setup

Headless rendering uses Camoufox. After installing dependencies, set up the browser binary and GeoIP database:

bunx camoufox-js fetch

When developing from this repo, use bun run setup:browser. On macOS, that command also re-signs the downloaded app bundle so Camoufox can launch.

If Camoufox aborts with SIGABRT in the Codex macOS sandbox, re-run browser-backed verification outside the sandbox. In this repo, bun run setup:browser was also used to repair/re-sign the local Camoufox install before verification.

Use bun run browser:path to print the installed Camoufox path.

CLI Usage

The CLI exposes every feature as a subcommand. See the CLI Reference for all commands and flags (scrape, html, pdf, render, links, search, tables, mcp).

Documentation

| Document | Description | |---|---| | Library API | Programmatic usage with code examples for every module | | CLI Reference | Command-line interface — all subcommands, flags, and examples | | MCP Server | Model Context Protocol server for AI clients (Claude, Cursor, etc.) |

Environment

cp env.example .env

| Variable | Required | Default | Description | |---|---|---|---| | PAGE_LOAD_TIMEOUT | No | 10000 | Browser page load timeout (ms) | | FRAME_TIMEOUT | No | 5000 | iframe load timeout (ms) | | PDF_DOWNLOAD_TIMEOUT | No | 30000 | PDF download timeout (ms) | | PROXY_URL | No | — | Proxy endpoint (http://user:pass@host:port or socks5://host:port) | | OPENAI_API_KEY | Yes* | — | Required for textToVector with OpenAI | | GEMINI_API_KEY | Yes* | — | Required for textToVector with Gemini | | LINKLOOM_DEBUG | No | — | Set to 1 (or DEBUG=linkloom) to log non-fatal cleanup failures |

* Only needed if you use the textToVector embedding feature. The core scraping and conversion pipeline works without any API keys.

Scripts (development)

Bun is the primary dev driver; equivalent Node variants are provided where useful.

| Command | Description | |---|---| | bun run build | Build ESM to dist/ (tsup) — required before publishing/npx | | bun run start | Run the entry point (Bun) | | bun run start:node | Run the entry point (Node via tsx) | | bun run cli / cli:node | Run the CLI | | bun run mcp / mcp:node | Run the MCP server | | bun run check | Format + lint (biome) | | bun run type_check | TypeScript type check | | bun run setup:browser | Download the Camoufox browser + GeoIP DB and re-sign the app bundle on macOS | | bun run browser:path | Print the local Camoufox install path | | bun run test | Run tests (vitest) | | bun run test:watch | Run tests in watch mode |

Publishing

The published artifact is the pre-built ESM in dist/ (built by tsup). npm publish automatically runs the build via the prepublishOnly hook — no Bun required to publish.

npm publish --access public

License

MIT © Boris Barac