LinkLoom

Web scraping, Brave search, content extraction, and markdown conversion — in TypeScript/Bun. Pass a URL, get clean markdown. Render JS-heavy pages. Search the web with Brave. Extract links, tables, and embeddings.

Features

• URL to Markdown — pass any URL, get clean markdown back. Auto-detects HTML pages vs PDF files
• HTML to Markdown — extract readable content from raw HTML using Readability + Turndown
• PDF to Markdown — parse PDF buffers into structured markdown
• Headless browser rendering — render JavaScript-heavy pages via Camoufox (stealth Firefox/Playwright)
• iframe support — extract content from nested frames with configurable wait strategies
• Link extraction — pull and classify URLs from text or HTML (PDF vs Page)
• Web search — query Brave search results and return structured links
• Table extraction — scrape HTML tables from pages and convert to markdown tables
• Text embeddings — generate vectors via OpenAI or Gemini (optional, requires API key)
• CLI — use every feature from the command line
• MCP Server — expose all tools to Claude Desktop, Cursor, and other MCP clients

Tech Stack

Bun, Camoufox, JSDOM + Readability + Turndown, pdf.js-extract, LangChain (optional embeddings)

Install

bun add @boris.barac/linkloom

Camoufox Setup

Headless rendering uses Camoufox. After installing dependencies, set up the browser binary and GeoIP database:

bunx camoufox-js fetch

When developing from this repo, use bun run setup:browser. On macOS, that command also re-signs the downloaded app bundle so Camoufox can launch.

If Camoufox aborts with SIGABRT in the Codex macOS sandbox, re-run browser-backed verification outside the sandbox. In this repo, bun run setup:browser was also used to repair/re-sign the local Camoufox install before verification.

Use bun run browser:path to print the installed Camoufox path.

CLI Usage (via bunx)

bunx @boris.barac/linkloom scrape https://example.com
bunx @boris.barac/linkloom mcp

See the CLI Reference for all commands and flags.

Documentation

| Document | Description | |---|---| | Library API | Programmatic usage with code examples for every module | | CLI Reference | Command-line interface — all subcommands, flags, and examples | | MCP Server | Model Context Protocol server for AI clients (Claude, Cursor, etc.) |

Environment

cp env.example .env

| Variable | Required | Default | Description | |---|---|---|---| | PAGE_LOAD_TIMEOUT | No | 10000 | Browser page load timeout (ms) | | FRAME_TIMEOUT | No | 5000 | iframe load timeout (ms) | | PDF_DOWNLOAD_TIMEOUT | No | 30000 | PDF download timeout (ms) | | OPENAI_API_KEY | Yes* | — | Required for textToVector with OpenAI | | GEMINI_API_KEY | Yes* | — | Required for textToVector with Gemini |

* Only needed if you use the textToVector embedding feature. The core scraping and conversion pipeline works without any API keys.

Scripts (development)

| Command | Description | |---|---| | bun run start | Run the entry point | | bun run cli | Run the CLI | | bun run mcp | Run the MCP server | | bun run check | Format + lint (biome) | | bun run type_check | TypeScript type check | | bun run setup:browser | Download the Camoufox browser + GeoIP DB and re-sign the app bundle on macOS | | bun run browser:path | Print the local Camoufox install path | | bun test | Run tests |