chidori ⚡️

A fast Rust-built web-to-Markdown fetcher for AI agents.

npx chidori-fetch https://example.com

chidori fetches a URL, extracts the main readable content, removes page noise, and prints Markdown to stdout by default. Logs and errors go to stderr so AI agents and shell scripts can safely pipe stdout into prompts, files, or other tools.

Quick demo

Web pages are built for browsers. Agents usually need the readable content, not the page chrome around it.

npx chidori-fetch https://taishikato.com

![profile image](https://taishikato.com/_next/static/media/profile.0cb7aa44.webp)

I'm Taishi, a fullstack developer and solofounder based in Japan and Canada.

Full-stack dev shipping AI infrastructure & tools to 18K+ users (Supabase hackathon winner). Two of them were ranked as the [#1 Product of the day on Product Hunt](https://www.producthunt.com/products/supavec#supavec).

==========

[Hire me](/cdn-cgi/l/email-protection#493d28203a212067227970797a092e24282025672a2624)

[My Work](/)[Stack](/stack)[Blog](/blog)

[![Supavec](https://taishikato.com/supavec.png)](https://www.supavec.com)

### Supavec ###

OSS RAG as a Service - spin up vector search + chat API in \<5 min. 630▲ Product Hunt, 620⭐ GitHub.

[supavec.com →](https://www.supavec.com)

## AI Tooling & Infrastructure

### [Supavec](https://www.supavec.com) ###

\#RAG#Supabase#OpenSource

Open-source RAG platform using pgvector.

### [MCP Server](https://www.supavec.com/blog/supavec-mcp-server) ###

\#MCP#RAG#OpenSource

MCP server streaming Supabase-secured RAG data to LLMs.

### [CiteAnalytics](https://x.com/supabase/status/1909552555333398630) ###

\#OpenSource#Hackathon

Supabase hackathon winning AI citation analytics app.

## Work Experience

<img alt="Supabase" class="rounded-full w-11 grayscale" height="44" loading="lazy" src="https://taishikato.com/_next/static/media/supabase.92465ed1.webp" width="44">

Supabase

Frontend Developer (contract)Remote

Here are [my PRs](https://github.com/supabase/supabase/issues?q=author%3Ataishikato)

Jul 2025 - Sep 2025

<img alt="Whisperit" class="rounded-full w-11 grayscale" height="44" loading="lazy" src="https://taishikato.com/_next/static/media/whisperit.302305fe.webp" width="44">

Whisperit

Full Stack Developer (freelance)Switzerland (Remote)

Aug 2024 - Jan 2025

<img alt="Semios" class="rounded-full w-11 grayscale" height="44" loading="lazy" src="https://taishikato.com/_next/static/media/semios.1e5ff783.webp" width="44">

Semios

Full Stack JS Developer (full-time)Vancouver

2020 - 2023

<img alt="Yahoo! Japan" class="rounded-full w-11 grayscale" height="44" loading="lazy" src="https://taishikato.com/_next/static/media/yahoo.2fd5bb07.webp" width="44">

Yahoo! Japan

Software Developer (full-time)Tokyo

2015 - 2019

<img alt="btrax" class="rounded-full w-11 grayscale" height="44" loading="lazy" src="https://taishikato.com/_next/static/media/btrax.204b56b0.webp" width="44">

btrax

Software Developer (intern)San Francisco

Apr 2013 - Jun 2013

## Hackathon Wins

[Supabase Hackathon for Launch Week 14 Winner Apr 2025](https://x.com/supabase/status/1909552555333398630)

[Supabase Hackathon for Launch Week 6 Runner Up - Best Storage Project Jan 2023](https://supabase.com/blog/launch-week-6-hackathon-winners#runner-up-2)

[Supabase Hackathon for Launch Week 5 Winner - Most Visually Pleasing Aug 2022](https://supabase.com/blog/launch-week-5-hackathon-winners#winner-1)

## OSS Contributions

[Pull Request #79 · supabase/embeddings-generator](https://github.com/supabase/embeddings-generator/pull/79)

[Pull Request #137 · langchain-ai/langchain-community](https://github.com/langchain-ai/langchain-community/pull/137)

[Pull Request #30 · ueberdosis/tiptap-docs](https://github.com/ueberdosis/tiptap-docs/pull/30)

[Pull Request #60822 · vercel/next.js](https://github.com/vercel/next.js/pull/60822)

[Pull Request #19009 · supabase/supabase](https://github.com/supabase/supabase/pull/19009)

[Pull Request #340 · modal-labs/modal-examples](https://github.com/modal-labs/modal-examples/pull/340)

[Pull Request #1799 · nuxt/docs](https://github.com/nuxt/docs/pull/1799)

[Pull Request #1797 · nuxt/docs](https://github.com/nuxt/docs/pull/1797)

[Pull Request #24589 · nodejs/node](https://github.com/nodejs/node/pull/24589)

That output can go straight into an agent prompt, a RAG ingestion job, a note, or another shell command:

chidori https://example.com/article | llm "summarize this for a code review"
chidori https://example.com/article --json | jq -r '.title, .markdown'

Why agents use it

• Pipeable by default: Markdown goes to stdout. Logs and errors go to stderr, so scripts can trust the output stream.
• Readable content first: chidori strips navigation, footers, forms, hidden content, related links, script noise, and common social/page chrome.
• Agent-friendly Markdown: code blocks, links, images, simple tables, footnotes, math, and callouts are preserved in text form where possible.
• Useful metadata: --json returns the final URL, canonical URL, title, description, domain, language, meta tags, schema.org data, word count, and the extracted Markdown.
• Fast local CLI: it is a Rust binary with no hosted service in the loop. JavaScript-heavy pages can opt into your own renderer with --render=auto.

How it is different

| Tool | What you usually get | | --- | --- | | curl | Raw HTML, scripts, navigation, and layout markup. | | Browser automation | Accurate rendering, but heavier and harder to pipe through shell workflows. | | Readability libraries | Good extraction primitives, usually embedded inside another app or runtime. | | chidori | One command that fetches, extracts, cleans, converts to Markdown, and keeps stdout safe for agents. |

Installation

npm install -g chidori-fetch

Usage

chidori --help
chidori https://example.com
chidori https://example.com --json
chidori https://example.com --output article.md
chidori https://example.com --max-chars 20000
chidori https://example.com --lang ja
chidori https://example.com --no-images
chidori https://example.com --debug
CHIDORI_RENDER_COMMAND=/path/to/render-page chidori https://example.com --render=auto

--json prints metadata plus markdown as JSON. Metadata includes url, finalUrl, canonicalUrl, domain, title/description fields, metaTags, schemaOrgData, and wordCount. With --json --debug, stdout also includes a debug object; human-readable debug lines still go to stderr.

--render=auto is an optional fallback for JavaScript-heavy pages. When static extraction fails or only finds a short app shell, chidori runs CHIDORI_RENDER_COMMAND <url> if configured and expects rendered HTML on stdout. If the renderer is unavailable and no custom user agent was supplied, chidori still tries its bot user-agent fallback. Renderer output uses the same timeout and 5 MiB fetch limit.

External renderer example

One simple renderer setup is a small Playwright script:

npm install -D playwright
npx playwright install chromium

Create render-page.mjs:

#!/usr/bin/env node
import { chromium } from "playwright";

const url = process.argv[2];

if (!url) {
  console.error("Usage: render-page.mjs <url>");
  process.exit(2);
}

const browser = await chromium.launch({ headless: true });

try {
  const page = await browser.newPage();
  await page.goto(url, {
    waitUntil: "networkidle",
    timeout: 10_000,
  });

  process.stdout.write(await page.content());
} finally {
  await browser.close();
}

Then make it executable and pass it to chidori:

chmod +x render-page.mjs
CHIDORI_RENDER_COMMAND=/absolute/path/to/render-page.mjs chidori https://example.com --render=auto

The renderer should write only HTML to stdout. Write logs and errors to stderr so chidori can safely read stdout as the rendered document.

Design goals

AI agents need web pages as clean, pipeable Markdown. chidori is built for fast CLI startup, deterministic shell behavior, and extraction that is easy to debug when a page does something strange.