dredge

Pull clean markdown out of any web page, from the terminal.

Paste a url, get the article — no nav, no ads, no cookie banners. Yaml frontmatter carries the title, source, author and date, so the files drop straight into a notes vault or a repo.

Install

npm install -g @quietengines/dredge

Without installing:

npx @quietengines/dredge <url>

The package lives under the Quiet Engines scope; the installed command is plain dredge.

Requires Node 18+.

Usage

$ dredge                                          # full-screen prompt
$ dredge https://example.com/article              # markdown to stdout
$ dredge https://example.com/article -o note.md   # write to a file
$ dredge url1 url2 url3 -d notes/                 # one file per url
$ cat urls.txt | dredge -                         # urls from stdin, one per line
$ dredge https://x.com/user/status/123 -c         # straight to the clipboard

Options:

-o, --output <file>  write to a file (single url)
-d, --dir <dir>      write one file per url into a directory
-c, --copy           copy the markdown to the clipboard
-b, --browser        force headless chrome rendering
--no-frontmatter     omit the yaml frontmatter block

Batch files are named from the article title: having-kids.md, not kids.html. Status messages go to stderr, markdown to stdout, so piping stays clean.

Bare dredge opens a full-screen terminal UI: paste a link (⇥ grabs one from the clipboard), then deliver the markdown to the clipboard or to a file in the current directory. ↵ submits, ↑↓ choose, esc goes back, ^c quits — and the mouse works too. ↑ at the prompt recalls previously dredged urls.

How it works

• Defuddle does the extraction — the same engine behind Obsidian Web Clipper. It strips boilerplate, keeps the article, and converts to markdown.
• Pages that need javascript are detected automatically: when a plain fetch yields almost no readable text, dredge re-renders the page in headless Chrome (found on your system, nothing bundled) and extracts again.
• No API keys, no services, nothing leaves your machine except the page request itself.

Development

npm install
npm test          # unit tests
npm run build     # bundle to dist/
node dist/cli.js <url>

License

MIT