httrackdoxy

Fix WinHTTrack issues and download missing JS files for Doxygen documentation.

Background

WinHTTrack is a tool for downloading websites to browse offline. However, when used with Doxygen-generated documentation sites, several issues arise that break the offline copy.

Problems

• Directory path rewriting — A directory URL (e.g. /doc/) and its default file URL (/doc/index.html) are two distinct server resources. When WinHTTrack crawls the directory URL, it rewrites directory paths to file paths for saving (search/ → search/index.html) and uses the saved file URL to replace the original directory URL. This breaks Doxygen's search system, which uses search/ as a prefix to construct JS file URLs. Additionally, the directory file and the actual index.html collide on the same filename — WinHTTrack renames the latter to index-2.html and rewrites all internal references accordingly (e.g. initNavTree('index.html', …) → initNavTree('index-2.html', …)), breaking navigation.

  Tip: Always specify the full file URL when crawling (e.g. /doc/index.html instead of /doc/).

• Character encoding — Special characters in filenames (like ~) may be altered during download.

• Missing auxiliary files — WinHTTrack may not download all JavaScript files that Doxygen needs, particularly:

  • navtreeindex{0..N}.js — navigation tree index shards
  • search/{prefix}_{hex}.js — search data shards partitioned by first character
  • JS files for in-page navtree data

How httrackdoxy helps

This tool addresses the above problems:

• Fix directory rewriting — Globally replaces WinHTTrack's path rewrites in all local HTML/JS files:

  • search/index.html → search/ (restores Doxygen search)
  • index-2.html → index.html (restores navigation references)

• Auto-detect renames via crawl comparison — Fetches the original crawl page from the server, compares its links with the local doxygen_crawl.html, and uses Levenshtein distance to find mangled filenames.

• Download missing JS files — Reconstructs the complete list of required JS files by parsing doxygen_crawl.html, navtreedata.js, and search/searchdata.js.

Usage

npx httrackdoxy <command> [options]

Commands

| Command | Description | | ---------- | ----------------------------------------------------- | | download | Download missing JS files from the documentation site | | fixpath | Fix WinHTTrack path rewriting issues |

Download Options

| Flag | Description | | --------------------- | --------------------------------------------------------------------------------------- | | -f, --force | Overwrite existing local files | | -c, --concurrency N | Max concurrent downloads (default: 5) | | -l, --limit N | Limit the number of files to process | | -n, --dry-run | Only show what would be downloaded | | --check | Send HEAD requests to verify remote file existence | | -g, --glob PATTERN | Filter file names by glob pattern(s), space-separated (e.g. "search/* navtreeindex*") | | --detect-renames | Detect mangled HTML filenames in crawl file by comparing with server version |

General Options

| Flag | Description | | --------------- | ---------------------------------------------- | | -d, --dir DIR | Working directory (default: current directory) | | -h, --help | Show help message and exit |

Example

# Fix path rewriting issues in the current directory
npx httrackdoxy fixpath

# Download missing JS files with rename detection
npx httrackdoxy download --detect-renames

# Dry run: see what would be downloaded, limited to search files
npx httrackdoxy download -n -g "search/*"

Tested With

• WinHTTrack 3.50
• Doxygen 1.16.1

License

MIT