Waterfall Tools

Waterfall Tools is a fast, zero-bloat library for parsing, analyzing, and visualizing network waterfalls in the browser or Node.js. It normalizes a wide range of network trace formats — PCAP, Netlog, Chrome Trace, Perfetto, CDP, WebPageTest JSON, HAR, rumcap .rcap field captures, qlog QUIC/HTTP-3 logs — into a single Extended HAR intermediate, then renders them via <canvas> in WebPageTest style without building thousands of DOM nodes.

Features

• Format agnostic. Parses HAR, Netlog, Chrome Trace, Perfetto protobuf, CDP, WebPageTest JSON, raw TCPDUMP captures (with automatic TLS/QUIC decryption, bandwidth estimation, per-chunk download timing, and HTTP/2 & HTTP/3 priority extraction), rumcap .rcap field captures (RUM beacons: navigation/resource timing, paints/LCP/CLS/INP, long tasks, LoAF, User Timing, JS Self-Profiling — .rcap inputs normalize duplicate User Timing marks and also light up embedded DevTools via synthesized Chrome trace JSON and Perfetto via native TrackEvent protobuf), and — in beta — qlog QUIC/HTTP-3 structured logs (.qlog plain JSON and .sqlog JSON-SEQ; one file per QUIC connection, with multi-file merging into a single waterfall — see qlog captures below).
• Unified API. WaterfallTools auto-detects the input format and produces a consistent Extended HAR payload regardless of source.
• Isomorphic. The core runs in Node.js and modern browsers with no polyfills — binary and cryptographic operations use Uint8Array, DataView, WebCrypto, and DecompressionStream.
• Canvas renderer. Scales cleanly from 50 to 50,000 requests without DOM thrashing.
• Offline (PWA). The standalone viewer registers a Service Worker and works offline.
• Response body inspection. When bodies are available (Netlog decoded bytes, wptagent nested _bodies.zip, tcpdump captures, or standard HAR response.content.text), the viewer syntax-highlights text formats (HTML, CSS, JS, JSON, XML) and renders images inline. For tcpdump imports, gzip, deflate, br, and zstd content-encoded bodies are automatically decompressed via native DecompressionStream with pure-JS fallbacks (brotli, fzstd) where the native stream isn't available.
• Optional CORS fetch proxy. A single-file Cloudflare Worker provides a CORS-safe fallback for URL imports. It proxies only recognised waterfall-tools formats, forwards the caller's IP upstream (non-anonymizing), blocks obvious SSRF targets, and applies a per-IP failure rate limit. No bindings — paste cloudflare-worker/worker.js into the Cloudflare dashboard.

For the design, module layout, and conventions, see Docs/Architecture.md.

Installation

npm install waterfall-tools

To host the bundled viewer (see Hosting the viewer) also install the Chrome DevTools frontend as a peer dependency. Install the optional qvis peer as well when you want the qlog visualization tab:

npm install waterfall-tools @chrome-devtools/index
# Optional, enables the qvis tab for qlog inputs once the qvis package is published:
npm install @pmeenan/qvis

The DevTools bundle is ~80 MB of prebuilt third-party code kept out of the waterfall-tools tarball on purpose, and qvis is also opt-in. Both peers are declared optional, so npm install waterfall-tools alone completes without errors or warnings. In this source checkout, qvis is temporarily wired as a local file:third_party/qvis/visualizations/qvis-0.1.0.tgz placeholder until Pat publishes @pmeenan/[email protected].

API usage

The library is pure ESM. Your project needs "type": "module" or an ESM-aware bundler. The single entry point is the WaterfallTools class.

[!NOTE] Optional asset bundles. The build emits tcpdump-[hash].js (packet capture pipeline) and decompress-[hash].js (tcpdump Brotli/zstd fallbacks) as separate dynamically-loaded chunks alongside waterfall-[hash].js and its proxy stub waterfall-tools.es.js. For a minimal integration you can host just the core payload and stub; the library falls back gracefully if the supplement chunks aren't reachable.

Process a local file (Node.js)

import { WaterfallTools } from 'waterfall-tools';

const wt = new WaterfallTools();
await wt.loadFile('./trace.cap.gz');  // format auto-detected
const har = wt.getHar();

console.log(`HAR has ${har.log.entries.length} requests`);

Process a stream (browser or Node.js)

loadStream requires options.format — it cannot sniff a raw stream without buffering.

import { WaterfallTools } from 'waterfall-tools';

const wt = new WaterfallTools();
await wt.loadStream(fileStream, {
    format: 'tcpdump',
    isGz: true,
    keyLogInput: keyLogStream  // optional TLS keylog for pcap decryption
});
const har = wt.getHar();

Process an in-memory buffer (browser or Node.js)

Use when you already have the full file as Buffer, ArrayBuffer, or Uint8Array. Format is sniffed from the bytes.

const wt = new WaterfallTools();
await wt.loadBuffer(await uploadedFile.arrayBuffer());

Merge multiple buffers into one page

loadBuffers(buffers, options) accepts an array of in-memory buffers and merges them into a single waterfall page. Every member is format-sniffed independently — the load fails loudly if the members don't all share one format, or if the shared format doesn't support multi-input merging (today only qlog, where each file carries one QUIC connection and a page load produces one file per origin). A single-member array behaves exactly like loadBuffer(). For qlog multi-file loads, pass options.bufferNames alongside buffers to preserve original filenames for later resource consumers.

const wt = new WaterfallTools();
await wt.loadBuffers(await Promise.all(files.map(f => f.arrayBuffer())), {
    bufferNames: files.map(f => f.name)
});
const har = wt.getHar();   // one page, all connections merged

qlog captures (beta)

qlog is the IETF structured logging format for QUIC/HTTP-3 (still an Internet-Draft; support here is beta — validated against curl/ngtcp2 and aioquic client captures plus Cloudflare quiche spec-final captures from both the client and server vantage points). Both serializations are auto-detected from content: plain JSON .qlog and JSON-SEQ .sqlog (RFC 7464), plain or gzipped, across both the draft-0.3 and final-spec dialects.

One qlog file describes one QUIC connection, so a full page load usually produces several files (one per origin). Drop them all on the standalone viewer at once, pass them together to loadBuffers(), or list them all on the CLI with --output, and they merge into a single page-wide waterfall. Fidelity depends on the producer: logs that include HTTP/3 events (e.g. aioquic/quiche) get real URLs, methods, statuses, headers, MIME types, RFC 9218 priority signals, and interim-response timing when present; transport-only logs (e.g. curl/ngtcp2, where headers stay QPACK-encoded on the wire) still render every request stream with synthetic per-stream URLs and assumed statuses. For producers that log HTTP/3 payload movement via quic:stream_data_moved instead of packet STREAM frame details, waterfall-tools uses that as a byte/chunk fallback, preferring transport/network movement times when available.

Connection-level details are also extracted when the producer logs them and attached netlog-style (visible in the viewer's request details / raw data): negotiated TLS cipher and ALPN, chosen QUIC version, the QUIC transport parameters of both sides, close initiator/error/reason, endpoint IP addresses and ports (from qlog connectivity events, when present), and a per-connection congestion summary (RTT, peak PTO count, packet-loss counters) alongside RTT / congestion-window / bytes-in-flight sample series on the page object.

Generating captures:

# curl (ngtcp2 backend, HTTP/3-enabled build): one .sqlog per connection in QLOGDIR
QLOGDIR=/tmp/qlogs curl --http3-only https://cloudflare-quic.com/

# aioquic: pass --quic-log to the example HTTP/3 client
python3 examples/http3_client.py --quic-log /tmp/qlogs https://example.com/

# quiche (spec-final qlog; QLOGDIR works on both example apps, so a local
# server + client run logs the same exchange from BOTH vantage points)
QLOGDIR=/tmp/qlogs ./quiche-server --listen 127.0.0.1:4433 --root ./htdocs \
  --cert cert.crt --key cert.key
QLOGDIR=/tmp/qlogs ./quiche-client --no-verify https://127.0.0.1:4433/

See Sample/Data/qlog/README.md for sample provenance and full regeneration recipes.

Progress tracking

Pass onProgress to keep the UI responsive on large files:

await wt.loadBuffer(buffer, {
    onProgress: (phase, percent) => {
        console.log(`${phase} — ${percent}%`);
    }
});

phase is a human-readable stage label ("Reading packets...", "Decrypting TLS...", "Building waterfall...", etc.). percent is 0–100. The tcpdump parser reports five distinct phases; other parsers report byte-consumption progress through their single streaming pass.

Load from a URL (browser or Node.js)

const wt = new WaterfallTools();
await wt.loadUrl('https://example.com/trace.json.gz');

Retrieve extracted assets (screenshots, traces, netlogs, qlog files)

Returns an isomorphic handle — a Blob URL in the browser ({url, mimeType}), a raw byte buffer in Node ({buffer}). Assets are extracted on demand from the parsed OPFS-backed archive rather than inflating everything into memory up front. qlog is the exception: getPageResource(pageId, 'qlog') returns raw retained input buffers directly as {files:[{name, mimeType, buffer}]} on both browser and Node paths, with no object URLs.

const resource = await wt.getPageResource('page_1_0_1', 'screenshot');
// resourceType: 'screenshot' | 'trace' | 'perfetto-trace' | 'netlog' | 'qlog' | 'lighthouse' | ...
if (resource?.url) document.querySelector('img').src = resource.url;
if (resource?.buffer) fs.writeFileSync('screen.jpg', resource.buffer);
if (resource?.files) resource.files.forEach(file => fs.writeFileSync(file.name, new Uint8Array(file.buffer)));

When you are finished with a WaterfallTools instance, call await wt.destroy(). This releases staged OPFS / temporary-file storage used by archive-backed resources such as wptagent screenshots, traces, and netlogs; in Node it also closes any lazy file handles opened by getPageResource().

Render to a container (browser)

import { WaterfallTools } from 'waterfall-tools';

const wt = new WaterfallTools();
await wt.loadUrl('https://example.com/trace.json.gz');

// Start from the canonical defaults; override what you need.
const options = {
    ...WaterfallTools.getDefaultOptions(),
    minWidth: 800
};

await wt.renderTo(document.getElementById('waterfall-container'), options);

WaterfallTools.getDefaultOptions() returns the canonical render configuration:

{
    pageId: null,            // specific page to render (defaults to the first page)
    connectionView: false,   // render per-connection rather than per-request
    thumbnailView: false,    // minimal thumbnail rendering
    minWidth: 0,             // minimum canvas width in px
    startTime: null,         // clip the view; seconds
    endTime: null,           // clip the view; seconds
    reqFilter: '',           // filter by request id substring
    showPageMetrics: true,   // vertical page metric lines (LCP, TTI, etc.)
    showMarks: false,        // user timing marks
    showCpu: true,           // CPU utilization graph
    showBw: true,            // bandwidth graph
    showMainthread: true,    // main thread flame chart (wptagent, chrome-trace) or activity blocks
    showLongtasks: true,     // long task warnings
    showMissing: false,      // ellipses for missing requests
    showLabels: true,        // row text labels
    showChunks: true,        // per-chunk download blocks
    showJsTiming: true,      // JS execution highlights
    showWait: true,          // TTFB / wait blocks
    showLegend: true,        // bottom legend
    rowHeight: null,         // null → 18 (or 4 in thumbnail view); override with any positive integer
    backgroundColor: null,   // null → '#ffffff'; any CSS color string
    palette: {}              // chrome color overrides; see "Theming" below
}

Theming

rowHeight, backgroundColor, and palette are inert when left at their defaults — the renderer reproduces the historical visual byte-for-byte. Override individual keys to retheme without forking. palette accepts:

• rowStripe (default '#f0f0f0') — alternating row background.
• border (default '#000000') — request-area outer border, label divider, and the chart-frame strokes for CPU / BW / Long Tasks bands.
• grid (default 'rgb(192,192,192)') — vertical time-grid lines.
• thumbnailGrid (default 'rgb(208,208,208)'), thumbnailBorder (default falls back to palette.grid) — thumbnail-view variants.
• longTask (default 'rgb(255, 82, 62)') — fill for blocked spans in the Long Tasks band.
• userTimingMark (default 'rgb(105, 0, 158)') — vertical-line stroke for _userTimes / _userTimingMeasures / _user_timing marks.
• text (default '#000') — primary text color (request URL labels, time-scale labels, bottom legend item text).
• titleText (defaults to palette.text, then '#333') — chart-frame titles (CPU Utilization / BW / Long Tasks). Set independently when the chart titles need a softer tone than the main text; otherwise they follow palette.text automatically.

Any CSS color string works. MIME-type colors and page-event metric colors (FCP, LCP, etc.) are not yet themable via palette — those keyed maps remain hard-coded and may open up in a follow-up. When a response MIME type is unavailable, the renderer falls back to well-known URL extensions (.js, .css, common images, and common font extensions) before using the generic gray style.

renderTo also accepts extra options not returned by getDefaultOptions():

• thumbMaxReqs (default 100) — max requests drawn in thumbnailView. 0 disables truncation.
• labelsCanvas — separate canvas for URL labels when you want to split them out.
• overlapLabels — draw request rows full-width, ignoring the label gutter.
• onHover(req, metrics) / onClick(req) — interaction callbacks. The first onHover argument stays backward-compatible with the request payload shape ({index, request, event} or null). The optional second argument is populated when the cursor is within roughly 5 CSS px of visible page-metric or user-timing vertical lines: {event, pageMetrics, userTiming}. Metric arrays are only populated when the corresponding display option (showPageMetrics, showMarks) is enabled.
• onZoom({startTime, endTime}) — fired whenever a user gesture changes the visible time window: a mouse drag-selection zoom or a touch pinch/pan. The renderer updates its own startTime/endTime options before firing, so the callback is a notification (sync UI, persist state), not a request for action.

Zooming

The rendered waterfall supports two zoom gestures out of the box:

• Mouse drag-to-zoom — press the left button inside the data area and drag horizontally; a translucent highlight tracks the selection across the full height of the waterfall, and releasing sets startTime/endTime to the selected range. Releasing outside the canvas (or pressing Escape mid-drag) cancels. Drags shorter than 5 px are treated as clicks, so onClick request selection still works, and the gesture is deliberately mouse-only — touch scrolling can't trigger it.
• Touch pinch/pan — two-finger pinch zooms around the gesture focal point; one-finger horizontal pan scrolls a zoomed view.

Both gestures re-render via updateOptions() and fire onZoom. To reset, call updateOptions({ startTime: null, endTime: null }) (the standalone viewer exposes this as a Reset button next to Options whenever the view is zoomed).

Embedding

renderTo(container, options) is the supported embedding API; it replaces the earlier div-embed.js bootstrap. Pass a container and your options (including interaction callbacks) and drop the waterfall into any page.

Hosting the viewer

A one-shot command materializes the viewer into a directory of your choice (typically your project's static web root):

npm install waterfall-tools @chrome-devtools/index
# Optional, enables the qvis tab for qlog inputs once the qvis package is published:
npm install @pmeenan/qvis
npx waterfall-tools install-viewer ./public/waterfall

This copies the viewer's static assets out of node_modules/waterfall-tools/dist/browser/ into the target directory, then copies node_modules/@chrome-devtools/index/ into ./public/waterfall/devtools-<version>/, patches the DevTools bundle for browser hosting, and rewrites the viewer's <meta name="waterfall-devtools-path"> to point at the versioned directory. If qvis is installed, it also copies node_modules/qvis/dist/ into ./public/waterfall/qvis-<version>/ and rewrites <meta name="waterfall-qvis-path">; if qvis is absent, that meta is emptied and the qvis tab stays hidden. Serve the target directory with any static file server.

Re-running the command updates the viewer in place and removes any stale devtools-<prev-version>/ and qvis-<prev-version>/ directories left over from earlier installs.

Standalone viewer

The library ships with a pre-built standalone viewer — deployable as a static page, embeddable in an iframe, or usable as a full-page tool without writing your own UI.

Loading a HAR with multiple runs (e.g. WebPageTest First View + Repeat View) presents an interactive Thumbnail Grid showing each run's paint metrics, load times, and request counts before drilling into a specific trace.

The viewer integrates tab-switching to self-hosted copies of the Perfetto Trace Viewer, the Chrome DevTools frontend, the legacy Chrome NetLog Viewer, and optionally qvis for qlog inputs. The DevTools frontend is pulled in from the @chrome-devtools/index npm package at build time and copied under dist/browser/devtools-<version>/ so it's served versioned alongside the viewer; qvis follows the same pattern into dist/browser/qvis-<version>/, preferring a freshly built third_party/qvis/visualizations/dist/ in this source checkout and falling back to node_modules/qvis/dist/ when the optional package is present. For rumcap captures, Perfetto receives a native TrackEvent protobuf (getPageResource(pageId, 'perfetto-trace')) with spec-aligned timestamps and a top-level Performance Profile custom track group whose populated child tracks are explicitly ordered for review, including request tracks split into request.* ResourceTiming phase categories when available, single-track Long Animation Frames details, one visual User Timing track with marks and merge-laned duration measures, one visual Interactions track with merge-laned event-type slices, plus plain Long Tasks and JS Self-Profiling tracks. DevTools receives gzipped Chrome trace JSON ('trace') with hidden request parser markers that feed its native Network-duration row, User Timing markers fed from a hidden parser process so they render in Timings without cluttering Main, visible LoAF script/render/style details, and JS Self-Profiling data as native Profile / ProfileChunk CPU-profile events with stable nonzero script ids for URL-backed frames plus (anonymous) fallback nodes for frame-less slices. For qlog pages, the qvis tab sends the retained raw qlog buffers through an embedded postMessage loader after inflating outer gzip members and normalizing filenames, so multi-connection page drops appear as loaded qvis connections without any server backend; embedded qvis starts on the Events view (a netlog-style raw event browser with a draggable split pane) and hides standalone chrome such as the Manage files tab and upstream issue buttons.

When inspecting an HTML response that has per-chunk timing and inflated byte counts (available from tcpdump, netlog, chrome-trace, cdp, and wptagent), the request inspector renders the Response Body as a hex-viewer-style table — one row per delivered wire chunk, with arrival timestamps and sizes in the left column and the syntax-highlighted HTML slice that arrived in that delivery on the right. This makes it easy to correlate "what arrived when" against the canvas waterfall.

Request details omit optional fields when the underlying capture did not provide meaningful data, so field-captured formats such as rumcap do not show empty or NaN rows for hidden cross-origin timing/header data. Canceled or unknown-status requests still display an explicit status note, and rumcap statuses that were assumed successful are labeled as assumed in the details/raw data via _statusAssumed.

A persistent Waterfall History in IndexedDB records every URL the viewer loads (from the landing page or via query parameters) along with test metadata.

Query parameters

The viewer tracks its state via the browser History API, so the URL updates as you click around and is always shareable.

• src=<url> — remote file to fetch and load.
• keylog=<url> — TLS keylog to pair with src for raw packet captures.
• page=<index> — open a specific multi-page run (aliases pageId); skips the thumbnail grid.
• tab=<name> — jump to a tab: summary, waterfall, trace (Perfetto), devtools, lighthouse, netlog, qvis, or RequestN (e.g. Request10).
• options=<csv> — override defaults in key:val pairs (e.g. options=showCpu:false,showBw:false).

https://your-domain.com/viewer/?src=https://example.com/trace.json.gz&page=1_Cached&tab=Request10&options=showCpu:false,showWait:false

Without src, the viewer shows a URL entry bar and a drag-and-drop upload zone. Paste a WebPageTest result URL (https://www.webpagetest.org/result/YYMMDD_.../) and it automatically rewrites it to the HAR export endpoint.

Iframe programmatic API

When embedding the viewer in an iframe, it exposes a global for pushing data without round-tripping through URL strings:

<iframe id="waterfall-iframe" src="/dist/viewer/index.html"></iframe>
<script>
    const viewer = document.getElementById('waterfall-iframe').contentWindow.WaterfallViewer;

    // Push an ArrayBuffer, Blob, or File directly.
    viewer.loadData(arrayBuffer);

    // Re-render with tweaked options.
    viewer.updateOptions({ showCpu: true });
</script>

CLI

npx waterfall-tools dump.cap.gz --keylog dump_keys.txt.gz > out.har

The CLI auto-detects every supported input format from the file bytes — HAR, WPT JSON, Chrome trace, Perfetto, netlog, CDP, tcpdump, wptagent zips, rumcap .rcap (plain or gzipped .rcap.gz), and qlog .qlog / .sqlog all work the same way:

npx waterfall-tools capture.rcap > out.har
npx waterfall-tools capture.sqlog > out.har

qlog files carry one QUIC connection each; to merge a whole page load's connections into one HAR, pass multiple files and use --output for the destination:

npx waterfall-tools www.example.com.qlog.gz cdn.example.com.qlog.gz --output out.har

Developer guide

Setup

npm install

Test

Tests are vitest suites that assert parsed outputs against golden Extended HAR fixtures.

npm test

Browser smoke tests use Playwright against the standalone viewer and embedded qvis UI. The default run builds the qvis fork dist, starts the viewer dev server, and uses headless Chromium/Chrome; set PLAYWRIGHT_ALL_BROWSERS=1 or use the all-browsers script after installing the extra Playwright browsers.

npm run test:browser
npm run test:browser:all

Lint

ESLint (flat config, eslint.config.js) is run automatically as the first step of npm run build — warnings fail the build (--max-warnings 0). Only first-party code under src/, tests/, scripts/, bin/, and cloudflare-worker/ is linted; dependencies and vendored third-party bundles are excluded.

npm run lint       # report lint issues
npm run lint:fix   # auto-fix what is safe to auto-fix

Build

# Builds ESM payloads under /dist/node/ and /dist/browser/waterfall-tools/.
# Output is hashed (waterfall-[hash].js, tcpdump-[hash].js, decompress-[hash].js)
# with a stable waterfall-tools.es.js stub that re-exports the hashed core —
# enabling immutable long-lived CDN caching.
# Each static artifact also gets a .br (Brotli level 11) sibling for zero-compute edge serving.
npm run build

# Local viewer preview with Hot Module Replacement against the live source.
npm run dev:viewer

Demos

A drag-and-drop demo harness in src/demo/ exercises the core pipelines graphically. Drop two files in sequence (e.g. a .pcap plus a .key_log) to combine packet tracing with live TLS decryption.

npm run dev:demo     # dev server with HMR
npm run build:demo   # bundled demo output under bin/demo/

Continuous integration

Every pull request to main triggers .github/workflows/ci.yml, which installs dependencies with npm ci and then runs npm run lint and npm run build on Node 22. Lint warnings and build failures block the PR. Periodic security audits are run using npm audit to keep third-party dependencies secure.

License

Apache 2.0 — full text in LICENSE. The dependency tree is deliberately free of GPL-licensed code.