GhostFetch

Ultra-lightweight network spoofing HTTP client for Node.js

GhostFetch is a Node.js HTTP client library that perfectly mimics the network fingerprints of real browsers (Chrome, Firefox, Safari) to bypass bot detection systems such as Cloudflare, Akamai, and Datadome.

"No GUI, No Browser Engine, Pure Speed & Smart Bypass."

Key Features

• TLS Fingerprint Spoofing — Reproduces JA3/JA4/JA4H identically to real browsers (Go uTLS engine)
• HTTP/2 Fingerprint Matching — SETTINGS, WINDOW_UPDATE, PRIORITY frame order and values match real browsers
• Automatic Cookie Management — RFC 6265 CookieJar, redirect cookie propagation, session isolation
• JS Challenge Auto-Solving — Cloudflare/Akamai/Datadome challenge detection + sandboxed execution (enabled by default)
• Proxy Support — HTTP/HTTPS/SOCKS5 proxy tunneling with full TLS fingerprint preservation
• Standard API — Web fetch() compatible interface, minimal learning curve
• Minimal Dependencies — Only runtime dependency is zod; isolated-vm is optional
• Cross-Platform — Linux, macOS, and Windows (x64 & arm64)

Installation

npm install ghostfetch

The Go engine binary for your platform is automatically installed via platform-specific optional dependencies (@ghostfetch/engine-*).

Requirements

| Requirement | Version | |-------------|---------| | Node.js | >= 22.0.0 | | OS | Linux x64/arm64, macOS x64/arm64, Windows x64/arm64 |

Platform Support

| Platform | Architecture | Status | |----------|-------------|--------| | Linux | x64 | :white_check_mark: Fully supported | | Linux | arm64 | :white_check_mark: Fully supported | | macOS | x64 (Intel) | :white_check_mark: Fully supported | | macOS | arm64 (Apple Silicon) | :white_check_mark: Fully supported | | Windows | x64 | :white_check_mark: Fully supported | | Windows | arm64 | :warning: Cross-compiled, limited testing |

Optional: Enhanced JS Sandbox

For stronger JS challenge solving isolation, install isolated-vm:

npm install isolated-vm

If isolated-vm is not installed (or fails to build on your platform), GhostFetch automatically falls back to Node.js's built-in vm module. This fallback works for the vast majority of JS challenges.

Quick Start

Simplest Usage — ghostFetch()

import { ghostFetch, ghostFetchCleanup } from 'ghostfetch';

// No instance creation needed — JS challenge auto-solving enabled by default
const res = await ghostFetch('https://protected-site.com', {
  browser: 'Chrome_131',
});
console.log(res.status);        // 200
console.log(await res.text());

// Clean up before program exit
await ghostFetchCleanup();

Full Control — GhostClient

import { GhostClient } from 'ghostfetch';

const client = new GhostClient({
  browser: 'Chrome_131',           // Browser fingerprint
  proxy: 'http://proxy:8080',      // Proxy (optional)
  timeout: 15_000,                 // Request timeout
});

// JS challenge auto-solving is enabled by default,
// so protected sites work without extra configuration
const res = await client.fetch('https://protected-site.com');
console.log(res.status, res.headers);
console.log(await res.json());

// POST request
const loginRes = await client.fetch('https://example.com/login', {
  method: 'POST',
  headers: { 'Content-Type': 'application/json' },
  body: JSON.stringify({ user: 'admin', pass: '1234' }),
});

// Cookie inspection
console.log(client.cookies.getCookieString('https://example.com'));

// Session save/restore
const saved = client.exportSession();
// ... later ...
client.importSession(saved);

await client.destroy();

Accessing a Cloudflare-Protected Site

import { GhostClient } from 'ghostfetch';

const client = new GhostClient({ browser: 'Chrome_131' });

// 1st request may trigger a 503 JS challenge
// GhostFetch detects it -> executes the challenge script in a sandbox
// -> automatically resubmits with the solution cookie
// -> returns the final 200 response
const res = await client.fetch('https://cf-protected-site.com/api/data');
const data = await res.json();
console.log('Success:', data);

await client.destroy();

Browser Profiles

GhostFetch ships with 6 browser profiles that reproduce the full TLS + HTTP/2 + header fingerprint:

| Profile ID | Browser | TLS | HTTP/2 | Headers | |-----------|---------|-----|--------|---------| | Chrome_120 | Chrome 120 | :white_check_mark: | :white_check_mark: | :white_check_mark: | | Chrome_124 | Chrome 124 | :white_check_mark: | :white_check_mark: | :white_check_mark: | | Chrome_131 | Chrome 131 | :white_check_mark: | :white_check_mark: | :white_check_mark: | | Firefox_119 | Firefox 119 | :white_check_mark: | :white_check_mark: | :white_check_mark: | | Firefox_128 | Firefox 128 | :white_check_mark: | :white_check_mark: | :white_check_mark: | | Safari_17 | Safari 17 | :white_check_mark: | :white_check_mark: | :white_check_mark: |

Each profile includes:

• TLS fingerprint — Cipher suites, extensions, curves, signature algorithms (JA3/JA4 match)
• HTTP/2 settings — SETTINGS frame values, WINDOW_UPDATE size, PRIORITY frames
• Headers — User-Agent, sec-ch-ua, sec-fetch-*, Accept-* and ordering

import { listProfileIds, getProfile } from 'ghostfetch';

console.log(listProfileIds());
// ['Chrome_120', 'Chrome_124', 'Chrome_131', 'Firefox_119', 'Firefox_128', 'Safari_17']

const profile = getProfile('Chrome_131');
// { tls, h2, headers, userAgent, ... }

Tip: If one profile triggers persistent challenges on a specific site, GhostFetch automatically tries fallback profiles (Firefox -> Safari) before giving up.

Proxy Support

Supported Protocols

| Protocol | URL Format | Default Port | |----------|-----------|-------------| | HTTP | http://[user:pass@]host:port | 8080 | | HTTPS | https://[user:pass@]host:port | 8443 | | SOCKS5 | socks5://[user:pass@]host:port | 1080 |

All proxy protocols preserve the full TLS fingerprint — the proxy tunneling is handled by the Go engine at the TCP level.

Single Proxy

const res = await ghostFetch('https://example.com', {
  browser: 'Chrome_131',
  proxy: 'socks5://user:pass@proxy:1080',
});

Per-Request Proxy Override

const client = new GhostClient({
  browser: 'Chrome_131',
  proxy: 'http://default-proxy:8080',
});

// Override proxy for a specific request
const res = await client.fetch('https://example.com', {
  proxy: 'socks5://special-proxy:1080',
});

await client.destroy();

Proxy Rotation

import { ProxyRotator, GhostClient } from 'ghostfetch';

const rotator = new ProxyRotator([
  'http://proxy1:8080',
  'http://proxy2:8080',
  'socks5://proxy3:1080',
], 'round-robin');  // or 'random'

const client = new GhostClient({ browser: 'Chrome_131' });

for (const url of urls) {
  const proxy = rotator.next();
  try {
    await client.fetch(url, { proxy });
  } catch {
    rotator.disable(proxy, 60_000); // Disable failed proxy for 1 min
  }
}

await client.destroy();

Built-in Pool Rotation

const client = new GhostClient({
  browser: 'Chrome_131',
  proxyPool: [
    'http://proxy1:8080',
    'http://proxy2:8080',
    'socks5://proxy3:1080',
  ],
  proxyRotation: 'round-robin',  // or 'random'
});

// Each fetch() call automatically picks the next proxy
const res = await client.fetch('https://example.com');

await client.destroy();

JS Challenge Auto-Solving

JS challenge auto-solving is enabled by default. GhostFetch detects and solves challenges from multiple bot detection systems without any extra configuration.

const client = new GhostClient({ browser: 'Chrome_131' });

// 503 JS Challenge -> auto-solve -> 200 response returned
const res = await client.fetch('https://protected-site.com');
console.log(res.status); // 200

await client.destroy();

Supported Challenge Types

| Provider | Detection | Auto-Solve | Notes | |----------|-----------|------------|-------| | Cloudflare | :white_check_mark: 503 + cf-mitigated header | :white_check_mark: | Fast path for managed challenges | | Akamai | :white_check_mark: 403 + _abck cookie pattern | :white_check_mark: | Sensor data generation | | Datadome | :white_check_mark: x-datadome header | :white_check_mark: | Cookie challenge | | Generic | :white_check_mark: JS cookie set, form submit | :white_check_mark: | Score-based detection | | Interactive CAPTCHA | :x: | :x: | Requires human interaction |

Customization

const client = new GhostClient({
  browser: 'Chrome_131',
  maxChallengeRetries: 5,       // Max retry attempts (default: 3)
  sandboxMemoryLimit: 64,       // V8 Isolate memory limit in MB (default: 128)
  sandboxTimeout: 5_000,        // Sandbox execution timeout in ms (default: 10000)
});

Disabling

const client = new GhostClient({
  browser: 'Chrome_131',
  autoSolveJsChallenge: false,  // Explicitly disable
});

Advanced Usage

Multi-Session Management

Each GhostClient instance maintains its own isolated session (cookies, challenge state):

import { GhostClient } from 'ghostfetch';

// Two independent sessions — different cookies, different identities
const session1 = new GhostClient({ browser: 'Chrome_131' });
const session2 = new GhostClient({ browser: 'Firefox_128' });

await session1.fetch('https://example.com/login', {
  method: 'POST',
  body: JSON.stringify({ user: 'user1', pass: 'pass1' }),
});
await session2.fetch('https://example.com/login', {
  method: 'POST',
  body: JSON.stringify({ user: 'user2', pass: 'pass2' }),
});

// Each client has its own cookies
console.log(session1.cookies.getCookieString('https://example.com'));
console.log(session2.cookies.getCookieString('https://example.com'));

await Promise.all([session1.destroy(), session2.destroy()]);

Session Serialization & Restoration

Persist sessions across program restarts:

import { GhostClient } from 'ghostfetch';
import fs from 'node:fs';

const client = new GhostClient({ browser: 'Chrome_131' });

// ... make requests, accumulate cookies ...
await client.fetch('https://example.com');

// Export session to JSON-serializable object
const session = client.exportSession();
fs.writeFileSync('session.json', JSON.stringify(session));

// Later — restore from disk
const saved = JSON.parse(fs.readFileSync('session.json', 'utf-8'));
const client2 = new GhostClient({ browser: 'Chrome_131' });
client2.importSession(saved);

// Continues with all cookies intact
await client2.fetch('https://example.com/dashboard');

await client2.destroy();

Custom Header Merging

GhostFetch normalizes headers to match the browser profile, but you can add custom headers:

const res = await client.fetch('https://api.example.com/data', {
  headers: {
    'Authorization': 'Bearer token123',
    'X-Custom-Header': 'my-value',
  },
});
// Browser-fingerprint headers (User-Agent, sec-ch-ua, etc.) are still
// auto-injected; your custom headers are merged on top.

Request-Level Options

const res = await client.fetch('https://example.com/api', {
  method: 'POST',
  headers: { 'Content-Type': 'application/json' },
  body: JSON.stringify({ key: 'value' }),
  timeout: 30_000,              // Override client timeout
  redirect: 'manual',           // Don't follow redirects
  proxy: 'socks5://proxy:1080', // Override client proxy
  referrer: 'https://example.com/',  // Set referrer for sec-fetch-site calc
  cache: 'no-cache',            // cache-control: no-cache
});

Diagnostic Tools

GhostFetch includes built-in diagnostic tools for analyzing site protection:

# Diagnose all sites in test_site.txt
npm run diagnose

# Diagnose a single site
npm run diagnose -- https://example.com

# Collect baseline (6 profiles x 10 sites matrix)
npm run baseline

# Generate analysis report
npm run report

Programmatic usage:

import { testSite, analyzeResponse } from 'ghostfetch';

const result = await testSite('https://example.com', {
  browser: 'Chrome_131',
  timeout: 15_000,
});
console.log(result.status, result.category);

const analysis = analyzeResponse(result);
console.log(analysis.suspiciousHeaders);

Cookies & Sessions

import { GhostClient } from 'ghostfetch';
import fs from 'node:fs';

const client = new GhostClient({ browser: 'Chrome_131' });

// Cookies are automatically collected and sent
await client.fetch('https://example.com/login', {
  method: 'POST',
  body: 'user=admin&pass=1234',
});
const res = await client.fetch('https://example.com/dashboard');

// Manual cookie inspection
console.log(client.cookies.getCookieString('https://example.com'));

// Session serialization (save/restore as JSON)
const session = client.exportSession();
fs.writeFileSync('session.json', JSON.stringify(session));

// Restore in a later session
const saved = JSON.parse(fs.readFileSync('session.json', 'utf-8'));
client.importSession(saved);

await client.destroy();

Low-Level API

import {
  parseProxy,
  validateProxyUrl,
  ProxyRotator,
  detectChallenge,
  JsSandbox,
  ChallengeSolver,
  CookieJar,
  parseSetCookie,
  normalizeHeaders,
  getProfile,
} from 'ghostfetch';

// Proxy URL parsing
const config = parseProxy('http://user:pass@proxy:8080');

// Proxy validation
const { valid, error } = validateProxyUrl('socks5://proxy:1080');

// Manual cookie management
const jar = new CookieJar();
jar.setCookie('session=abc123; Path=/; HttpOnly', 'https://example.com');
console.log(jar.getCookieString('https://example.com'));

// Manual challenge detection/solving
const detection = detectChallenge(response, bodyText);
if (detection.isChallenge) {
  const sandbox = await JsSandbox.create();
  const solver = new ChallengeSolver(sandbox);
  const result = await solver.solve(detection, context);
  sandbox.dispose();
}

// Header normalization for a profile
const headers = normalizeHeaders(
  getProfile('Chrome_131'),
  'https://example.com',
  'GET',
  { 'Authorization': 'Bearer token' },
);

Error Handling

GhostFetch provides a hierarchy of typed errors for precise error handling:

import {
  GhostFetchError,   // Base class for all errors
  TimeoutError,       // Request timed out
  NetworkError,       // DNS failure, connection refused, etc.
  ProxyError,         // Proxy auth failure, connection failure
  ChallengeError,     // Challenge solving failed
  EngineCrashError,   // Go engine process crashed
  SandboxError,       // JS sandbox execution error
} from 'ghostfetch';

try {
  const res = await ghostFetch('https://example.com');
} catch (err) {
  if (err instanceof TimeoutError) {
    console.log(`Timed out after ${err.timeoutMs}ms`);
  } else if (err instanceof ProxyError) {
    console.log(`Proxy error [${err.code}]: ${err.message}`);
  } else if (err instanceof NetworkError) {
    console.log(`Network error [${err.code}]: ${err.message}`);
  } else if (err instanceof ChallengeError) {
    console.log(`Challenge failed (${err.challengeType})`);
  } else if (err instanceof EngineCrashError) {
    console.log(`Engine crash: code=${err.exitCode}, signal=${err.signal}`);
  } else if (err instanceof SandboxError) {
    console.log(`Sandbox error: ${err.message}`);
  } else if (err instanceof GhostFetchError) {
    console.log(`GhostFetch error: ${err.message}`);
  }
}

Comparison

| Feature | GhostFetch | axios | node-fetch | got | Playwright | |---------|-----------|-------|------------|-----|------------| | TLS Fingerprint Spoofing | :white_check_mark: JA3/JA4 | :x: | :x: | :x: | :white_check_mark: (real browser) | | HTTP/2 Fingerprint | :white_check_mark: Full match | :x: | :x: | :x: | :white_check_mark: (real browser) | | Bot Detection Bypass | :white_check_mark: CF/Akamai/DD | :x: | :x: | :x: | :warning: Detectable | | JS Challenge Auto-Solve | :white_check_mark: Sandboxed | :x: | :x: | :x: | :white_check_mark: (full renderer) | | Memory Usage | ~30 MB | ~10 MB | ~8 MB | ~10 MB | ~300+ MB | | Install Size | ~15 MB | ~2 MB | ~0.3 MB | ~1 MB | ~200+ MB | | Headless Browser | :x: Not needed | :x: | :x: | :x: | :white_check_mark: Required | | Speed | Fast | Fast | Fast | Fast | Slow | | Cookie Management | :white_check_mark: Built-in | :x: Manual | :x: Manual | :white_check_mark: Plugin | :white_check_mark: Built-in | | Proxy Support | :white_check_mark: HTTP/SOCKS5 | :warning: HTTP only | :x: | :white_check_mark: HTTP/SOCKS | :white_check_mark: HTTP/SOCKS |

When to use GhostFetch over Playwright/Puppeteer:

• You need bot detection bypass without a ~300 MB browser engine
• Your workload is API scraping / data collection, not page rendering
• You need high concurrency with low memory overhead
• You need precise TLS/HTTP/2 fingerprint control

Architecture

User Code (TypeScript/JavaScript)
    |
    v
+-----------------------------------------------+
|  GhostClient / ghostFetch()                   |  fetch()-compatible API
|  +-- Header Normalizer                        |  Browser-profile headers
|  +-- CookieJar (RFC 6265)                     |  Automatic cookie management
|  +-- RedirectHandler                          |  Redirect cookie propagation
|  +-- ChallengeInterceptor                     |  JS challenge detect/solve
|  |   +-- JsSandbox (isolated-vm / vm)         |  Sandboxed script execution
|  |   +-- ChallengeSolver                      |  CF/Akamai/DD/Generic
|  +-- ProxyRotator                             |  Proxy pool management
|  +-- IPC Bridge (ndjson / UDS|Named Pipe)     |  Go engine communication
+---------------+-------------------------------+
                | Unix Domain Socket (Linux/macOS)
                | Named Pipe (Windows)
                v
+-----------------------------------------------+
|  Go Engine (ghostengine)                      |  uTLS-based core network
|  +-- uTLS                                     |  TLS fingerprint spoofing
|  +-- HTTP/2 Frame Control                     |  SETTINGS/PRIORITY match
|  +-- Proxy Tunneling                          |  HTTP CONNECT / SOCKS5
|  +-- Content-Encoding                         |  gzip/brotli/zstd decode
|  +-- Profile System                           |  Chrome/Firefox/Safari
+-----------------------------------------------+

The Node.js layer handles session logic, cookie management, and challenge solving. The Go engine handles all network I/O with browser-identical TLS and HTTP/2 fingerprints. Communication uses ndjson over Unix Domain Sockets (Linux/macOS) or Named Pipes (Windows).

Build

# TypeScript + Go build
npm run build

# Go binary only (current platform)
npm run build:go

# All platform Go binaries (cross-compile)
npm run build:go:all

# TypeScript only
npm run build:ts

Building from Source

If the prebuilt Go binary doesn't work for your platform, build from source:

# Requirements: Go 1.22+
cd go-engine && go build -o bin/ghostengine ./cmd/ghostengine

Testing

# All tests (unit + integration)
npm test

# Including E2E
npm run test:all

# Unit tests only
npm run test:unit

# Integration tests only
npm run test:integration

# TLS fingerprint verification (external server required)
npm run test:fingerprint

# Real-site smoke tests
GHOST_TEST_REAL_SITES=1 npm run test:live

# Coverage
npm run test:coverage

Test Statistics

| Category | Count | |----------|-------| | Unit Tests | 727 | | Integration Tests | 181 | | E2E Tests | 23 | | Total | 931+ | | Coverage | 91%+ |

Diagnostic & Report Tools

# Site diagnostics (deep analysis)
npm run diagnose                           # All sites in test_site.txt
npm run diagnose -- https://example.com    # Single site

# Baseline collection (6 profiles x 10 sites matrix)
npm run baseline

# Generate analysis report
npm run report

Supported Bot Detection Systems

| System | Detection | JS Challenge Solve | Cookie Challenge | Notes | |--------|-----------|-------------------|------------------|-------| | Cloudflare | :white_check_mark: | :white_check_mark: | :white_check_mark: | 503 managed challenge + cf-mitigated fast path | | Akamai | :white_check_mark: | :white_check_mark: | :white_check_mark: | _abck / ak_bmsc sensor data | | Datadome | :white_check_mark: | :white_check_mark: | :white_check_mark: | x-datadome header detection | | Generic | :white_check_mark: | :white_check_mark: | :white_check_mark: | Score-based pattern matching | | reCAPTCHA / hCaptcha | :x: | :x: | :x: | Interactive CAPTCHA — not supported | | Imperva/Incapsula | :warning: | :warning: | :warning: | Partial — depends on challenge type |

Known Limitations

• Interactive CAPTCHAs — reCAPTCHA, hCaptcha, and Turnstile interactive challenges require human interaction and cannot be solved automatically
• Browser rendering — GhostFetch does not render HTML/CSS/JS; it only executes challenge scripts in a sandboxed environment
• Rate limiting — Some sites may rate-limit based on IP regardless of fingerprint quality; use proxy rotation
• Windows arm64 — Cross-compiled binary; limited native testing coverage
• isolated-vm on Windows — Native build may fail; vm module fallback is used automatically
• Dynamic fingerprint updates — Browser profile data is static; new browser versions require profile updates

Dependencies

| Package | Type | Purpose | |---------|------|---------| | zod | Runtime | IPC message schema validation | | isolated-vm | Optional | V8 Isolate-based JS sandbox (falls back to vm if unavailable) |

API Overview

Core

| Export | Type | Description | |--------|------|-------------| | GhostClient | Class | Main client with full session management | | ghostFetch() | Function | Standalone fetch with singleton client pooling | | ghostFetchCleanup() | Function | Clean up all singleton clients | | GhostResponse | Class | Web Response-like object with json(), text(), arrayBuffer() |

Profile

| Export | Type | Description | |--------|------|-------------| | getProfile(id) | Function | Get a browser profile by ID | | listProfileIds() | Function | List all available profile IDs | | BROWSER_PROFILES | Object | All profile definitions |

Cookie & Session

| Export | Type | Description | |--------|------|-------------| | CookieJar | Class | RFC 6265 cookie store | | GhostSession | Class | Isolated session (cookies + state) | | RedirectHandler | Class | Redirect chain with cookie propagation | | parseSetCookie() | Function | Parse Set-Cookie header values |

Challenge

| Export | Type | Description | |--------|------|-------------| | detectChallenge() | Function | Detect challenge type from response | | ChallengeSolver | Class | Solve detected challenges | | ChallengeInterceptor | Class | Auto-intercept and solve challenges | | JsSandbox | Class | JS execution sandbox (isolated-vm / vm) |

Proxy

| Export | Type | Description | |--------|------|-------------| | parseProxy() | Function | Parse proxy URL to config object | | validateProxyUrl() | Function | Validate proxy URL format | | ProxyRotator | Class | Proxy pool with round-robin / random rotation |

Errors

| Error Class | Trigger | |-------------|---------| | GhostFetchError | Base class for all errors | | TimeoutError | Request exceeded timeout | | NetworkError | DNS failure, connection refused, etc. | | ProxyError | Proxy auth failure, connection error | | ChallengeError | Challenge solving failed | | EngineCrashError | Go engine process crashed | | SandboxError | JS sandbox execution error | | EngineResponseError | Engine returned an error response |

Diagnostic

| Export | Type | Description | |--------|------|-------------| | testSite() | Function | Test a single site with diagnostics | | testAllSites() | Function | Test multiple sites | | analyzeResponse() | Function | Analyze response for bot detection signals | | collectBaseline() | Function | Collect baseline across profiles and sites |

For complete API documentation, see docs/api-reference.md.

Credits & Acknowledgements

GhostFetch is built on the shoulders of these excellent projects:

• uTLS — TLS fingerprint mimicry in Go
• isolated-vm — V8 Isolate-based sandboxing for JS challenge execution
• zod — TypeScript-first schema validation for IPC messages

License

MIT