Cloudflare Proxy enabled Fetch

A robust HTTP/HTTPS proxy client implementation for Cloudflare Workers, leveraging the cloudflare:sockets API. This library provides a fetch-compatible interface with advanced proxy support, including automatic retries, proxy rotation, and comprehensive error handling.

Features

• 🔒 Full HTTPS Support: Complete HTTPS tunneling through HTTP/HTTPS proxies
• 🔄 Proxy Rotation: Built-in support for proxy rotation and failover
• 🔐 Authentication: Proxy authentication with username/password
• 📝 Verbose Logging: Detailed connection logging for debugging
• 🗜️ Compression: Automatic handling of gzip compression
• 🔁 Chunked Transfer: Support for chunked transfer encoding
• 🛡️ Error Handling: Comprehensive proxy error detection and recovery
• 🌐 Fetch API Compatible: Drop-in replacement for the standard fetch API
• 🔀 Redirect Handling: Automatic redirect following with configurable limits

Installation

npm install @dividenconquer/cloudflare-proxy-fetch

Wrangler Configuration

⚠️ Important: This library requires the nodejs_compat compatibility flag to function properly.

Make sure your wrangler.jsonc includes the required compatibility flag:

{
  "compatibility_flags": ["nodejs_compat"]
}

This flag is necessary because the library uses node:zlib for gzip decompression, which is only available with the Node.js compatibility layer enabled.

Quick Start

import { proxyFetch } from '@dividenconquer/cloudflare-proxy-fetch';

// Basic usage with a single proxy
const response = await proxyFetch('https://api.example.com/data', {
  proxy: 'http://username:[email protected]:8080'
});

const data = await response.json();

Response Methods

The returned Response object supports all standard methods:

const response = await proxyFetch('https://api.example.com/data', {
  proxy: 'http://proxy.example.com:8080'
});

// Parse as JSON
const jsonData = await response.json();

// Get as text
const textData = await response.text();

// Get as HTML (same as text)
const htmlData = await response.html();

// Get as Blob
const blobData = await response.blob();

// Get as ArrayBuffer
const bufferData = await response.arrayBuffer();

Advanced Usage

Proxy Rotation

const response = await proxyFetch('https://api.example.com/data', {
  proxy: async (attempt) => {
    const proxies = [
      'http://proxy1.example.com:8080',
      'http://proxy2.example.com:8080',
      'http://proxy3.example.com:8080'
    ];
    return proxies[attempt % proxies.length];
  },
  maxRetries: 3,
  retryDelay: 1000
});

Custom Headers and Body

const response = await proxyFetch('https://api.example.com/data', {
  proxy: 'http://proxy.example.com:8080',
  method: 'POST',
  headers: {
    'Content-Type': 'application/json',
    'Authorization': 'Bearer token'
  },
  body: JSON.stringify({ key: 'value' })
});

Redirect Handling

// Enable redirect following (default: 10 redirects)
const response = await proxyFetch('https://api.example.com/data', {
  proxy: 'http://proxy.example.com:8080',
  maxRedirects: 5  // Follow up to 5 redirects
});

// Disable redirect following
const response = await proxyFetch('https://api.example.com/data', {
  proxy: 'http://proxy.example.com:8080',
  maxRedirects: 0  // Don't follow redirects
});

Debug Mode

const response = await proxyFetch('https://api.example.com/data', {
  proxy: 'http://proxy.example.com:8080',
  verbose: true  // Enables detailed logging
});

API Reference

proxyFetch(url, options)

Parameters

• url (string | URL): The target URL to fetch
• options (ProxyFetchOptions): Configuration options

ProxyFetchOptions

Extends the standard RequestInit interface with:

| Option | Type | Default | Description | |--------|------|---------|-------------| | proxy | string \| ((attempt: number) => Promise<string>) | Required | Proxy URL or function returning proxy URL | | maxRetries | number | 3 | Maximum number of retry attempts | | retryDelay | number | 1000 | Delay between retries (ms) | | maxRedirects | number | 10 | Maximum number of redirects to follow (0 to disable) | | verbose | boolean | false | Enable debug logging |

All standard fetch options (method, headers, body, etc.) are also supported.

Returns

Returns a standard Response object with:

• All standard Response methods:
  • json() - Parse response as JSON
  • text() - Get response as text
  • html() - Get response as HTML (alias for text)
  • blob() - Get response as Blob
  • arrayBuffer() - Get response as ArrayBuffer
• Standard Response properties (status, headers, etc.)

Error Handling

The library automatically detects and handles common proxy errors:

HTTP Status Codes

• 407: Proxy Authentication Required
• 403: Forbidden (Access denied by proxy)
• 502: Bad Gateway (Proxy server error)
• 504: Gateway Timeout (Proxy timeout)

Error Patterns

The library uses pattern matching to detect proxy-related errors:

• Proxy blocking/denial messages
• Connection refused errors
• Timeout messages
• Forbidden responses
• Bad gateway errors

Retry Behavior

Errors are automatically retried with different proxies (if configured) up to the specified maxRetries limit. Non-proxy related errors (like 404 Not Found) are not retried.

Testing

You can test this library using the included example Cloudflare Worker. The example provides a web interface for testing proxy functionality:

Running the Example

1. Navigate to the example directory:

   cd example

2. Install dependencies:

   npm install

3. Start the development server:

   npm run dev

4. Open your browser and visit the local development URL (usually http://localhost:8787)

Example Features

The test interface allows you to:

• Test different HTTP methods (GET, POST, PUT, DELETE)
• Configure target URLs and proxy settings
• Send JSON request bodies
• Enable verbose logging for debugging
• View detailed response information including timing and headers

Pre-configured Test Cases

The example includes several pre-configured test cases:

• HTTPBin GET Test: Tests basic GET requests through a proxy
• Local Echo Test: Tests requests to the local echo endpoint
• POST Request Test: Tests POST requests with JSON payloads
• Redirect Test: Tests automatic redirect following (e.g., naver.com)
• No Redirect Test: Tests with redirect following disabled

Requirements

• Cloudflare Workers environment
• Access to cloudflare:sockets API
• Node.js 16.0.0 or later for development

Contributing

Contributions are welcome! Please feel free to submit a Pull Request. For major changes, please open an issue first to discuss what you would like to change.

License

This project is licensed under the MIT License - see the LICENSE file for details.

Security

For security concerns, please open an issue or contact the maintainers directly.

Acknowledgments

• Built on Cloudflare Workers platform
• Inspired by the standard fetch API
• Uses pako for gzip decompression