cdn-proxy-cache
v0.3.0
Published
A caching proxy for CDN resources with URL rewriting, content transformation, and Express integration
Maintainers
osteeleReadme
cdn-proxy-cache
cdn-proxy-cache is an Express middleware library for caching CDN resources on disk. It rewrites CDN URLs in HTML and CSS so applications can continue to load cached assets while offline.
Features
- Stores responses in a content-addressable cache managed by
cacache. - Rewrites matching
<script src>, stylesheet<link href>, and CSSurl()values. - Preserves gzip and deflate encoding while rewriting CSS.
- Coalesces simultaneous misses and stale refreshes for the same cache key.
- Revalidates cached responses with ETag and Last-Modified validators.
- Serves ordinary expired entries while refreshing them in the background.
- Honors
no-store,private,no-cache, andmust-revalidatecache directives. - Warms the cache from seed URLs and follows references found in CSS.
- Supports Express 4 and 5 on Node.js 18.17 or later.
- Reports structured request, cache, and error events.
Installation
Install the library and its Express peer dependency:
npm install cdn-proxy-cache expressWith Bun:
bun add cdn-proxy-cache expressQuick start
import express from 'express';
import { createProxyCache } from 'cdn-proxy-cache';
import os from 'node:os';
import path from 'node:path';
const app = express();
const cdnHosts = new Set(['cdn.jsdelivr.net', 'cdnjs.cloudflare.com']);
const cache = createProxyCache({
proxyPrefix: '/__proxy_cache',
cachePath: path.join(os.homedir(), '.cache', 'my-app'),
cacheSeeds: [
'https://cdn.jsdelivr.net/npm/[email protected]/lib/p5.min.js',
'https://cdnjs.cloudflare.com/ajax/libs/jquery/3.6.0/jquery.min.js',
],
shouldProxyPath: (url) => /^https?:/.test(url) && cdnHosts.has(new URL(url).hostname),
});
app.use(cache.proxyPrefix, cache.router);
app.get('/', (_req, res) => {
const html = `
<!doctype html>
<html>
<head>
<script src="https://cdn.jsdelivr.net/npm/[email protected]/lib/p5.min.js"></script>
</head>
<body><h1>Hello</h1></body>
</html>
`;
res.send(cache.replaceUrlsInHtml(html));
});
app.listen(3000);shouldProxyPath is the proxy allowlist. Rewriting methods leave rejected URLs unchanged. Requests that address a rejected URL through cache.router receive HTTP 403.
Malformed, credential-bearing, or non-HTTP proxy targets receive HTTP 400 before the allowlist or network is consulted. The router and rewriting methods pass only canonical HTTP(S) URLs without embedded credentials to shouldProxyPath.
Cache warming
cache.warm() fetches seed URLs concurrently and follows relative or proxied URLs found in CSS. It does not inspect HTML or JavaScript for more dependencies.
const stats = await cache.warm({ concurrency: 8 }, (message) => {
if (message.type === 'progress') {
const completed = message.stats.hits + message.stats.misses + message.stats.failures;
console.log(`Completed ${completed} requests`);
}
});
console.log(stats);Pass an AbortSignal to cancel warming and its active origin requests:
const controller = new AbortController();
process.once('SIGINT', () => controller.abort());
try {
await cache.warm({ signal: controller.signal });
} catch (error) {
if (!controller.signal.aborted) throw error;
}Cancellation is scoped to the warming operation that receives the signal. If it is waiting for origin work owned by another caller, that shared transfer continues. If it owns a transfer that another caller is waiting for, the waiter retries after the canceled or failed generation finishes.
API
createProxyCache(options)
Creates a ProxyCache instance.
| Option | Type | Default | Description |
| --- | --- | --- | --- |
| proxyPrefix | string | required | URL prefix mounted on the Express application. |
| cachePath | string | required | Directory used for cached content and metadata; resolved to its canonical physical path at construction. |
| cacheSeeds | string[] | required | Starting URLs for cache.warm(). Use an empty array when warming is not needed. |
| shouldProxyPath | (url: string) => boolean | required | Return true for URLs that the cache may rewrite and proxy. |
| cssTransformVersion | string | '' | Bump when configuration captured by shouldProxyPath changes without changing the callback source. |
| requestTimeoutMs | number | 30000 | Maximum origin-request time, including response streaming. |
| maxCssTransformBytes | number | 5242880 | Maximum decompressed CSS size buffered for rewriting. |
| maxCacheSizeBytes | number | none | Maximum bytes retained across unique live response bodies. Use only for a cache directory owned by one process. |
| warmConcurrency | number | 20 | Default maximum number of simultaneous warming requests. |
| onEvent | (event: ProxyCacheEvent) => void | none | Receives structured lifecycle events. |
Configured numeric options must be positive integers. cachePath must not be empty, a filesystem root (including through a symlink), or contain a dangling symlink. An origin timeout produces HTTP 504 if response headers have not been sent. CSS that exceeds maxCssTransformBytes fails the response and emits a stream error event.
When maxCacheSizeBytes is configured, the cache counts content-addressed response bodies rather than index entries, so entries that share identical content consume the body size once. After a mutating request settles, the cache removes the oldest unique bodies until it is within the bound. A response larger than the bound is delivered but not retained.
cache.router(req, res)
Handles a proxy request. Mount it at the configured prefix:
app.use(cache.proxyPrefix, cache.router);Responses include these diagnostic headers:
x-cdn-proxy-cache-hit:HITorMISS.x-cdn-proxy-origin-url: decoded origin URL.
The cache does not store responses marked no-store or private, or responses with Vary: *. It fetches no-cache responses before serving them. Expired responses marked must-revalidate are also fetched before they are served. Other expired responses are served immediately and refreshed in the background.
The proxy removes hop-by-hop response headers, fields named by Connection, origin-wide state headers such as Strict-Transport-Security, Clear-Site-Data, authentication challenges, and cookies that would otherwise affect the proxy application's origin. Origin responses cannot replace the proxy diagnostic headers. The origin's Server value is exposed as Origin-Server.
cache.replaceUrlsInHtml(html)
Rewrites allowed URLs in <script src> and <link rel="stylesheet" href> attributes.
const rewrittenHtml = cache.replaceUrlsInHtml(originalHtml);cache.replaceUrlsInCss(css)
Rewrites allowed absolute URLs in CSS url() values. Relative URLs and data URLs remain unchanged.
const rewrittenCss = cache.replaceUrlsInCss(originalCss);cache.warm(options, callback?)
Fetches seed URLs and CSS dependencies. The options are:
force: bypass existing cache entries.reload: start from URLs already in the cache instead ofcacheSeeds. Combine it withforceto refetch them.concurrency: overridewarmConcurrencyfor this operation.signal: cancel warming with anAbortSignal.
The callback receives one of these messages:
type CacheWarmMessage =
| { type: 'initial'; total: number }
| { type: 'prefetch'; url: string }
| { type: 'progress'; stats: CacheWarmStats }
| { type: 'error'; url: string; status: number };The returned CacheWarmStats contains total, hits, misses, and failures.
Cache management
await cache.clear();
const pruneStats = await cache.prune();
const entries = await cache.ls();
cache.isProxyPath('/__proxy_cache/cdn.jsdelivr.net/example.js');
const proxyPath = cache.encodeProxyPath('https://cdn.jsdelivr.net/example.js');
const originUrl = cache.decodeProxyPath(proxyPath);cache.clear() removes cache-owned entries, bodies, temporary writes, and verification metadata while preserving unrelated files in the cache directory. cache.prune() checks live body integrity, removes corrupt and missing entries, reclaims orphaned content, cleans temporary writes, and returns typed reclamation statistics. Within one process, clear and prune wait for active operations on the same physical cache directory, including path aliases, and new operations wait for maintenance to finish. Failures reject the maintenance promise and release that barrier.
Multiple processes may share one cache directory for ordinary requests and warming. This lets a CLI warm prime the cache used by a separate server or editor extension. Completed entries are reusable by matching requests across processes, and overlapping cold writes remain safe, although they can make duplicate origin requests because request coalescing is process-local.
Cache-wide maintenance is not coordinated across processes. Stop every process that uses the directory before calling clear() or prune(). For the same reason, do not configure maxCacheSizeBytes on a directory shared by multiple processes: its automatic eviction is a maintenance operation. Use separate directories only when the consumers should not share warmed content or cannot be quiesced for maintenance.
cache.ls() returns the current entries using package-owned public types. The encoding methods preserve an origin query string inside the proxy's search parameter. This leaves room for proxy-specific query parameters without changing the origin URL.
Lifecycle events
The optional onEvent callback receives these events:
type ProxyCacheEvent =
| { type: 'request'; url: string }
| { type: 'cache-hit'; url: string; stale: boolean }
| { type: 'cache-miss'; url: string }
| { type: 'cache-write'; url: string; bytes: number }
| { type: 'cache-skip'; url: string; reason: 'no-store' | 'private' | 'vary-star' }
| { type: 'error'; url: string; phase: 'fetch' | 'stream'; error: Error };const cache = createProxyCache({
// Other options...
onEvent: (event) => {
if (event.type === 'error') console.error(event.url, event.error);
},
});Command helpers
The package exports functions for applications that provide their own command-line interface:
import { clearCache, listCache, showCacheInfo, warmCache } from 'cdn-proxy-cache';
await clearCache(cache);
await warmCache(cache, { force: false, verbose: true });
await listCache(cache, { json: false, verbose: true });
await showCacheInfo(cache);
await showCacheInfo(cache, 'https://cdn.jsdelivr.net/example.js');The package does not install a command-line executable.
Request flow
Each request uses a cache key built from the origin URL and canonicalized Accept, Accept-Language, and Accept-Encoding values. These are all the representation-selecting request headers forwarded to the origin; the browser's User-Agent is not forwarded. The proxy removes Brotli from Accept-Encoding so browsers can share gzip or deflate entries without creating duplicates for equivalent header orderings.
On a miss, ordinary origin bodies are sent to the client and cacache at the same time. CSS takes a separate transformation path because css-tree needs the complete decompressed stylesheet; the rewritten result is cached so subsequent hits do not repeat parsing and compression. The cache records a fingerprint of the transformation configuration and refetches transformed CSS when that fingerprint changes. Set cssTransformVersion when shouldProxyPath depends on closed-over configuration that may change between cache instances. Other response bodies remain streaming.
Development
The project uses Bun for dependency management and tests, Biome for formatting and linting, and TypeScript for type checking.
bun install
just check
just buildRun just to list the available development tasks.
License
MIT © Oliver Steele
Related project
p5-server contains the implementation from which this package was extracted.