Idempotency Middleware

This middleware enables your API to handle requests idempotently, ensuring that the same operation is not executed multiple times. It is built on the concept of idempotence, a property in mathematics and computer science where certain operations can be repeated without changing the result. The middleware is compatible with Connect-like frameworks, such as Restana and Express.js.

Features

• Idempotent Request Handling: Ensures that duplicate requests with the same idempotency key are processed only once, preventing unintended side effects.
• Original Response Replay: On a cache hit, the middleware replays the original HTTP status code, headers, and body instead of returning a generic empty response.
• Concurrent Request Deduplication: Uses an in-flight lock so that simultaneous requests with the same idempotency key execute the handler only once.
• Cache Key Scoping: Cache keys are derived from the HTTP method, full request URL (including query string), and idempotency key to prevent cross-route and cross-method collisions.
• Customizable Cache Integration: Supports any cache library that implements get and set methods, allowing flexibility in your caching strategy.
• Configurable Idempotency Key: Lets you define the key used to identify requests. By default, it uses the x-request-id header.
• Adjustable TTL (Time-to-Live): Provides the ability to configure the expiration time for cache entries, balancing performance and resource usage (max 24 hours).
• HTTP Method Support: Compatible with the following HTTP methods: POST, PUT, PATCH, and DELETE.

Installation

npm install idempotency-middleware

Usage

import {idempotencyMiddleware} from 'idempotency-middleware'
import {createCache} from 'cache-manager'
import express, {type Request, type Response} from 'express'
import Keyv from 'keyv'
import {CacheableMemory} from 'cacheable'

const cache = createCache({
  stores: [
    new Keyv({
      // for Redis support: https://www.npmjs.com/package/cache-manager#update-on-redis-and-ioredis-support
      store: new CacheableMemory({ttl: 60000, lruSize: 5000}),
    }),
  ],
})

const app = express()

app.use(
  idempotencyMiddleware({
    ttl: 5000, // 5 seconds
    cache: {
      get: async (key: string) => {
        return cache.get(key)
      },
      set: async (key: string, value: any, options) => {
        return cache.set(key, value, options?.ttl)
      },
    },
  }),
)

app.post('/create', (req: Request, res: Response) => {
  res.send('Resource created!')
})

app.listen(3000, () => {
  console.log('Server is running on port 3000')
})

Calling the API

curl -X POST http://localhost:3000/create -H "x-request-id: 123"  # 200 -> Resource created!
curl -X POST http://localhost:3000/create -H "x-request-id: 123"  # 200 -> Resource created! (replayed from cache)
# after 5 seconds
curl -X POST http://localhost:3000/create -H "x-request-id: 123"  # 200 -> Resource created!

Options

| Option | Type | Default | Description | | ------------------------- | -------------------------------------- | ----------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | cache | Cache | required | A cache instance with .get(key) and .set(key, value, {ttl}) methods. | | ttl | number | required | Cache TTL in milliseconds. Must be between 1 and 86,400,000 (24 hours). | | idempotencyKeyExtractor | (req) => string \| undefined \| null | req.headers['x-request-id'] | Extracts the idempotency key from the request. The returned key must match ^[a-zA-Z0-9_.~-]{1,128}$. Duplicate headers are exposed as arrays by Node.js; the default extractor does not normalize them, so idempotency is skipped for such requests. | | keyPrefix | string | 'idemp-key-' | Prefix prepended to every cache key. | | maxResponseSize | number | 1,048,576 (1 MB) | Maximum response body size (in bytes) that will be cached. Larger responses are not cached. | | logger | Logger | console | Logger used for error reporting. Must expose an .error(...args) method. |

Behavior notes

• Only successful responses with a 2xx status code are cached.
• Hop-by-hop and connection-level headers such as Connection, Keep-Alive, Transfer-Encoding, Content-Length, and Date are stripped before replay and are not restored from the cache.
• Responses larger than maxResponseSize are still served to the client; only the cache write is skipped.
• Previous versions stored a plain string ("1") in the cache. Those entries are ignored after upgrading, so only new responses will be replayed.

Customizing idempotency key

By default, the middleware uses the x-request-id header to identify the request. You can customize the key that will be used to identify the request by passing a custom idempotencyKeyExtractor function to the middleware.

In production environments, it is strongly recommended to combine the x-request-id header with user/tenant identifiers (e.g., a hashed user ID or session token) to ensure the key's uniqueness and prevent cross-user collisions.

function extractIdempotencyKey(req: Request) {
  const header = req.headers['x-custom-req-id']
  const value = Array.isArray(header) ? header[0] : header
  if (!value || !/^[a-zA-Z0-9_.~-]{1,128}$/.test(value)) {
    return undefined
  }
  // Scope the key with a service and user identifier to prevent cross-user collisions.
  const userId = req.user?.id ?? 'anonymous'
  return `${SERVICE_NAME}-${userId}-${value}`
}

app.use(
  idempotencyMiddleware({
    ttl: 5000,
    idempotencyKeyExtractor: extractIdempotencyKey,
    //...,
  }),
)

Security Considerations

The middleware is designed to operate safely in untrusted or partially trusted environments when configured correctly. Keep the following risks and mitigations in mind:

1. Cache Flooding

An attacker could overwhelm the cache by sending a high volume of requests with unique x-request-id values, exhausting resources and degrading performance.

Mitigation:

• Implement rate limiting and throttling mechanisms at the middleware or API gateway level.
• Set a maximum capacity for the idempotency cache, with a defined eviction policy (e.g., Least Recently Used (LRU) strategy).
• Use the maxResponseSize option to avoid caching very large responses.
• Monitor and log unusual traffic patterns to detect and respond to potential attacks promptly.

2. Identity Spoofing

An attacker could forge the x-request-id header to impersonate another user's requests, potentially interfering with their operations.

Mitigation:

• Use a secure idempotency key that combines the x-request-id header with user-specific information, such as a hashed user identifier or session token.
• Encrypt or digitally sign the x-request-id value to ensure its authenticity and prevent tampering.

3. Concurrent Duplicate Processing

Without locking, two simultaneous requests with the same idempotency key could both execute the underlying handler.

Mitigation:

• This middleware now maintains an in-flight lock per idempotency key so that duplicate concurrent requests wait for the first request to finish and then replay its cached response.

General Recommendations

• Regularly audit the middleware's security practices and ensure compliance with your organization's security standards.
• Use HTTPS to protect the x-request-id header and prevent interception or tampering during transmission.
• Test your middleware against common attack scenarios, such as denial-of-service (DoS) or injection attacks, to ensure robust protection.

License

See the LICENSE file for license rights and limitations (MIT).