SingleFlight

A TypeScript/JavaScript utility class that solves the Thundering Herd Problem by ensuring that multiple concurrent requests for the same resource execute only once, with all callers receiving the same result.

The Problem: Thundering Herd

When multiple clients simultaneously request the same expensive resource (like a database query or API call), traditional caching can fail if the cache is empty. This results in:

• Multiple identical database queries executing simultaneously
• Resource waste and increased server load
• Database connection pool exhaustion
• Degraded performance for all users

Example of the Problem:

// Without SingleFlight - BAD ❌
async function getUserData(userId: string) {
  const cached = cache.get(userId)
  if (cached) return cached

  // If cache is empty, ALL concurrent requests will hit the database
  const userData = await database.query(
    "SELECT * FROM users WHERE id = ?",
    userId
  )
  cache.set(userId, userData)
  return userData
}

// 100 concurrent requests = 100 database queries! 😱

The Solution: SingleFlight

SingleFlight ensures that only one execution happens per unique key, regardless of how many concurrent requests are made.

// With SingleFlight - GOOD ✅
async function getUserData(userId: string) {
  return SingleFlight.do(`user:${userId}`, async () => {
    const cached = cache.get(userId)
    if (cached) return cached

    // Only ONE database query executes, even with 100 concurrent requests
    const userData = await database.query(
      "SELECT * FROM users WHERE id = ?",
      userId
    )
    cache.set(userId, userData)
    return userData
  })
}

// 100 concurrent requests = 1 database query! 🎉

Installation

npm install your-package-name
# or
yarn add your-package-name

Quick Start

import { SingleFlight } from "your-package-name"

// Basic usage
const result = await SingleFlight.do("expensive-operation", async () => {
  // This expensive operation will only run once,
  // even if called simultaneously by multiple requests
  return await fetchDataFromDatabase()
})

API Reference

SingleFlight.do<T>(key: string, fn: () => Promise<T>): Promise<T>

Executes the provided function only once per unique key, regardless of concurrent calls.

Parameters:

• key: Unique identifier for the operation
• fn: Async function to execute

Returns: Promise that resolves to the function's result

Example:

const userData = await SingleFlight.do("user:123", async () => {
  return await database.getUser("123")
})

SingleFlight.hasLock(key: string): boolean

Checks if an operation is currently running for the given key.

Parameters:

• key: The key to check

Returns: true if operation is in progress, false otherwise

Example:

if (SingleFlight.hasLock("user:123")) {
  console.log("User data fetch is already in progress")
}

SingleFlight.clear(): void

Clears all active locks. Useful for testing or cleanup.

Example:

// Clear all locks (typically used in tests)
SingleFlight.clear()

Usage Examples

1. Database Query Deduplication

class UserService {
  async getUser(userId: string) {
    return SingleFlight.do(`user:${userId}`, async () => {
      console.log(`Fetching user ${userId} from database`)
      return await this.database.findUser(userId)
    })
  }
}

// Even if called 100 times simultaneously, database query runs only once
const userService = new UserService()
const promises = Array.from({ length: 100 }, () => userService.getUser("123"))
const results = await Promise.all(promises) // Only 1 database query!

2. API Call Deduplication

class WeatherService {
  async getWeather(city: string) {
    return SingleFlight.do(`weather:${city}`, async () => {
      console.log(`Calling weather API for ${city}`)
      const response = await fetch(`https://api.weather.com/${city}`)
      return response.json()
    })
  }
}

// Multiple concurrent requests for the same city = single API call

Performance Benefits

Before SingleFlight:

• 100 concurrent requests → 100 database queries
• High CPU usage, memory consumption
• Database connection pool exhaustion
• Slower response times

After SingleFlight:

• 100 concurrent requests → 1 database query
• Reduced resource usage
• Better database performance
• Faster overall response times

Testing

The package includes comprehensive tests. To run them:

pnpm test

Common Use Cases

• Database query deduplication
• API call optimization
• Expensive computation results
• Third-party service calls

TypeScript Support

SingleFlight is written in TypeScript and provides full type safety:

// Type is automatically inferred
const user: User = await SingleFlight.do(
  "user:123",
  async (): Promise<User> => {
    return await userRepository.findById("123")
  }
)