nv-random-npm-pkg

A small utility library for generating random npm package data, primarily designed for testing a local in-memory npm registry implement

Overview

This project provides helpers to:

• Randomly sample package IDs from the npm registry
• Fetch package metadata and README content
• Iterate through the full npm dataset via _changes
• Generate realistic test data for registry implementations

The main goal is to support development and testing of a local in-memory npm server, especially one backed by mmap-based storage.

Use Case

This library is not intended for production use against the public registry at scale.

Instead, it is useful for:

• Testing custom npm registry implementations
• Benchmarking in-memory or mmap-based package stores
• Generating random real-world package data
• Stress-testing metadata parsing and storage layers

Features

• Random package ID generation (via CouchDB _all_docs)
• Metadata fetching from the official registry
• README extraction
• Full dataset traversal using _changes
• File export for offline indexing

Installation

npm install nv-random-npm-pkg

Usage

Get a random package

const rnd = require("nv-random-npm-pkg");

const pkg = await rnd();
console.log(pkg.name);

Get a random README

const { readme } = require("nv-random-npm-pkg");

const md = await readme();
console.log(md);

Get total document count

const { get_total_count } = require("nv-random-npm-pkg");

const count = await get_total_count();
console.log(count);

Iterate all package IDs

const { get_all_ids } = require("nv-random-npm-pkg");

await get_all_ids(512, "./ids.json");

Notes

• Many randomly sampled IDs are not valid packages (deleted or incomplete documents).

• The library includes retry and filtering logic, but success is probabilistic.

• For high-performance scenarios, it is recommended to:

  • Pre-build a local index
  • Store data using mmap or similar techniques

Architecture Hint

This library is often used together with:

• A local registry mirror
• A memory-mapped storage layer
• Zero-copy lookup structures (e.g. offset tables + buffers)

Limitations

• Depends on public npm registry endpoints
• Subject to network latency and rate limits
• Random sampling is not uniform over valid packages

License

MIT