reproduce

Can we reproduce a package with the "origin" information provided?

Features · How It Works · Configuration · Strategies · Usage · Insights · FAQs

Features

• ✅ determines whether or not a package can be reproduced from it's referenced repository metadata (ie. repository, repository.type, repository.url, repository.directory & gitHead)
• 🔍 validates repository information against package.json if the package referenced lives on a registry (will fallback to package.json inside the tarball if the package is not in a registry)
  • 🔀 mismatching repository information is considered "manifest confusion" & will return false for "reproducibility"
• 🗄️ provides persistent caching of results
• 🔄 currently only supports npm as a "strategy" but will expand to support other package managers in the future

How It Works

1. ⬇️ fetches the package & any corresponding metadata
2. 📂 if available, does a clone/checkout of the corresponding source repository
3. 🔄 attempts to prepare & pack the source repository using one or more strategies
4. 🔍 validates the integrity value of #3 against the package fetched in #1
5. 📄 returns results and caches them for future use

Usage

$ npm i -g reproduce # install globally
$ reproduce axios

$ npx reproduce axios # execute with npx

import reproduce from 'reproduce'

// Basic usage
const result = await reproduce('package-name')

// With custom configuration
const result = await reproduce('package-name', {
  cache: {},
  cacheDir: './custom-cache',
  cacheFile: 'custom-cache.json'
})

CLI

reproduce tsc # exit code 0 - reproducible

reproduce esbuild # exit code 1 - not reproducible

reproduce axios --json  # exit code 1 - not reproducible
{
  "reproduceVersion": "0.0.1-pre.1",
  "timestamp": "2025-02-25T10:40:24.947Z",
  "os": "darwin",
  "arch": "arm64",
  "strategy": "npm:10.9.1",
  "reproduced": false,
  "package": {
    "spec": "axios@latest",
    "name": "axios",
    "version": "1.2.3",
    "location": "https://registry.npmjs.org/axios/-/axios-1.7.9.tgz",
    "integrity": "sha512-LhLcE7Hbiryz8oMDdDptSrWowmB4Bl6RCt6sIJKpRB4XtVf0iEgewX3au/pJqm+Py1kCASkb/FFKjxQaLtxJvw=="
  },
  "source": {
    "spec": "github:axios/axios#b2cb45d5a533a5465c99559b16987e4d5fc08cbc",
    "name": "axios",
    "version": "1.2.3",
    "location": "git+https://github.com/axios/axios.git",
    "integrity": "null"
  },
  "diff": "..."
}

reproduce axios --force  # force revalidation, bypass cache

CLI Options

| Option | Short | Description | |--------|-------|-------------| | --strategy | -s | Choose a strategy (default: "npm") | | --json | -j | Output result as JSON | | --force | -f | Force revalidation, bypass cache | | --help | -h | Print usage information |

reproduce require --json  # exit code 0 - reproducible
{
  "reproduceVersion": "0.0.1-pre.1",
  "timestamp": "2025-02-25T10:22:09.303Z",
  "os": "darwin",
  "arch": "arm64",
  "strategy": "npm:10.9.1",
  "reproduced": true,
  "package": {
    "spec": "sleepover@latest",
    "version": "1.2.3",
    "location": "https://registry.npmjs.org/sleepover/-/sleepover-1.2.3.tgz",
    "integrity": "sha512-yNAIVUqbQifyy5+hfzAzK2Zt21wXjwXqPyWLu+tOvhOcYKG2ffUiSoBXwt/yo4KJ51IcJfUS0Uq0ktOoMWy9Yw=="
  },
  "source": {
    "spec": "github:darcyclarke/sleepover#f2586e91b3faf085583c23ed6e00819916e85c28",
    "version": "1.2.3",
    "location": "git+ssh://[email protected]/darcyclarke/sleepover.git",
    "integrity": "sha512-yNAIVUqbQifyy5+hfzAzK2Zt21wXjwXqPyWLu+tOvhOcYKG2ffUiSoBXwt/yo4KJ51IcJfUS0Uq0ktOoMWy9Yw=="
  }
}

Configuration

The reproduce function accepts an options object with the following configuration:

{
  cache: {},                      // Optional in-memory cache object (persisted to disk if provided)
  cacheDir: '~/.cache/reproduce', // OS-specific cache directory
  cacheFile: 'cache.json',        // Cache file name
  strategy: 'npm'                 // Strategy to use
}

Cache Locations

The cache is stored in OS-specific locations:

• macOS: ~/Library/Caches/reproduce/
• Windows: %LOCALAPPDATA%/reproduce/Cache/
• Linux: $XDG_CACHE_HOME/reproduce/ or ~/.cache/reproduce/

Strategies

A strategy is a set of operations to take to recreate a package. Strategies should represent common patterns for preparing/building/packing packages to cast wide nets. If a set successfully recreates a package then its ID will be stored inside the returned metadata.

| Name | UUID | Description | | --- | --- | | npm npm:<version> | clones, checks out ref, installs deps & then runs pack |

Note: one-off/bespoke or complex configurations will not be supported but we will continue to add more strategies as we find common patterns.

Insights

Top 5,000 High Impact Packages

Note: "High Impact" packages are defined as having >=1M downloads per week and/or >=500 dependants. This list was originally generated here. This test was run on 2025-02-26.

• 5.78% (289) are reproducible
• 3.72% (186) have provenance

FAQs

Why look into "reproducibility"?

We believe the strategy of leveraging reproducible builds for the purpose of associating artifacts with a source/repository outperforms the current provenance strategy with the added benefit of being backwards compatible.

Will reproducibility get better with time?

Yes. As we add more strategies, we should see the percentatge of reproducible packages grow over time both net-new & previously published packages will benefit from the additional strategies. Feel free to contribute!

Credits

Big thanks to @siddharthkp for gifting the package name reproduce to us!

Learn More

We wrote a blog post about this project & the results we found which you can read here: https://blog.vlt.sh/blog/reproducibility