npm package discovery and stats viewer.

Discover Tips

  • General search

    [free text search, go nuts!]

  • Package details

    pkg:[package-name]

  • User packages

    @[username]

Sponsor

Optimize Toolset

I’ve always been into building performant and accessible sites, but lately I’ve been taking it extremely seriously. So much so that I’ve been building a tool to help me optimize and monitor the sites that I build to make sure that I’m making an attempt to offer the best experience to those who visit them. If you’re into performant, accessible and SEO friendly sites, you might like it too! You can check it out at Optimize Toolset.

About

Hi, 👋, I’m Ryan Hefner  and I built this site for me, and you! The goal of this site was to provide an easy way for me to check the stats on my npm packages, both for prioritizing issues and updates, and to give me a little kick in the pants to keep up on stuff.

As I was building it, I realized that I was actually using the tool to build the tool, and figured I might as well put this out there and hopefully others will find it to be a fast and useful way to search and browse npm packages as I have.

If you’re interested in other things I’m working on, follow me on Twitter or check out the open source projects I’ve been publishing on GitHub.

I am also working on a Twitter bot for this site to tweet the most popular, newest, random packages from npm. Please follow that account now and it will start sending out packages soon–ish.

Open Software & Tools

This site wouldn’t be possible without the immense generosity and tireless efforts from the people who make contributions to the world and share their work via open source initiatives. Thank you 🙏

© 2026 – Pkg Stats / Ryan Hefner

@hestia-earth/caching-api

v0.2.8

Published

FastAPI application for caching Site data

Readme

HESTIA Caching API

Getting started

To use this API and query the raster files, you need to download the assets first. Steps 2 to 5 are optional — take them only if you need altitude, climate data, or gadm-ids.

1. Download the HESTIA assets (~2.1 GB, required)

mkdir -p assets
curl -O https://hestia.earth/data/hestia-geospatial-assets.zip
unzip hestia-geospatial-assets.zip -d assets

This includes GMTED2010_1KM, elevation on the same 0.01° (~1.1 km) grid as the soil rasters — 164 MB rather than the 6.4 GB of step 2. This is what the altitude model reads by default, and it exists only here: it is a HESTIA-made resampling, not an Earth Engine asset, so a caller without it falls back to the 232 m dataset of step 2 on Earth Engine. Across 20,000 land points it is within 10 m of the native-resolution value 70% of the time and within 50 m 92% of the time, but that average hides the terrain: 1 m of error on flat ground, ~70 m in hilly country and ~140 m in mountains, where a 1.1 km cell genuinely has no single elevation.

It also includes MCD12Q1, the MODIS land cover layer hestia-data-validation checks a Site's declared siteType against — 169 MB, reprojected from MODIS's sinusoidal grid onto a 1/240° one. No model reads it; it is here so validation needs no Earth Engine account either. It reproduces Earth Engine exactly: 1000 of 1000 random land points agree, which scripts/export_modis_landcover.py verify re-measures against a live Earth Engine.

SoilGrids instead of HWSD (~7.2 GB, optional)

The soil layers in this bundle come from the Harmonized World Soil Database, which is licensed for non-commercial use only. SoilGrids 2.0 (CC-BY 4.0) replacements are published separately, on the same grid, units and depths, under soilgrids_* names:

curl -O https://hestia.earth/data/hestia-geospatial-soilgrids.zip
unzip hestia-geospatial-soilgrids.zip -d assets

No model reads them yet, and their values differ from HWSD's — see SoilGrids in place of HWSD before switching.

2. Download the elevation data (~6.4 GB, only for altitude)

Only needed to serve altitude at its native 232 m resolution, which the models ask for under USGS/GMTED2010_FULL when MODELS_ALTITUDE_USE_FULL_PRECISION=true. The 1 km raster from step 1 does not stand in for it: a caller has to name USGS/GMTED2010_FULL to get it. It is published separately because it is the largest single collection: GMTED2010 is 232 m pixels over the whole globe, ~11.6 billion of them, on a surface that barely compresses.

curl -O https://hestia.earth/data/hestia-geospatial-altitude.zip
unzip hestia-geospatial-altitude.zip -d assets

It unpacks into the same assets/ee folder as step 1, so nothing else changes. The raster is split into 18 tiles; the API treats them as one image. Skip it and altitude routes to Earth Engine as it did before, with no other effect.

3. Download the climate data (~10.6 GB, only for temperature and precipitation)

ERA5-Land, read by the precipitation* and temperature* models. Held as one raster per month, so it is published in 20-year archives — take only the periods you need.

# the long-term means: fetched for every Site, so take these if you take any
curl -O https://hestia.earth/data/hestia-geospatial-era5-longterm.zip
unzip hestia-geospatial-era5-longterm.zip -d assets

# monthly and annual values, ~2.8 GB per period
curl -O https://hestia.earth/data/hestia-geospatial-era5-2010-2025.zip
unzip hestia-geospatial-era5-2010-2025.zip -d assets

Periods are 20-year blocks: 1950-1969, 1970-1989, 1990-2009, and 2010-2025. The last is the block still being filled, and is named for the years it holds — so it becomes 2010-2026 when 2026 is published, superseding the URL above. Check this page for the current name.

The 2010-2025 archive published before #16 carries incorrect precipitation for September 2022 – February 2024, taken from ECMWF's monthly averaged product, which is wrong for accumulations over those months. Download it again once the corrected block is published, then rewrite the cached values with python scripts/backfill_climate_cache.py --dataset era5-precipitation.

Taking a subset is supported rather than merely tolerated: GET /geospatial/collections reports the years actually installed under periods, so a caller knows which years to ask elsewhere for. A query for a year you do not have returns 404 naming the archive period.

This is Copernicus data rather than an Earth Engine export, and is CC-BY — so unlike GADM below, we can redistribute it and you need no account.

4. Download the evapotranspiration data (~7.9 GB, only for potentialEvapotranspiration*)

TerraClimate pet, read by the potentialEvapotranspiration* models. Same layout as step 3 — one raster per month, published in the same 20-year periods, and a subset is supported the same way.

curl -O https://hestia.earth/data/hestia-geospatial-terraclimate-longterm.zip
unzip hestia-geospatial-terraclimate-longterm.zip -d assets

curl -O https://hestia.earth/data/hestia-geospatial-terraclimate-2010-2025.zip
unzip hestia-geospatial-terraclimate-2010-2025.zip -d assets

⚠️ These are not the numbers Earth Engine returns

This is TerraClimate v1.1 (WorldClim v2.1 + ERA5). Earth Engine's IDAHO_EPSCOR/TERRACLIMATE still serves v1.0 (CRU Ts4.0 + JRA55). Sampling pet for March 2003, every point tested differs — in both directions, by −12% to +18%:

| point | Earth Engine (v1.0) | here (v1.1) | difference | | --- | --- | --- | --- | | Brazil (−11.77, −45.7689) | 1146 | 1009 | −12.0% | | Netherlands (52.0, 5.0) | 495 | 454 | −8.3% | | India (23.28, 88.91) | 1247 | 1140 | −8.6% | | Kenya (−1.2, 36.8) | 1542 | 1623 | +5.3% | | US plains (40.0, −100.0) | 883 | 796 | −9.9% | | China (35.0, 105.0) | 488 | 576 | +18.0% | | Australia (−25.0, 134.0) | 1986 | 2109 | +6.2% |

This is a genuine change of input data, not a sampling artefact: the Earth Engine value is absent from the entire 7×7 neighbourhood, the ratio varies month to month so it is not a scale factor, and the masks agree exactly. The provider serves only the current release, so v1.0 cannot be retrieved — using these means accepting v1.1.

Take them if you want current science; v1.1 is the provider's own recommendation. Skip them if you need continuity with values already derived from v1.0 — leave the collection out and it keeps going to Earth Engine, with no other effect. The same is not true of ERA5 in step 3, which agrees with Earth Engine to within a few mm.

5. Download GADM boundaries (only for gadm-ids queries)

HESTIA cannot redistribute GADM, so it comes from the source. Below is a script to run in your terminal to download the files:

mkdir -p assets/gadm

for country in \
  ABW AFG AGO AIA ALA ALB AND ARE ARG ARM ASM ATA ATF ATG AUS AUT AZE \
  BDI BEL BEN BES BFA BGD BGR BHR BHS BIH BLM BLR BLZ BMU BOL BRA BRB \
  BRN BTN BVT BWA CAF CAN CCK CHE CHL CHN CIV CMR COD COG COK COL COM \
  CPV CRI CUB CUW CXR CYM CYP CZE DEU DJI DMA DNK DOM DZA ECU EGY ERI \
  ESH ESP EST ETH FIN FJI FLK FRA FRO FSM GAB GBR GEO GGY GHA GIB GIN \
  GLP GMB GNB GNQ GRC GRD GRL GTM GUF GUM GUY HKG HMD HND HRV HTI HUN \
  IDN IMN IND IOT IRL IRN IRQ ISL ISR ITA JAM JEY JOR JPN KAZ KEN KGZ \
  KHM KIR KNA KOR KWT LAO LBN LBR LBY LCA LIE LKA LSO LTU LUX LVA MAC \
  MAF MAR MCO MDA MDG MDV MEX MHL MKD MLI MLT MMR MNE MNG MNP MOZ MRT \
  MSR MTQ MUS MWI MYS MYT NAM NCL NER NFK NGA NIC NIU NLD NOR NPL NRU \
  NZL OMN PAK PAN PCN PER PHL PLW PNG POL PRI PRK PRT PRY PSE PYF QAT \
  REU ROU RUS RWA SAU SDN SEN SGP SGS SHN SJM SLB SLE SLV SMR SOM SPM \
  SRB SSD STP SUR SVK SVN SWE SWZ SXM SYC SYR TCA TCD TGO THA TJK TKL \
  TKM TLS TON TTO TUN TUR TUV TWN TZA UGA UKR UMI URY USA UZB VAT VCT \
  VEN VGB VIR VNM VUT WLF WSM XAD XCA XCL XKO XNC XPI XSP YEM ZAF ZMB \
  ZWE
do
  archive="assets/gadm/gadm36_${country}_shp.zip"
  [ -f "$archive" ] && continue
  curl -fsS -o "$archive" \
    "https://geodata.ucdavis.edu/gadm/gadm3.6/shp/gadm36_${country}_shp.zip" \
    || echo "failed: $country"
done

You can also download the files on https://gadm.org/download_country36.html (the "Shapefile").

Also for point-to-region lookups (~1.1 GB, optional)

The step above answers "what is the polygon for GADM id X". Two callers ask the opposite — "which region contains this point" — and that needs a global indexed layer, because choosing a per-country archive would mean already knowing the country. The region model asks at level 1; hestia-data-validation asks at whatever level a Site's own region or country id sits at, so anywhere from 0 to 5.

We cannot ship these, for the same licensing reason as above, so build them yourself:

# 1. the global layer, all levels (~1.1 GB download)
curl -O https://geodata.ucdavis.edu/gadm/gadm3.6/gadm36_levels_shp.zip
unzip gadm36_levels_shp.zip -d gadm36_levels

# 2. convert levels 0 and 1 to FlatGeobuf, into the asset folder
pip install -r requirements.txt
python scripts/build_gadm_vector.py --source gadm36_levels --levels 0 1

That writes assets/ee/gadm36_0.fgb (~530 MB) and assets/ee/gadm36_1.fgb (~610 MB).

Two levels cover all six. With level 0 present, the API answers levels 2 to 5 in two reads — country first, then that country's archive from step 5 — so those levels need no global layer of their own, which is where the disk goes: level 2 alone is tens of thousands of polygons against level 0's 256. Build a deeper level anyway if you would rather spend the disk than the second read; a global layer is always preferred when present.

Levels 2+ therefore need the country archives from step 5. A country you have not downloaded returns 404 naming the archive, and the caller falls back to Earth Engine — deliberately, rather than answering "this point is in no region", which hestia-data-validation would report as an error against correct data.

Skip this entirely and the lookups go to Earth Engine, as they did before.

Or geoBoundaries instead of GADM (optional)

Set ADMIN_BOUNDARY_SOURCE=geoboundaries to answer admin-boundary queries from geoBoundaries (gbOpen) instead of GADM, whose licence forbids commercial use. In that mode:

  • gadm36_0 point lookups, country gadm-ids, /gadm/area and /gadm/distance answer from the geoBoundaries country layer, with the same HESTIA ids and payloads;
  • sub-national GADM ids and gadm36_1 to gadm36_5 are refused with a 422, and no GADM file is read even when one is on disk;
  • GET /geospatial/collections reports the mode as adminBoundarySource;
  • a cached Site written under the other source is not served, and is replaced on its next write.

Download the published build (~640 MB, ~1.3 GB unpacked) into the same assets folder:

curl -O https://hestia.earth/data/hestia-geospatial-geoboundaries.zip
unzip hestia-geospatial-geoboundaries.zip -d assets

Or build it yourself. Everything comes from one pinned geoBoundaries commit, the one the glossary's region-geoBoundaries lookup names in its .md sidecar, which the build checks:

python scripts/build_geoboundaries.py --commit <40-character geoBoundaries SHA>

# against a lookup not published yet
python scripts/build_geoboundaries.py --commit <sha> --lookup <region-geoBoundaries-lookup.csv>

# rebuild one part, keeping the other from the same commit
python scripts/build_geoboundaries.py --commit <sha> --only countries --lookup <csv>
python scripts/build_geoboundaries.py --commit <sha> --only units --countries LUX FRA

Into assets/geoboundaries that writes:

  • countries.fgb (~200 MB) and countries/<id>.fgb: one feature per HESTIA country the lookup gives a geometry, the union of its units less the territories it carves out. The 19 countries with none (Jersey, Saint-Martin and others) resolve to whichever country surrounds them;
  • units/<ISO>_ADM<level>.fgb: geoBoundaries' own simplified geometry for every sub-national layer, except those under "Other - Direct Permission" or "Other - Humanitarian";
  • manifest.json: the commit, the citation, and every layer's and country's licence and source.

shapeID changes between geoBoundaries builds, so the commit and the lookup are the only inputs, and every file is checked against the sha256 the commit records. A lookup row naming a unit the commit does not have stops the build. The CGAZ downloads (~300 MB) are kept in assets/geoboundaries/.cache for the next run; --cache puts them elsewhere. python scripts/publish_geospatial_assets.py --archive geoboundaries --upload --mirror publishes the build, leaving the cache out. The layers are served in either mode:

# a country's levels, with each layer's unit count, licence and source
curl -H 'X-API-KEY: <token>' <api>/geospatial/geoboundaries/FRA

# the units at one level, without geometry
curl -H 'X-API-KEY: <token>' <api>/geospatial/geoboundaries/FRA/2

# one unit's simplified polygon as a GeoJSON Feature, with its area in km2, licence and source
curl -H 'X-API-KEY: <token>' <api>/geospatial/geoboundaries/FRA/2/<shapeID>

The country is the geoBoundaries shapeGroup, an ISO 3166-1 alpha-3 code. Attribute geoBoundaries as Runfola et al. (2020), PLoS ONE 15(4): e0231866, and each layer under its own licence.

6. Run the container

docker run -p 8000:80 \
  -e API_TOKEN=<token> \
  -e DATABASE_URL=<postgres-url> \
  -v "$(pwd)/assets:/app/assets:ro" \
  hestiae/hestia-caching-api

Allow ~800 MB of memory.

The image is published for linux/amd64 and linux/arm64 under the same tag, and Docker pulls the one matching the host. Tags up to 0.2.1 are amd64 only.

| Variable | Default | Purpose | | --- | --- | --- | | API_TOKEN | — | Required: the API refuses to start without it. Admin token. Used to limit access to the whole API. Sent as the X-API-KEY header | | DATABASE_URL | — | PostgreSQL connection string | | GEOSPATIAL_FOLDER | assets/ee | Where the rasters are | | GADM_FOLDER | assets/gadm | Where the GADM archives are | | ADMIN_BOUNDARY_SOURCE | gadm | geoboundaries answers countries from geoBoundaries and refuses sub-national GADM ids. Any other value fails at start-up | | GEOBOUNDARIES_FOLDER | assets/geoboundaries | Where the geoBoundaries dataset is | | GDAL_CACHEMAX | 64 | Tile cache, MB. Left unset, GDAL takes 5% of host RAM and ignores the container limit | | GEOSPATIAL_CACHE_TTL | 86400 | How long, in seconds, the list of available collections is held before the assets folder is read again. 0 reads it on every request | | DATASETS_INDEX_URL | https://hestia.earth/data/hestia-geospatial-index.json | The index of published archives GET /datasets serves | | DATASETS_INDEX_TTL | 3600 | How long, in seconds, the index is held before it is fetched again. A failed fetch keeps serving the last one | | GEOSPATIAL_OPEN_ASSETS | 128 | How many rasters to keep open. Each holds a file descriptor, so with the climate data installed this must stay well under the container's nofile limit | | GEOSPATIAL_MAX_REDUCTIONS | 2 | Boundary and GADM reductions running at once, across all requests; the rest wait. One five-year query peaks at ~660 MB, so size it against the container's memory limit. Coordinate samples, vector lookups and the listing are not counted |

If you add assets to a running container, DELETE /geospatial/cache makes it pick them up immediately rather than after GEOSPATIAL_CACHE_TTL. It needs a token with the geospatial:write scope, and returns the collections as they then stand.

Datasets and their licences

# every published archive: URL, sizes, collections, years, licences, whether it is
# in the recommended preset, and whether it is installed here (complete, partial, missing)
curl -H 'X-API-KEY: <token>' <api>/datasets

# for each collection this instance holds: dataset name, models, citations and licence
curl -H 'X-API-KEY: <token>' <api>/geospatial/collections/sources

The recommended preset is the main bundle with the ERA5 and TerraClimate long-term means. Licences live in scripts/asset-bibliography.json with each collection's citation; one nobody has confirmed yet reads TO BE CONFIRMED. Publishing ships that file with the assets as sources.json, inside every archive and in the mirror, and the API reads it from GEOSPATIAL_FOLDER, so a new dataset or a corrected licence needs no release: after editing the file, python scripts/publish_geospatial_assets.py --sources --upload copies it to the mirror and to https://hestia.earth/data/hestia-geospatial-sources.json, and instances serve it once their asset folder has it, from their next sync or from fetching that URL into GEOSPATIAL_FOLDER. An asset folder without sources.json falls back to the copy in the image. With ADMIN_BOUNDARY_SOURCE=geoboundaries, gadm36_0 is credited to geoBoundaries from the dataset's manifest.json instead, with each country's licence, and publishing an archive warns about it. After publishing, python scripts/build_dataset_index.py --upload rebuilds the index from the zips in the bucket, reading each one's file list without downloading it.

Measuring a GADM region

Two endpoints answer questions about a region itself rather than sampling data over it. Both read the country archives from step 5, and both 404 naming the archive when the country is not installed.

# area in km2 -- replaces earth_engine.gadm.get_size_km2
curl -X POST -H 'X-API-KEY: <token>' -H 'Content-Type: application/json' \
  -d '{"gadm-id": "GADM-LUX"}' <api>/geospatial/gadm/area

# distance in metres from a point, 0 when inside -- replaces
# earth_engine.gadm.get_distance_to_coordinates
curl -X POST -H 'X-API-KEY: <token>' -H 'Content-Type: application/json' \
  -d '{"gadm-id": "GADM-LUX", "latitude": 48.8566, "longitude": 2.3522}' \
  <api>/geospatial/gadm/distance

They agree with Earth Engine to within 0.35% on area and about a metre on distance, across every region tested. scripts/verify_gadm_measures.py re-measures that against a live Earth Engine rather than a fixture.

Note: a docker-compose.yml is available to get started locally quicker — it brings up the API with a PostgreSQL database. The full source is at gitlab.com/hestia-earth/hestia-caching-api.