HESTIA Caching API

Getting started

To use this API and query the raster files, you need to download the assets first. Steps 2 to 5 are optional — take them only if you need altitude, climate data, or gadm-ids.

1. Download the HESTIA assets (~2.1 GB, required)

mkdir -p assets
curl -O https://hestia.earth/data/hestia-geospatial-assets.zip
unzip hestia-geospatial-assets.zip -d assets

This includes GMTED2010_1KM, elevation on the same 0.01° (~1.1 km) grid as the soil rasters — 164 MB rather than the 6.4 GB of step 2. Across 20,000 land points it is within 10 m of the native-resolution value 70% of the time and within 50 m 92% of the time, but that average hides the terrain: 1 m of error on flat ground, ~70 m in hilly country and ~140 m in mountains, where a 1.1 km cell genuinely has no single elevation.

It also includes MCD12Q1, the MODIS land cover layer hestia-data-validation checks a Site's declared siteType against — 169 MB, reprojected from MODIS's sinusoidal grid onto a 1/240° one. No model reads it; it is here so validation needs no Earth Engine account either. It reproduces Earth Engine exactly: 1000 of 1000 random land points agree, which scripts/export_modis_landcover.py verify re-measures against a live Earth Engine.

2. Download the elevation data (~6.4 GB, only for altitude)

This is what the altitude model reads — it asks for USGS/GMTED2010_FULL, so the 1 km raster from step 1 does not stand in for it; a caller has to name GMTED2010_1KM to get it. It is published separately because it is the largest single collection: GMTED2010 is 232 m pixels over the whole globe, ~11.6 billion of them, on a surface that barely compresses.

curl -O https://hestia.earth/data/hestia-geospatial-altitude.zip
unzip hestia-geospatial-altitude.zip -d assets

It unpacks into the same assets/ee folder as step 1, so nothing else changes. The raster is split into 18 tiles; the API treats them as one image. Skip it and altitude routes to Earth Engine as it did before, with no other effect.

3. Download the climate data (~10.6 GB, only for temperature and precipitation)

ERA5-Land, read by the precipitation* and temperature* models. Held as one raster per month, so it is published in 20-year archives — take only the periods you need.

# the long-term means: fetched for every Site, so take these if you take any
curl -O https://hestia.earth/data/hestia-geospatial-era5-longterm.zip
unzip hestia-geospatial-era5-longterm.zip -d assets

# monthly and annual values, ~2.8 GB per period
curl -O https://hestia.earth/data/hestia-geospatial-era5-2010-2025.zip
unzip hestia-geospatial-era5-2010-2025.zip -d assets

Periods are 20-year blocks: 1950-1969, 1970-1989, 1990-2009, and 2010-2025. The last is the block still being filled, and is named for the years it holds — so it becomes 2010-2026 when 2026 is published, superseding the URL above. Check this page for the current name.

Taking a subset is supported rather than merely tolerated: GET /geospatial/collections reports the years actually installed under periods, so a caller knows which years to ask elsewhere for. A query for a year you do not have returns 404 naming the archive period.

This is Copernicus data rather than an Earth Engine export, and is CC-BY — so unlike GADM below, we can redistribute it and you need no account.

4. Download the evapotranspiration data (~7.9 GB, only for potentialEvapotranspiration*)

TerraClimate pet, read by the potentialEvapotranspiration* models. Same layout as step 3 — one raster per month, published in the same 20-year periods, and a subset is supported the same way.

curl -O https://hestia.earth/data/hestia-geospatial-terraclimate-longterm.zip
unzip hestia-geospatial-terraclimate-longterm.zip -d assets

curl -O https://hestia.earth/data/hestia-geospatial-terraclimate-2010-2025.zip
unzip hestia-geospatial-terraclimate-2010-2025.zip -d assets

⚠️ These are not the numbers Earth Engine returns

This is TerraClimate v1.1 (WorldClim v2.1 + ERA5). Earth Engine's IDAHO_EPSCOR/TERRACLIMATE still serves v1.0 (CRU Ts4.0 + JRA55). Sampling pet for March 2003, every point tested differs — in both directions, by −12% to +18%:

| point | Earth Engine (v1.0) | here (v1.1) | difference | | --- | --- | --- | --- | | Brazil (−11.77, −45.7689) | 1146 | 1009 | −12.0% | | Netherlands (52.0, 5.0) | 495 | 454 | −8.3% | | India (23.28, 88.91) | 1247 | 1140 | −8.6% | | Kenya (−1.2, 36.8) | 1542 | 1623 | +5.3% | | US plains (40.0, −100.0) | 883 | 796 | −9.9% | | China (35.0, 105.0) | 488 | 576 | +18.0% | | Australia (−25.0, 134.0) | 1986 | 2109 | +6.2% |

This is a genuine change of input data, not a sampling artefact: the Earth Engine value is absent from the entire 7×7 neighbourhood, the ratio varies month to month so it is not a scale factor, and the masks agree exactly. The provider serves only the current release, so v1.0 cannot be retrieved — using these means accepting v1.1.

Take them if you want current science; v1.1 is the provider's own recommendation. Skip them if you need continuity with values already derived from v1.0 — leave the collection out and it keeps going to Earth Engine, with no other effect. The same is not true of ERA5 in step 3, which agrees with Earth Engine to within a few mm.

5. Download GADM boundaries (only for gadm-ids queries)

HESTIA cannot redistribute GADM, so it comes from the source. Below is a script to run in your terminal to download the files:

mkdir -p assets/gadm

for country in \
  ABW AFG AGO AIA ALA ALB AND ARE ARG ARM ASM ATA ATF ATG AUS AUT AZE \
  BDI BEL BEN BES BFA BGD BGR BHR BHS BIH BLM BLR BLZ BMU BOL BRA BRB \
  BRN BTN BVT BWA CAF CAN CCK CHE CHL CHN CIV CMR COD COG COK COL COM \
  CPV CRI CUB CUW CXR CYM CYP CZE DEU DJI DMA DNK DOM DZA ECU EGY ERI \
  ESH ESP EST ETH FIN FJI FLK FRA FRO FSM GAB GBR GEO GGY GHA GIB GIN \
  GLP GMB GNB GNQ GRC GRD GRL GTM GUF GUM GUY HKG HMD HND HRV HTI HUN \
  IDN IMN IND IOT IRL IRN IRQ ISL ISR ITA JAM JEY JOR JPN KAZ KEN KGZ \
  KHM KIR KNA KOR KWT LAO LBN LBR LBY LCA LIE LKA LSO LTU LUX LVA MAC \
  MAF MAR MCO MDA MDG MDV MEX MHL MKD MLI MLT MMR MNE MNG MNP MOZ MRT \
  MSR MTQ MUS MWI MYS MYT NAM NCL NER NFK NGA NIC NIU NLD NOR NPL NRU \
  NZL OMN PAK PAN PCN PER PHL PLW PNG POL PRI PRK PRT PRY PSE PYF QAT \
  REU ROU RUS RWA SAU SDN SEN SGP SGS SHN SJM SLB SLE SLV SMR SOM SPM \
  SRB SSD STP SUR SVK SVN SWE SWZ SXM SYC SYR TCA TCD TGO THA TJK TKL \
  TKM TLS TON TTO TUN TUR TUV TWN TZA UGA UKR UMI URY USA UZB VAT VCT \
  VEN VGB VIR VNM VUT WLF WSM XAD XCA XCL XKO XNC XPI XSP YEM ZAF ZMB \
  ZWE
do
  archive="assets/gadm/gadm36_${country}_shp.zip"
  [ -f "$archive" ] && continue
  curl -fsS -o "$archive" \
    "https://geodata.ucdavis.edu/gadm/gadm3.6/shp/gadm36_${country}_shp.zip" \
    || echo "failed: $country"
done

You can also download the files on https://gadm.org/download_country36.html (the "Shapefile").

Also for point-to-region lookups (~1.1 GB, optional)

The step above answers "what is the polygon for GADM id X". Two callers ask the opposite — "which region contains this point" — and that needs a global indexed layer, because choosing a per-country archive would mean already knowing the country. The region model asks at level 1; hestia-data-validation asks at whatever level a Site's own region or country id sits at, so anywhere from 0 to 5.

We cannot ship these, for the same licensing reason as above, so build them yourself:

# 1. the global layer, all levels (~1.1 GB download)
curl -O https://geodata.ucdavis.edu/gadm/gadm3.6/gadm36_levels_shp.zip
unzip gadm36_levels_shp.zip -d gadm36_levels

# 2. convert levels 0 and 1 to FlatGeobuf, into the asset folder
pip install -r requirements.txt
python scripts/build_gadm_vector.py --source gadm36_levels --levels 0 1

That writes assets/ee/gadm36_0.fgb (~530 MB) and assets/ee/gadm36_1.fgb (~610 MB).

Two levels cover all six. With level 0 present, the API answers levels 2 to 5 in two reads — country first, then that country's archive from step 5 — so those levels need no global layer of their own, which is where the disk goes: level 2 alone is tens of thousands of polygons against level 0's 256. Build a deeper level anyway if you would rather spend the disk than the second read; a global layer is always preferred when present.

Levels 2+ therefore need the country archives from step 5. A country you have not downloaded returns 404 naming the archive, and the caller falls back to Earth Engine — deliberately, rather than answering "this point is in no region", which hestia-data-validation would report as an error against correct data.

Skip this entirely and the lookups go to Earth Engine, as they did before.

6. Run the container

docker run -p 8000:80 \
  -e API_TOKEN=<token> \
  -e DATABASE_URL=<postgres-url> \
  -v "$(pwd)/assets:/app/assets:ro" \
  hestiae/hestia-caching-api

Allow ~800 MB of memory.

| Variable | Default | Purpose | | --- | --- | --- | | API_TOKEN | — | Admin token. Used to limit access to the whole API. Sent as the X-API-KEY header | | DATABASE_URL | — | PostgreSQL connection string | | GEOSPATIAL_FOLDER | assets/ee | Where the rasters are | | GADM_FOLDER | assets/gadm | Where the GADM archives are | | GDAL_CACHEMAX | 64 | Tile cache, MB. Left unset, GDAL takes 5% of host RAM and ignores the container limit | | GEOSPATIAL_CACHE_TTL | 86400 | How long, in seconds, the list of available collections is held before the assets folder is read again. 0 reads it on every request | | GEOSPATIAL_OPEN_ASSETS | 128 | How many rasters to keep open. Each holds a file descriptor, so with the climate data installed this must stay well under the container's nofile limit |

If you add assets to a running container, DELETE /geospatial/cache makes it pick them up immediately rather than after GEOSPATIAL_CACHE_TTL. It needs a token with the geospatial:write scope, and returns the collections as they then stand.

Measuring a GADM region

Two endpoints answer questions about a region itself rather than sampling data over it. Both read the country archives from step 5, and both 404 naming the archive when the country is not installed.

# area in km2 -- replaces earth_engine.gadm.get_size_km2
curl -X POST -H 'X-API-KEY: <token>' -H 'Content-Type: application/json' \
  -d '{"gadm-id": "GADM-LUX"}' <api>/geospatial/gadm/area

# distance in metres from a point, 0 when inside -- replaces
# earth_engine.gadm.get_distance_to_coordinates
curl -X POST -H 'X-API-KEY: <token>' -H 'Content-Type: application/json' \
  -d '{"gadm-id": "GADM-LUX", "latitude": 48.8566, "longitude": 2.3522}' \
  <api>/geospatial/gadm/distance

They agree with Earth Engine to within 0.35% on area and about a metre on distance, across every region tested. scripts/verify_gadm_measures.py re-measures that against a live Earth Engine rather than a fixture.

Note: a docker-compose.yml is available to get started locally quicker — it brings up the API with a PostgreSQL database. The full source is at gitlab.com/hestia-earth/hestia-caching-api.