npm package discovery and stats viewer.

Discover Tips

  • General search

    [free text search, go nuts!]

  • Package details

    pkg:[package-name]

  • User packages

    @[username]

Sponsor

Optimize Toolset

I’ve always been into building performant and accessible sites, but lately I’ve been taking it extremely seriously. So much so that I’ve been building a tool to help me optimize and monitor the sites that I build to make sure that I’m making an attempt to offer the best experience to those who visit them. If you’re into performant, accessible and SEO friendly sites, you might like it too! You can check it out at Optimize Toolset.

About

Hi, 👋, I’m Ryan Hefner  and I built this site for me, and you! The goal of this site was to provide an easy way for me to check the stats on my npm packages, both for prioritizing issues and updates, and to give me a little kick in the pants to keep up on stuff.

As I was building it, I realized that I was actually using the tool to build the tool, and figured I might as well put this out there and hopefully others will find it to be a fast and useful way to search and browse npm packages as I have.

If you’re interested in other things I’m working on, follow me on Twitter or check out the open source projects I’ve been publishing on GitHub.

I am also working on a Twitter bot for this site to tweet the most popular, newest, random packages from npm. Please follow that account now and it will start sending out packages soon–ish.

Open Software & Tools

This site wouldn’t be possible without the immense generosity and tireless efforts from the people who make contributions to the world and share their work via open source initiatives. Thank you 🙏

© 2025 – Pkg Stats / Ryan Hefner

sanity-export

v1.0.6

Published

One-command solution to clone public Sanity datasets with parallel asset uploads and automatic import

Vulnerabilities

Powered byPowered by Snyk
  • 0High
  • 0Medium
  • 0Low

Links

Git

Maintainers

ajay-developsajay-develops

Keywords

sanityexportimportdatasetbackupclonemigrationassetscmssanity-io

Readme

Sanity Export

npm version npm downloads License: MIT

Truly one-command solution! Export public Sanity datasets and import them to your project with all assets and documents automatically uploaded.

⚠️ IMPORTANT: This CLI tool must be run from your target Sanity project directory (where sanity is installed as a dependency). The tool uses the official Sanity CLI for importing, so it needs access to the sanity command.

cd ~/your-sanity-project  # Navigate to your Sanity project first
npx sanity-export         # Then run the tool

Features

  • ✅ Fully automated - Export, upload assets, AND import documents in one command
  • ✅ Parallel uploads - Upload 10 assets simultaneously for 10x faster performance
  • ✅ No manual import needed - Tool handles everything via Sanity API
  • ✅ No source auth required - Works with public datasets
  • ✅ Assets properly uploaded - All images/files uploaded to your project
  • ✅ CLI + npm package - Use with npx or install globally
  • ✅ Interactive mode - Just run npx sanity-export and answer prompts
  • ✅ Configurable concurrency - Adjust parallel uploads (1-15) for your network
  • ✅ Progress tracking - See download, upload, and import progress in real-time

Prerequisites

  • Node.js 18 or higher
  • A public Sanity dataset (the source you want to clone from)
  • A target Sanity project (where Sanity is installed as a dependency)
  • Sanity auth token for your target project

🚀 Quick Start (CLI Method - Recommended)

⚠️ Important: Run from Your Target Sanity Project Directory

The CLI must be run from your target Sanity project directory (where sanity is installed as a dependency). This ensures the sanity dataset import command works correctly.

# Navigate to your Sanity project first
cd ~/your-sanity-project

# Then run the CLI
npx sanity-export

Choose Your Approach:

🎯 Option 1: Interactive Mode (Recommended for Most Users)

The easiest way! Just answer the prompts:

npx sanity-export

The tool will ask for:

  • Source project ID (the public dataset)
  • Target project ID (your project)
  • Your Sanity auth token
  • Dataset names (defaults to 'production')
  • Output directory
  • Concurrency (defaults to 10)

Then it does everything automatically! ✅

⚡ Option 2: One-Line Command (For Advanced Users/Scripts)

If you prefer command-line flags or need automation:

npx sanity-export clone \
  --source-project r7m53vrk \
  --target-project your-project-id \
  --token sk_your_sanity_token

Both approaches do the same thing:

  1. ✅ Exports all documents from the source
  2. ✅ Downloads all assets (images/files)
  3. ✅ Uploads assets in parallel to your target project (10x faster!)
  4. ✅ Updates all references
  5. ✅ Runs sanity dataset import with your credentials
  6. ✅ Complete! Dataset ready to use!

No manual steps required! The tool sets environment variables and runs the official Sanity CLI for you! 🎉

Get Your Auth Token

  1. Go to https://sanity.io/manage
  2. Select your project
  3. Click API → Tokens → Add API token
  4. Permissions: Editor
  5. Copy the token (starts with sk)

📦 Installation

Use with npx (No Installation)

npx sanity-export clone --source-project abc --target-project xyz --token sk_...

Global Installation

npm install -g sanity-export
sanity-export clone --source-project abc --target-project xyz --token sk_...

Local Development

git clone https://github.com/ajay-develops/export-import-sanity-public-dataset-without-source-project-api-token.git
cd export-import-sanity-public-dataset-without-source-project-api-token
npm install
npm link  # Makes CLI available globally

💻 CLI Commands

Default: Interactive Mode ⭐ (Recommended for First-Time Users)

Just run without any arguments:

npx sanity-export

The tool will prompt you for:

  • Source project ID
  • Target project ID
  • Auth token
  • Datasets (defaults provided)
  • Output directory
  • Concurrency (parallel uploads)

This is the easiest way to use the tool! No need to remember flags or syntax. 🎯

clone - One-Line Complete Solution (Recommended for Automation/Scripts)

For advanced users or CI/CD pipelines. Fully automated with command-line flags:

npx sanity-export clone \
  --source-project r7m53vrk \
  --target-project yfxnt4n8 \
  --token sk_your_token

What it does automatically:

  1. Exports from source
  2. Downloads assets
  3. Uploads assets to target (in parallel!)
  4. Updates references
  5. Runs sanity dataset import automatically (sets credentials via env vars)
  6. Done! Dataset ready to use! 🎉

Use this if you:

  • Want a scriptable/repeatable command
  • Are setting up CI/CD automation
  • Prefer command-line flags over interactive prompts

Otherwise, use interactive mode (simpler): npx sanity-export ⭐

Options:

  • -s, --source-project <id> - Source project ID (required)
  • --source-dataset <name> - Source dataset (default: production)
  • -t, --target-project <id> - Target project ID (required)
  • --target-dataset <name> - Target dataset (default: production)
  • -k, --token <token> - Sanity auth token (required)
  • -o, --output <dir> - Output directory (default: current directory)
  • -c, --concurrency <number> - Parallel uploads (default: 10, max: 15)

interactive - Explicit Interactive Mode

Same as running with no arguments:

npx sanity-export interactive

export - Export Only

Just export without uploading assets:

npx sanity-export export \
  --source-project r7m53vrk \
  --output ./my-export

prepare-import - Prepare Existing Export

Upload assets and prepare import for an existing export:

npx sanity-export prepare-import \
  --target-project yfxnt4n8 \
  --token sk_your_token \
  --input ./my-export

📁 Output

After running clone or export, you'll have:

  • production.tar.gz - Ready-to-import archive
  • data-updated.ndjson - Documents with updated asset references
  • asset-id-mapping.json - Old → New asset ID mapping
  • assets-metadata.json - Asset metadata
  • images/ - Downloaded image files
  • files/ - Downloaded file assets

📖 How It Works

Visual Workflow

┌─────────────────────────────────────────────────────────────────┐
│                    SOURCE PROJECT (Public)                       │
│                    r7m53vrk / production                         │
│  📄 Documents + 🖼️ Assets                                        │
└────────────┬────────────────────────────────────────────────────┘
             │
             │ npm run export
             │ (downloads via public API)
             ▼
┌─────────────────────────────────────────────────────────────────┐
│                    LOCAL MACHINE                                 │
│  📁 data.ndjson (163 docs, no asset docs)                       │
│  📁 assets-metadata.json (392 asset metadata)                   │
│  📁 images/ (389 files)                                         │
│  📁 files/ (3 files)                                            │
└────────────┬────────────────────────────────────────────────────┘
             │
             │ npm run prepare-import
             │ (uploads assets, updates refs)
             │
             ├──────────────────────┐
             ▼                      ▼
    ┌─────────────────┐    ┌──────────────────┐
    │ Upload Assets   │    │ Update Refs      │
    │ to Target       │    │ OLD → NEW IDs    │
    └────────┬────────┘    └────────┬─────────┘
             │                      │
             ▼                      ▼
┌─────────────────────────────────────────────────────────────────┐
│                  TARGET PROJECT (Yours)                          │
│                  yfxnt4n8 / production                           │
│  🖼️ 392 Assets uploaded ✅                                       │
└─────────────────────────────────────────────────────────────────┘
             │
             │ Auto-import via Sanity CLI
             │ (SANITY_PROJECT_ID=yfxnt4n8 sanity dataset import)
             ▼
┌─────────────────────────────────────────────────────────────────┐
│              TARGET PROJECT (Complete)                           │
│              yfxnt4n8 / production                               │
│  📄 163 Documents ✅ (auto-imported via CLI)                     │
│  🖼️ 392 Assets ✅ (parallel uploaded)                            │
│  🔗 All references working ✅                                    │
│  🎉 Dataset ready to use!                                        │
└─────────────────────────────────────────────────────────────────┘

🛠️ Alternative: npm Scripts Method

If you cloned the repo locally, you can use npm scripts:

# 1. Configure .env.local
cp .env.example .env.local
nano .env.local  # Add SOURCE_PROJECT_ID, TARGET_PROJECT_ID, SANITY_AUTH_TOKEN

# 2. Run export
npm run export

# 3. Prepare import
npm run prepare-import

# 4. Import
cd ~/your-sanity-project
pnpm dlx sanity dataset import ./export-import-sanity-public-dataset-without-source-project-api-token/production.tar.gz --replace

See the npm scripts section for all available commands.

📝 Examples

⚠️ All examples should be run from your target Sanity project directory:

cd ~/your-sanity-project  # Navigate to your Sanity project first!

Example 1: Interactive Mode (Easiest - Recommended!) 🎯

Just run and answer the prompts:

npx sanity-export

When to use:

  • First time using the tool
  • Don't want to remember command flags
  • Want guided step-by-step process

Example 2: One-Line Clone with Flags (Advanced) ⚡

npx sanity-export clone \
  --source-project r7m53vrk \
  --target-project yfxnt4n8 \
  --token sk_your_token_here

When to use:

  • You need a repeatable command
  • Setting up automation/scripts
  • Running in CI/CD pipeline

Example 3: Custom Output Directory

npx sanity-export clone \
  --source-project r7m53vrk \
  --target-project yfxnt4n8 \
  --token sk_your_token \
  --output ./my-dataset-backup

Example 4: Different Source Dataset

npx sanity-export clone \
  --source-project r7m53vrk \
  --source-dataset staging \
  --target-project yfxnt4n8 \
  --token sk_your_token

Example 5: Export Only (No Upload)

npx sanity-export export \
  --source-project r7m53vrk \
  --output ./backup

Example 6: Custom Concurrency (Faster Uploads!)

npx sanity-export clone \
  --source-project r7m53vrk \
  --target-project yfxnt4n8 \
  --token sk_token \
  --concurrency 15

Performance:

  • --concurrency 1 - Sequential (slowest, ~20 minutes)
  • --concurrency 5 - Conservative (safe, ~10-12 minutes)
  • --concurrency 10 - Default (fast, ~5-8 minutes) ⭐
  • --concurrency 15 - Maximum (fastest, ~3-5 minutes)

Note: Higher concurrency = faster uploads but requires stable network. Start with default (10) and adjust based on your connection.

🔧 npm Scripts

For local development or custom workflows:

  • npm run export - Export documents and assets from source project
  • npm run prepare-import - ⭐ Upload assets, update references, create import package
  • npm run upload-assets - Upload assets only (for manual workflows)
  • npm run transform - Re-run transformation on existing data
  • npm run clean - Remove all generated files

How It Works

Export Process (npm run export)

  1. Downloads documents from Sanity's public API
  2. Downloads assets (images and files) from their URLs
  3. Removes asset documents from data.ndjson and saves metadata
  4. Packages everything into production.tar.gz

Import Process (Fully Automated!)

  1. Uploads assets in parallel (10 concurrent) via Sanity API
  2. Gets new asset IDs from Sanity's response
  3. Updates references in documents to use new IDs
  4. Creates final package with updated documents
  5. Auto-runs sanity dataset import with credentials set via environment variables:
    SANITY_PROJECT_ID=your-project \
SANITY_AUTH_TOKEN=your-token \
npx sanity dataset import production.tar.gz --replace

Why this works:

  • Assets are uploaded FIRST, so all references resolve immediately
  • Official Sanity CLI is used for import (reliable and standard)
  • Environment variables provide authentication (no prompts needed)

Limitations

  • Public datasets only - This tool works without authentication, so it can only access public datasets
  • No drafts - Draft documents are private and won't be included
  • No private content - Any content marked as private won't be exported

Troubleshooting

⚠️ CLI Must Run from Sanity Project Directory

Issue: "sanity: command not found" or import fails

Solution: Run the CLI from your target Sanity project directory (where sanity is installed):

# ✅ CORRECT - From your Sanity project
cd ~/your-sanity-project
npx sanity-export clone -s SOURCE -t TARGET -k TOKEN

# ❌ WRONG - From random directory
cd ~/Downloads
npx sanity-export  # Will fail at import step!

The tool needs access to the Sanity CLI to run sanity dataset import.

Memory Issues

Issue: JavaScript heap out of memory

Solution:

  • Use the latest version: npx sanity-export@latest
  • Reduce concurrency: --concurrency 5 (instead of 10)
  • Ensure Node.js is up to date: node --version (18+ required)

During Export

"Dataset not found"

  • Ensure dataset is public in source project settings
  • Verify SOURCE_PROJECT_ID in .env.local is correct

"Assets fail to download"

  • Check network connection
  • Script continues with other assets if some fail

During prepare-import

"TARGET_PROJECT_ID is required"

  • Check .env.local has TARGET_PROJECT_ID, SANITY_AUTH_TOKEN

"Upload failed with status 401"

  • Auth token is invalid or lacks Editor permissions
  • Get new token from https://sanity.io/manage

"Upload failed with status 404"

  • Dataset doesn't exist in target project
  • Create it: pnpm dlx sanity dataset create production

During Import

Import succeeds but images don't show

  • Check prepare-import completed successfully
  • Verify assets were uploaded (check console output)
  • Run GROQ query: count(*[_type == "sanity.imageAsset"])

Project Structure

export-sanity-public-dataset/
├── package.json                # Dependencies and scripts
├── .env.example               # Environment variable template
├── .env.local                 # Your configuration (git-ignored)
├── index.js                   # Export orchestrator
├── src/
│   ├── download-documents.js  # Downloads NDJSON from Sanity
│   ├── download-assets.js     # Downloads asset files
│   ├── transform-assets.js    # Removes asset docs, saves metadata
│   ├── package-export.js      # Creates tar.gz archive
│   ├── prepare-import.js      # Uploads assets, updates refs, packages
│   └── upload-assets.js       # Alternative asset upload script
└── README.md                  # This file

🤝 Contributing

Contributions are welcome! Please feel free to submit a Pull Request.

Ideas for contributions:

  • Progress bars for uploads
  • Retry logic for failed uploads
  • Support for private datasets
  • Dry-run mode
  • Enhanced error messages

📤 Publishing (For Maintainers)

Quick Publish Command

To publish a new version to npm and GitHub:

# For bug fixes (1.0.0 → 1.0.1)
npm version patch && git push origin main --tags

# For new features (1.0.0 → 1.1.0)
npm version minor && git push origin main --tags

# For breaking changes (1.0.0 → 2.0.0)
npm version major && git push origin main --tags

This will:

  1. Update version in package.json
  2. Create a git tag
  3. Push to GitHub
  4. Trigger GitHub Actions to auto-publish to npm 🤖

See: GITHUB-ACTIONS-SETUP.md for complete setup guide

📄 License

MIT

🔗 Resources