Prisma Sharding

Lightweight database sharding library for Prisma with durable ownership, health monitoring, and CLI tools.

Features

• Prisma 7 support
• PostgreSQL shards
• Durable routing-key ownership
• Stable shard resolution
• Weighted placement for new records
• Shard draining
• Health monitoring and automatic recovery
• Safe bounded cross-shard search
• Multi-shard database updates
• Multi-shard Prisma Studio
• Read-only shard diagnostics
• TypeScript support
• Custom ownership directory for advanced systems

Installation

npm install prisma-sharding @prisma/client@^7 prisma@^7 @prisma/adapter-pg pg

or:

yarn add prisma-sharding @prisma/client@^7 prisma@^7 @prisma/adapter-pg pg

Requires Prisma 7 and Node.js ^20.19, ^22.12, or >=24.

Minimal Setup

1. Configure environment variables

# .env
DATABASE_URL=postgresql://USER:PASSWORD@localhost:5432/app
SHARD_DIRECTORY_URL=postgresql://USER:PASSWORD@localhost:5432/app

SHARD_COUNT=2
SHARD_1_URL=postgresql://USER:PASSWORD@localhost:5432/app_shard_1
SHARD_2_URL=postgresql://USER:PASSWORD@localhost:5432/app_shard_2

• SHARD_N_URL — application shards
• SHARD_DIRECTORY_URL — control-plane database that remembers ownership
• DATABASE_URL — Prisma CLI datasource, and directory fallback if SHARD_DIRECTORY_URL is unset

2. Configure Prisma

Keep your existing Prisma schema. The same schema is applied to every shard.

prisma/
├── schema/
│   ├── base.prisma
│   ├── user.prisma
│   └── ...
└── migrations/

// prisma.config.ts
import 'dotenv/config';
import { defineConfig } from 'prisma/config';

export default defineConfig({
  schema: 'prisma/schema',
  migrations: {
    path: 'prisma/migrations',
  },
  datasource: {
    url: process.env.DATABASE_URL,
  },
});

// base.prisma
generator client {
  provider = "prisma-client"
  output   = "../../src/generated/prisma"
}

datasource db {
  provider = "postgresql"
}

3. Configure PrismaSharding

// sharding.ts

import 'dotenv/config';
import { PrismaPg } from '@prisma/adapter-pg';
import { PrismaClient } from '../generated/prisma/client.js';
import { PrismaSharding } from 'prisma-sharding';

export const sharding = new PrismaSharding<PrismaClient>({
  namespace: 'my-app-users',

  shards: [
    { id: 'shard_1', url: process.env.SHARD_1_URL! },
    { id: 'shard_2', url: process.env.SHARD_2_URL! },
  ],

  createClient: (url) => {
    const adapter = new PrismaPg({ connectionString: url });
    return new PrismaClient({ adapter });
  },
});

namespace is a stable name for this ownership domain. Keep it unchanged after production data exists.

Ownership directory in production

The built-in ownership directory stores the canonical (namespace, routingKey) → shardId mapping. Treat its PostgreSQL database as critical application infrastructure: use high availability, backups, point-in-time recovery, monitoring, and restricted credentials appropriate for production data.

4. Connect on application startup

// server.ts
await sharding.connect();

const server = app.listen(3000);

const shutdown = async () => {
  server.close();
  await sharding.disconnect();
};

process.on('SIGINT', shutdown);
process.on('SIGTERM', shutdown);

Call connect() before accepting traffic and disconnect() during shutdown.

5. Add database scripts

{
  "scripts": {
    "db:update": "prisma-sharding-update",
    "db:studio": "prisma-studio-next",
    "test:shards": "prisma-sharding-test"
  }
}

yarn db:update

db:update generates the Prisma Client and updates every shard. With no committed migration history it performs direct schema synchronization (db push). Once committed migrations exist, the migration-history workflow is authoritative. Full details are in Migrations.

6. Start the application

Your application is now ready to use Prisma Sharding.

If you need extra configuration or other features, continue below.

Basic Usage

Route by User. Child data stays on that User's shard:

User
 ├── Project
 ├── Invoice
 ├── Settings
 └── other User-owned data

Signup                 findAcrossShards(email) → generate ID → allocateShard() → create User
Login / middleware     known User ID → resolveShard() → one Prisma client

Signup

Email is not a routing key, so applications may use findAcrossShards() first to discover an existing account with that email. Then generate the User ID and call allocateShard().

// signup.ts
import { randomUUID } from 'node:crypto';
import { sharding } from './config/sharding';

const existing = await sharding.findAcrossShards((db) => db.user.findUnique({ where: { email } }));

if (existing.data) {
  throw new Error('Email already in use');
}

const userId = randomUUID();
const db = await sharding.allocateShard(userId);

const user = await db.user.create({
  data: {
    id: userId,
    name: 'John Doe',
    email,
  },
});

For signup purposes, findAcrossShards() can detect existing duplicates, but it does not guarantee global uniqueness. For strict uniqueness of email, username, phone, or slug, manage a separate application-owned lookup table outside the shards with database-level UNIQUE constraints.

generate User ID
      ↓
allocateShard(User.id)
      ↓
ownership established
      ↓
User.create()

Login and middleware

Call resolveShard() when you already have the User ID — after login, or in auth middleware on each request.

That returned db is the client for the physical shard containing this User. Every User, Project, Invoice, or other User-owned operation must use it. Put it on the request context in middleware, then read it in resolvers.

// middleware.ts
const userId = session.userId;
const db = await sharding.resolveShard(userId);

return {
  userId,
  db,
};

// resolver.ts
me: (_, __, { db, userId }) => db.user.findUnique({ where: { id: userId } });

projects: (_, __, { db, userId }) => db.project.findMany({ where: { userId } });

createProject: (_, { name }, { db, userId }) => db.project.create({ data: { name, userId } });

Do not resolve the shard again in each resolver. Do not search all shards for User-owned data.

JWT/session
   ↓
User ID
   ↓
resolveShard()
   ↓
request db client
   ↓
all User-owned work

Searching When the Routing Key Is Unknown

Use findAcrossShards() only when the User ID is unknown. It is useful for normal duplicate discovery, login by email, password reset, lookup by other non-routing attributes such as a public slug, and admin discovery.

const found = await sharding.findAcrossShards((db) => db.user.findUnique({ where: { email } }));

if (!found.data) {
  // User does not exist
} else {
  const canonicalDb = await sharding.resolveShard(found.data.id);
}

Direct Shard Access

const db = sharding.selectShard('shard_2'); // diagnostics / recovery
const db = sharding.randomShard(); // non-owning random access

Never use randomShard() to create a durable User or other routed entity.

Running Work Across All Shards

const results = await sharding.runAcrossShards((db) => db.user.count());

for (const result of results) {
  if (result.error) {
    console.error(`Shard ${result.shardId}`, result.error);
  } else {
    console.log(`Shard ${result.shardId}`, result.data);
  }
}

Use this for analytics and admin work, not normal User routing. Unavailable shards return ShardUnavailableError in that entry.

Health

const health = sharding.inspectShards();

Example:

[
  {
    shardId: 'shard_1',
    status: 'healthy',
    latencyMs: 11,
  },
  {
    shardId: 'shard_2',
    status: 'unhealthy',
    latencyMs: null,
  },
];

Most applications should keep the health defaults.

Prisma Studio

yarn db:studio

📦 Studio  http://localhost:51212

One Studio, one tab. Switch databases with the shard selector in the header. The current table/view is preserved. Credentials stay on the server.

Shard 1
Shard 2
Shard 3

SHARD_STUDIO_TABLE_GROUPING=true   # optional sidebar grouping

Host architecture, embedding, and connection-string details: docs/studio-host-architecture.md.

Migrations

One command applies schema changes to every shard:

yarn db:update

The behavior depends on whether the project has committed migration history:

| Project state | db:update behavior | | --------------------------------------- | --------------------------------------------------- | | No committed Prisma migrations | Direct schema synchronization with Prisma db push | | One or more committed Prisma migrations | Migration-history workflow is authoritative |

Once a project starts using committed migrations, keep those migration files committed and include them in every deployment. db:update does not fall back to direct schema synchronization while committed migration history exists.

Prisma Client generated
↓
committed migration history checked
├─ none → each shard synchronized directly
└─ exists → each shard checked
             → existing databases adopt represented history when safely detectable
             → pending migrations applied
             → schema verified

✅ client   Generated
✅ shard_1  Synced
✅ shard_2  Synced

A failed shard stops the run. Fix it and run yarn db:update again. Already-synced shards are skipped. Before recording history for any existing database, the command preflights the complete selected fleet.

When committed migration history exists, db:update automatically handles:

• empty databases by applying the full committed history;
• databases already managed by Prisma by applying only pending migrations;
• existing databases without Prisma history by safely detecting and recording an exact represented migration prefix.

Git stores the ordered migration files. Every physical database independently stores which of those files it has applied in its own _prisma_migrations table. There is no shared migration cursor in project configuration, so development, staging, production, and individual shards can safely be at different points in the same committed history.

Git: prisma/migrations/001 → 002 → 003 → 004
                         │
                         ├─ dev A       _prisma_migrations: 001, 002, 003
                         ├─ dev B       _prisma_migrations: 001
                         ├─ staging     _prisma_migrations: 001, 002, 003, 004
                         └─ production  _prisma_migrations: 001, 002

Each database advances from its own recorded state when yarn db:update targets it. If an existing database cannot be adopted safely, the command stops without changing database history.

1. New project — add a migration

1. Change files in prisma/schema/.
2. Create the migration SQL (do not apply it yourself):

npx prisma migrate dev --name add_project_status --create-only

That adds a folder:

prisma/migrations/
  20260301000000_add_project_status/
    migration.sql

3. Apply it to every shard:

yarn db:update

Empty shards run the full history. Shards that already have a migration skip it.

2. Existing database — already has tables

For a database built with db push or another schema-management workflow, commit the matching Prisma migrations and run the same command:

yarn db:update

The command compares that physical database's live schema with contiguous migration prefixes. If it can prove an exact represented prefix safely, it records those migrations directly in that database's _prisma_migrations table:

prisma/migrations/
  20260101000000_init/            already in the database
  20260201000000_add_projects/    already in the database   ← adopted history
  20260301000000_add_invoices/    not in the database yet

db:update never writes project configuration or a source-controlled state file. A different database may safely match a different prefix; the complete selected fleet is still preflighted before any database history is written.

represented migrations   recorded as applied in this database — SQL is not run
later migrations         SQL runs normally in this database

Automatic adoption fails closed. A represented migration containing a backfill, correction, routine, or other custom SQL cannot be proven from schema equality alone. A pending data migration is eligible only when the safe DDL before its first unprovable statement creates an independently detectable schema delta, proving that the ordered data operation is still pending. Schema changes after data/custom SQL never count as that proof. Otherwise, no history is written and advanced operator review is required.

Shadow database for automatic adoption

Migration-prefix detection uses Prisma migrate diff --from-migrations, which requires Prisma shadow-database capability. Local PostgreSQL commonly works without extra setup when the connecting role can create and drop temporary databases.

Restricted production, staging, and cloud roles commonly cannot do that. Configure a separate shadow database with standard Prisma 7 configuration:

// prisma.config.ts
import { defineConfig } from 'prisma/config';

export default defineConfig({
  schema: 'prisma/schema.prisma',
  migrations: {
    path: 'prisma/migrations',
  },
  datasource: {
    url: process.env.DATABASE_URL,
    shadowDatabaseUrl: process.env.SHADOW_DATABASE_URL,
  },
});

SHADOW_DATABASE_URL must point to a separate disposable database that Prisma may reset. It must never point at the real target database. If shadow-database creation or access fails, db:update stops before writing migration history and tells you to configure datasource.shadowDatabaseUrl before rerunning yarn db:update.

prisma-sharding-baseline remains available only as advanced operator recovery for a reviewed exception. It is not part of normal setup or day-to-day migration work.

A new empty shard is never adopted. It runs the full history from the first migration.

3. PostgreSQL extensions

Optional. Same prisma-sharding.config.json. Created on each shard before migrations.

Most new projects can omit migrations unless they use the optional bootstrap configuration.

4. Local reset

yarn db:update --force-reset

Local databases only. Refuses production, staging, remote hosts, and Docker service names.

Other commands

| Command | When | | ---------------------------------- | -------------------------------------------------------------------------------------- | | prisma-sharding-baseline | Advanced operator recovery when automatic adoption proof is impossible | | prisma-sharding-verify-bootstrap | Optional CI check that empty databases can run the full history | | prisma-sharding-push | Explicit direct db push for developers intentionally invoking schema synchronization |

Set SHARD_CLI_VERBOSE=true for detailed output.

Environment Variables

Core

| Variable | Description | | --------------------- | ----------------------------------------- | | DATABASE_URL | Prisma CLI datasource; directory fallback | | SHARD_DIRECTORY_URL | Ownership-directory URL | | SHARD_COUNT | Number of application shards | | SHARD_N_URL | Connection URL for shard N |

Database Update

| Variable | Description | | ------------------------ | ------------------------------------- | | SHARD_CLI_VERBOSE | Verbose CLI output | | SHARD_UPDATE_VERBOSE | Alias for update verbosity | | SHARD_STRICT_DRIFT | Fail on schema drift | | SHARD_STRICT_BOOTSTRAP | Require a verified bootstrap contract | | PRISMA_MIGRATIONS_PATH | Migrations directory override | | PRISMA_SCHEMA_PATH | Schema path override |

Studio

| Variable | Default | Description | | ----------------------------------------- | ------- | -------------------------------- | | SHARD_STUDIO_BASE_PORT | 51212 | Preferred port | | SHARD_STUDIO_REUSE_EXISTING | true | Reuse a matching host | | SHARD_STUDIO_STRICT_PORT_CHECK | false | Fail if the host cannot start | | SHARD_STUDIO_START_TIMEOUT_MS | 15000 | Startup timeout | | SHARD_STUDIO_STABILITY_MS | 500 | Listen stability window | | SHARD_STUDIO_SHUTDOWN_TIMEOUT_MS | 5000 | Shutdown timeout | | SHARD_STUDIO_PORT_SCAN_LIMIT | 100 | Ports to scan | | SHARD_STUDIO_REGISTRY_DIR | OS temp | Host identity registry | | SHARD_STUDIO_MAX_OPEN_CONNECTIONS | 3 | Open shard connections | | SHARD_STUDIO_IDLE_CONNECTION_TIMEOUT_MS | 60000 | Idle connection lifetime | | SHARD_STUDIO_TABLE_GROUPING | false | Group sidebar by table prefix | | SHARD_STUDIO_VERBOSE | false | Startup diagnostics | | SHARD_STUDIO_DEBUG | — | Alias for SHARD_STUDIO_VERBOSE |

The built-in CLI binds to 127.0.0.1. Network-reachable Studio: docs/studio-host-architecture.md.

Logging

| Variable | Description | | ------------------------- | ---------------------- | | PRISMA_SHARDING_VERBOSE | Library lifecycle logs |

Advanced

| Variable | Description | | ---------------------------------------- | --------------------------------------- | | SHARD_DIRECTORY_PROVIDER | custom skips PostgreSQL directory DDL | | SHARD_PUSH_VERBOSE | Verbose push output | | SHARD_BASELINE_VERBOSE | Verbose baseline output | | PRISMA_SHARDING_BOOTSTRAP_DATABASE_URL | Disposable bootstrap-verification URL |

Configuration

Basic

| Option | Type | Description | | -------------- | --------------------------- | ---------------------------------- | | namespace | string | Stable ownership domain. Required. | | shards | ShardConfig[] | Application shards. Required. | | createClient | (url, shardId) => TClient | Prisma client factory. Required. |

Optional

| Option | Type | Default | Description | | -------------------- | ---------------- | -------- | -------------------------- | | directory | ShardDirectory | Built-in | Custom ownership provider | | placementCacheSize | number | 1000 | In-process ownership cache | | logger | ShardingLogger | Default | info / warn / error |

Health

Most applications should keep the defaults.

| Option | Default | Description | | ------------------------- | --------------------- | ------------------------------- | | healthCheck | Built-in Prisma probe | Custom probe | | healthCheckIntervalMs | 30000 | Background interval | | healthCheckTimeoutMs | 5000 | Failure-classification timeout | | healthFailureThreshold | 3 | Failures before isolation | | healthRecoveryThreshold | 2 | Successes required for recovery |

interface ShardConfig {
  id: string;
  url: string;
  weight?: number;
  acceptNewPlacements?: boolean;
}

logger: {
  info: (msg) => myLogger.info(msg),
  warn: (msg) => myLogger.warn(msg),
  error: (msg) => myLogger.error(msg),
}

API Reference

import {
  PrismaSharding,
  ShardUnavailableError,
  ShardOwnershipNotFoundError,
  ShardSearchIncompleteError,
  CrossShardTimeoutError,
  CrossShardOverloadedError,
} from 'prisma-sharding';

| API | Purpose | | -------------------- | --------------------------------------------------- | | connect() | Start the shard runtime | | allocateShard(key) | Establish durable ownership for a new key | | resolveShard(key) | Resolve existing ownership | | selectShard(id) | Direct physical shard access | | randomShard() | Random non-owning shard access | | findAcrossShards() | Search when the routing key is unknown | | runAcrossShards() | Execute administrative/aggregate work across shards | | inspectShards() | Inspect health | | disconnect() | Graceful shutdown |

await sharding.connect()
const db = await sharding.allocateShard(routingKey)
const db = await sharding.resolveShard(routingKey)
const db = sharding.selectShard('shard_2')
const db = sharding.randomShard()
const found = await sharding.findAcrossShards(...)
const results = await sharding.runAcrossShards(...)
const health = sharding.inspectShards()
await sharding.disconnect()

Errors

| Error | Meaning | | ----------------------------- | ----------------------------------------------------- | | ShardOwnershipNotFoundError | No ownership exists for the routing key | | ShardUnavailableError | The canonical shard is currently unavailable | | ShardSearchIncompleteError | A global search could not inspect all relevant shards | | CrossShardTimeoutError | One cross-shard operation exceeded its deadline | | CrossShardOverloadedError | Cross-shard capacity is currently exhausted | | StartupReadinessError | Startup could not verify any usable shard |

If findAcrossShards() finds nothing but could not check every shard, that is not a not-found result.

try {
  const found = await sharding.findAcrossShards((db) => db.user.findUnique({ where: { email } }));
} catch (error) {
  if (error instanceof ShardSearchIncompleteError) {
    // Do not treat this as "not found"
  }
}

Advanced Health Monitoring

Most applications should keep the defaults.

Standard Prisma clients need no health configuration. prisma-sharding uses $queryRaw SELECT 1, continues probing unavailable shards, and recovers them automatically. Generic clients may supply healthCheck(client, shardId, signal).

The returned client is the real Prisma client. Health probes do not wrap or intercept application queries.

Architecture

Application
    │
    │ routingKey
    ▼
PrismaSharding
    │
    ├── Ownership Directory   routingKey → shardId
    ├── Shard Manager         clients + health
    └── Placement             new ownership only
                │
                ▼
       shard_1 / shard_2 / shard_3

Ownership is stable. Availability, weights, draining, and outages do not reassign existing keys.

| Layer | Responsibility | | --------------------- | ------------------------------------------ | | Public API | Canonical public surface | | Directory | Durable routingKey → shardId | | Placement selector | Weighted placement for new ownership only | | Shard manager | Clients, health, recovery, shutdown | | Cross-shard executor | Search/run semantics, deadlines, isolation | | Cross-shard scheduler | Bounded admission and physical concurrency | | CLI | Update, test, and Studio orchestration |

findAcrossShards() and runAcrossShards() are for discovery and admin work. Exhausted capacity raises CrossShardOverloadedError. Deadlines do not cancel the underlying database query — set PostgreSQL statement_timeout as well.

Connection Pooling

createClient() owns the Prisma pool. Budget:

instances × shards × connections-per-client

20 × 8 × 10 = 1,600 connections. Use PgBouncer at larger fleet sizes.

Advanced Configuration

Shard Availability

User owns shard_2 → shard_2 goes offline → resolveShard(userId) → ShardUnavailableError

The User is not moved to another shard. When shard_2 recovers, the same ownership works again.

Safe Static-Topology Rollouts

Shard topology is application configuration. Every running application instance must know a shard before ownership-directory records can safely point to it.

Adding a new shard

Stage 1 — Register

Add the shard to every application instance with new placements disabled:

shards: [
  { id: 'shard_1', url: process.env.SHARD_1_URL! },
  { id: 'shard_2', url: process.env.SHARD_2_URL! },
  {
    id: 'shard_3',
    url: process.env.SHARD_3_URL!,
    acceptNewPlacements: false,
  },
];

Apply and verify the schema on the new shard with the normal database workflow. Deploy this configuration until all running application instances know about shard_3. Existing ownership continues to resolve normally, and no new ownership is assigned to shard_3 yet.

Stage 2 — Enable placement

After every instance knows the shard, set acceptNewPlacements: true or omit the option, then deploy that change. This prevents a newer instance from creating ownership for shard_3 while an older instance does not know that shard exists. Existing ownership remains unchanged; the new shard participates only in future allocations.

Draining or removing a shard

Set acceptNewPlacements: false first. Existing ownership must continue resolving to that shard. Never remove a shard from application configuration while ownership-directory records still reference it. prisma-sharding does not automatically move tenants or reshard existing ownership.

Weighted placement

Weight affects only future allocations. Existing ownership never changes.

shards: [
  { id: 'shard_1', url: process.env.SHARD_1_URL!, weight: 1 },
  { id: 'shard_2', url: process.env.SHARD_2_URL!, weight: 2 },
];

Custom ownership directory

Most applications should use the built-in PostgreSQL directory. A custom directory must implement get() and atomic claim():

directory: {
  get(namespace, routingKey) {
    return controlPlane.getOwnership(namespace, routingKey)
  },
  claim(namespace, routingKey, proposedShardId) {
    return controlPlane.claimOwnership(namespace, routingKey, proposedShardId)
  },
}

Set SHARD_DIRECTORY_PROVIDER=custom so db:update does not provision PostgreSQL ownership metadata.

Example Application

examples/example-1 — Express + TypeScript + Prisma, User ownership and User-owned Projects.

POST /users      allocateShard(userId)
GET  /users/me   resolveShard(userId)
POST /projects   resolve User shard → create Project

Testing

yarn test:shards

Read-only connection diagnostic. Creates no database state.

Library-maintainer commands (not application setup):

yarn test
yarn release:check
yarn release:pack-check

Common Problems

User exists but resolveShard() says ownership is missing

Create through allocateShard(), or migrate historical data into the directory. There is no automatic repair.

User's shard is offline

ShardUnavailableError. Wait for recovery. Do not move the User.

Login by email does not know the User ID

findAcrossShards(), then resolveShard(found.data.id). Incomplete search is not "not found".

New shard receives no existing users

Expected. Existing ownership is immutable.

randomShard() created data that cannot later be resolved

randomShard() does not record ownership. Use allocateShard().

Database update fails on one shard

Fix the migration/database issue and rerun yarn db:update.

Studio shows the wrong or missing database

Check SHARD_COUNT and each SHARD_N_URL. Details: docs/studio-host-architecture.md.

Author

safdar-azeem

License

MIT