@sylphx/rosetta

Lightweight i18n library with production-time string collection and LLM-powered translation.

Features

• Zero config source strings - Write English directly in code: t("Hello World")
• Auto-collection - Strings collected in production as users hit code paths
• LLM translation - Generate translations using OpenRouter, Anthropic, etc.
• Server + Client - Full Next.js App Router support with RSC
• Type-safe - Full TypeScript support
• Adapter pattern - Bring your own storage (Drizzle, Prisma, etc.)
• Admin-ready - Built-in methods for translation management dashboards

Installation

bun add @sylphx/rosetta

# React bindings (for React/Next.js projects)
bun add @sylphx/rosetta-react

# Optional: Drizzle adapter with pre-built schema
bun add @sylphx/rosetta-drizzle

Quick Start

1. Set Up Database Schema (Drizzle)

Option A: Use @sylphx/rosetta-drizzle (Recommended)

// db/schema.ts
import { pgTable, text, timestamp, integer, boolean, unique, serial } from 'drizzle-orm/pg-core';
import { createRosettaSchema } from '@sylphx/rosetta-drizzle/schema';

export const { rosettaSources, rosettaTranslations } = createRosettaSchema({
  pgTable, text, timestamp, integer, boolean, unique, serial
});

// Your other tables...

Option B: Manual schema

// db/schema.ts
import { pgTable, text, timestamp, boolean, serial, unique } from 'drizzle-orm/pg-core';

export const rosettaSources = pgTable('rosetta_sources', {
  id: serial('id').primaryKey(),
  hash: text('hash').notNull().unique(),
  text: text('text').notNull(),
  context: text('context'),
  occurrences: integer('occurrences').default(1),
  firstSeenAt: timestamp('first_seen_at').defaultNow(),
  lastSeenAt: timestamp('last_seen_at').defaultNow(),
});

export const rosettaTranslations = pgTable('rosetta_translations', {
  id: serial('id').primaryKey(),
  locale: text('locale').notNull(),
  hash: text('hash').notNull(),
  text: text('text').notNull(),
  autoGenerated: boolean('auto_generated').default(false),
  reviewed: boolean('reviewed').default(false),
  createdAt: timestamp('created_at').defaultNow(),
  updatedAt: timestamp('updated_at').defaultNow(),
}, (t) => [unique().on(t.locale, t.hash)]);

2. Create Storage Adapter

Option A: Use @sylphx/rosetta-drizzle (Recommended)

// lib/rosetta/storage.ts
import { DrizzleStorageAdapter } from '@sylphx/rosetta-drizzle';
import { db } from '@/db';
import { rosettaSources, rosettaTranslations } from '@/db/schema';

export const storage = new DrizzleStorageAdapter({
  db,
  sources: rosettaSources,
  translations: rosettaTranslations,
});

Option B: Implement StorageAdapter manually

// lib/rosetta/storage.ts
import type { StorageAdapter } from '@sylphx/rosetta';
import { db } from '@/db';
import { eq, inArray, notInArray } from 'drizzle-orm';
import { rosettaSources, rosettaTranslations } from '@/db/schema';

export const storage: StorageAdapter = {
  async getTranslations(locale) {
    const rows = await db
      .select({ hash: rosettaTranslations.hash, text: rosettaTranslations.text })
      .from(rosettaTranslations)
      .where(eq(rosettaTranslations.locale, locale));
    return new Map(rows.map(r => [r.hash, r.text]));
  },

  async saveTranslation(locale, hash, text, options) {
    await db.insert(rosettaTranslations)
      .values({
        locale,
        hash,
        text,
        autoGenerated: options?.autoGenerated ?? false,
      })
      .onConflictDoUpdate({
        target: [rosettaTranslations.locale, rosettaTranslations.hash],
        set: { text, updatedAt: new Date() },
      });
  },

  async getSources() {
    return db.select().from(rosettaSources);
  },

  async getUntranslated(locale) {
    const translated = await db
      .select({ hash: rosettaTranslations.hash })
      .from(rosettaTranslations)
      .where(eq(rosettaTranslations.locale, locale));
    const hashes = translated.map(t => t.hash);

    if (hashes.length === 0) {
      return db.select().from(rosettaSources);
    }
    return db.select().from(rosettaSources)
      .where(notInArray(rosettaSources.hash, hashes));
  },

  async getAvailableLocales() {
    const results = await db
      .select({ locale: rosettaTranslations.locale })
      .from(rosettaTranslations)
      .groupBy(rosettaTranslations.locale);
    return results.map(r => r.locale);
  },
};

3. Initialize Rosetta

// lib/rosetta/index.ts
import { Rosetta } from '@sylphx/rosetta-next/server';
import { OpenRouterAdapter } from '@sylphx/rosetta/adapters';
import { cookies } from 'next/headers';
import { storage } from './storage';

export const rosetta = new Rosetta({
  storage,
  translator: new OpenRouterAdapter({
    apiKey: process.env.OPENROUTER_API_KEY!,
  }),
  defaultLocale: 'en',
  // Languages are discovered automatically from DB - no need to configure!
  localeDetector: async () => {
    const cookieStore = await cookies();
    return cookieStore.get('locale')?.value ?? 'en';
  },
});

export { t, flushCollectedStrings, getTranslationsForClient, getLocale } from '@sylphx/rosetta-next/server';

4. Set Up Layout

// app/layout.tsx
import { rosetta, flushCollectedStrings, getTranslationsForClient, getLocale } from '@/lib/rosetta';
import { RosettaProvider } from '@sylphx/rosetta-react';

export default async function RootLayout({ children }: { children: React.ReactNode }) {
  return rosetta.init(async () => {
    const content = (
      <html lang={getLocale()}>
        <body>
          <RosettaProvider
            locale={getLocale()}
            translations={getTranslationsForClient()}
          >
            {children}
          </RosettaProvider>
        </body>
      </html>
    );

    // Flush collected strings at end of request
    await flushCollectedStrings();
    return content;
  });
}

5. Use Translations

Server Components:

import { t } from '@/lib/rosetta';

export function ServerComponent() {
  return (
    <div>
      <h1>{t("Welcome to our app")}</h1>
      <p>{t("Hello {name}", { name: "World" })}</p>
    </div>
  );
}

Client Components:

'use client';
import { useT } from '@sylphx/rosetta-react';

export function ClientComponent() {
  const t = useT();
  return <button>{t("Sign In")}</button>;
}

Admin Dashboard

API Routes

Create API routes to manage translations:

// app/api/rosetta/sources/route.ts
import { rosetta } from '@/lib/rosetta';
import { NextResponse } from 'next/server';

// GET /api/rosetta/sources - Get all sources with translation status
export async function GET() {
  const sources = await rosetta.getSourcesWithStatus();
  return NextResponse.json(sources);
}

// app/api/rosetta/stats/route.ts
import { rosetta } from '@/lib/rosetta';
import { NextResponse } from 'next/server';

// GET /api/rosetta/stats - Get translation statistics
export async function GET() {
  const stats = await rosetta.getStats();
  return NextResponse.json(stats);
}

// app/api/rosetta/translate/route.ts
import { rosetta } from '@/lib/rosetta';
import { NextResponse } from 'next/server';

// POST /api/rosetta/translate - Generate translation for a string
export async function POST(req: Request) {
  const { text, locale, context } = await req.json();
  const translation = await rosetta.generateAndSave(text, locale, context);
  return NextResponse.json({ translation });
}

// app/api/rosetta/translate/batch/route.ts
import { rosetta } from '@/lib/rosetta';
import { NextResponse } from 'next/server';

// POST /api/rosetta/translate/batch - Batch translate strings
export async function POST(req: Request) {
  const { items, locale } = await req.json();
  const result = await rosetta.batchTranslate(items, locale);
  return NextResponse.json(result);
}

// app/api/rosetta/translations/route.ts
import { rosetta } from '@/lib/rosetta';
import { NextResponse } from 'next/server';

// PUT /api/rosetta/translations - Save manual translation
export async function PUT(req: Request) {
  const { locale, hash, text } = await req.json();
  await rosetta.saveTranslationByHash(locale, hash, text, { autoGenerated: false });
  return NextResponse.json({ success: true });
}

// app/api/rosetta/review/route.ts
import { rosetta } from '@/lib/rosetta';
import { NextResponse } from 'next/server';

// POST /api/rosetta/review - Mark translation as reviewed
export async function POST(req: Request) {
  const { hash, locale } = await req.json();
  await rosetta.markAsReviewed(hash, locale);
  return NextResponse.json({ success: true });
}

// app/api/rosetta/export/route.ts
import { rosetta } from '@/lib/rosetta';
import { NextResponse } from 'next/server';

// GET /api/rosetta/export?locale=zh-TW - Export translations
export async function GET(req: Request) {
  const { searchParams } = new URL(req.url);
  const locale = searchParams.get('locale') ?? 'zh-TW';
  const data = await rosetta.exportTranslations(locale);
  return NextResponse.json(data);
}

// POST /api/rosetta/export - Import translations
export async function POST(req: Request) {
  const { locale, data } = await req.json();
  const count = await rosetta.importTranslations(locale, data);
  return NextResponse.json({ imported: count });
}

Admin UI Example

'use client';

import { useState, useEffect } from 'react';

interface Source {
  id: string;
  text: string;
  hash: string;
  context?: string;
  translations: Record<string, {
    text: string | null;
    autoGenerated: boolean;
    reviewed: boolean;
  } | null>;
}

export function TranslationDashboard() {
  const [sources, setSources] = useState<Source[]>([]);
  const [stats, setStats] = useState<any>(null);
  const [selectedLocale, setSelectedLocale] = useState('zh-TW');

  useEffect(() => {
    fetch('/api/rosetta/sources').then(r => r.json()).then(setSources);
    fetch('/api/rosetta/stats').then(r => r.json()).then(setStats);
  }, []);

  const handleTranslate = async (source: Source) => {
    const res = await fetch('/api/rosetta/translate', {
      method: 'POST',
      headers: { 'Content-Type': 'application/json' },
      body: JSON.stringify({
        text: source.text,
        locale: selectedLocale,
        context: source.context,
      }),
    });
    const { translation } = await res.json();
    // Refresh sources
    fetch('/api/rosetta/sources').then(r => r.json()).then(setSources);
  };

  const handleBatchTranslate = async () => {
    const untranslated = sources.filter(s => !s.translations[selectedLocale]);
    await fetch('/api/rosetta/translate/batch', {
      method: 'POST',
      headers: { 'Content-Type': 'application/json' },
      body: JSON.stringify({
        items: untranslated.map(s => ({ hash: s.hash, text: s.text, context: s.context })),
        locale: selectedLocale,
      }),
    });
    fetch('/api/rosetta/sources').then(r => r.json()).then(setSources);
  };

  return (
    <div>
      <h1>Translation Dashboard</h1>

      {stats && (
        <div>
          <p>Total strings: {stats.totalStrings}</p>
          <p>Translated ({selectedLocale}): {stats.locales[selectedLocale]?.translated ?? 0}</p>
        </div>
      )}

      <select value={selectedLocale} onChange={e => setSelectedLocale(e.target.value)}>
        <option value="zh-TW">Chinese (Traditional)</option>
        <option value="zh-CN">Chinese (Simplified)</option>
        <option value="ja">Japanese</option>
      </select>

      <button onClick={handleBatchTranslate}>
        Translate All Missing
      </button>

      <table>
        <thead>
          <tr>
            <th>Source</th>
            <th>Translation</th>
            <th>Status</th>
            <th>Actions</th>
          </tr>
        </thead>
        <tbody>
          {sources.map(source => {
            const translation = source.translations[selectedLocale];
            return (
              <tr key={source.hash}>
                <td>{source.text}</td>
                <td>{translation?.text ?? '-'}</td>
                <td>
                  {!translation ? 'Missing' :
                   translation.reviewed ? 'Reviewed' :
                   translation.autoGenerated ? 'Auto' : 'Manual'}
                </td>
                <td>
                  {!translation && (
                    <button onClick={() => handleTranslate(source)}>
                      Translate
                    </button>
                  )}
                </td>
              </tr>
            );
          })}
        </tbody>
      </table>
    </div>
  );
}

How It Works

┌─────────────────────────────────────────────────────────────────┐
│  PRODUCTION                                                      │
│  Real users → t("Hello") → 1. Return translation                │
│                           → 2. Queue for collection (async)     │
│                                                                  │
│  End of request → flushCollectedStrings() → Save to DB          │
└─────────────────────────────────────────────────────────────────┘
                              ↓
┌─────────────────────────────────────────────────────────────────┐
│  ADMIN DASHBOARD                                                 │
│  • View all collected strings (getSourcesWithStatus)            │
│  • LLM auto-translate (batchTranslate / generateAndSave)        │
│  • Manual translation / review (saveTranslationByHash)          │
│  • Export → External tools → Import                             │
└─────────────────────────────────────────────────────────────────┘
                              ↓
┌─────────────────────────────────────────────────────────────────┐
│  PRODUCTION (after translations saved)                          │
│  Users → t("Hello") → DB lookup → Return "你好"                 │
└─────────────────────────────────────────────────────────────────┘

API Reference

Server (@sylphx/rosetta-next/server)

Rosetta class

const rosetta = new Rosetta({
  storage: StorageAdapter,       // Required: your storage adapter
  translator?: TranslateAdapter, // Optional: for auto-translation
  defaultLocale?: string,        // Default: 'en'
  cacheTTL?: number,             // Default: 60000 (1 minute)
  localeDetector?: () => string, // Function to detect current locale
});

// Core methods
await rosetta.init(fn)                  // Initialize context and run function
await rosetta.getClientData()           // Get data for client hydration
await rosetta.loadTranslations(locale)  // Load translations for a locale

// Source/translation management
await rosetta.getSources()              // Get all source strings
await rosetta.getUntranslated(locale)   // Get untranslated strings for locale
await rosetta.saveTranslation(locale, text, translation, context?)

// Auto-translation
await rosetta.generateTranslation(text, locale, context?)
await rosetta.generateAndSave(text, locale, context?)
await rosetta.generateAllUntranslated(locale, onProgress?)
await rosetta.batchTranslate(items, locale)

// Admin methods
await rosetta.getSourcesWithStatus(locales)  // Get sources with translation status
await rosetta.getStats(locales)              // Get translation statistics
await rosetta.markAsReviewed(hash, locale)
await rosetta.saveTranslationByHash(locale, hash, text, options?)
await rosetta.exportTranslations(locale)
await rosetta.importTranslations(locale, data, options?)

// Utilities
await rosetta.getAvailableLocales()     // Get locales that have translations (from DB)
rosetta.getDefaultLocale()              // Get default locale
rosetta.invalidateCache()               // Clear translation cache

t(text, params?) function

t("Hello World")                      // Simple translation
t("Hello {name}", { name: "John" })   // With interpolation
t("Submit", { context: "form" })      // With context for disambiguation

Other exports

flushCollectedStrings()    // Flush pending strings to storage
getLocale()                // Get current locale
getTranslationsForClient() // Get translations for client provider

React (@sylphx/rosetta-react)

import { RosettaProvider, useT, useLocale } from '@sylphx/rosetta-react';

<RosettaProvider locale="en" translations={translations}>
  {children}
</RosettaProvider>

const t = useT();           // Get translation function
const locale = useLocale(); // Get current locale

Adapters (@sylphx/rosetta/adapters)

import { OpenRouterAdapter } from '@sylphx/rosetta/adapters';

const translator = new OpenRouterAdapter({
  apiKey: string,           // Required
  model?: string,           // Default: 'openai/gpt-4.1-mini'
  temperature?: number,     // Default: 0.3
  maxTokens?: number,       // Default: 500
});

Drizzle Package (@sylphx/rosetta-drizzle)

// Schema helpers
import { createRosettaSchema } from '@sylphx/rosetta-drizzle/schema';
import { createRosettaSchemaSQLite } from '@sylphx/rosetta-drizzle/schema';
import { createRosettaSchemaMySQL } from '@sylphx/rosetta-drizzle/schema';

// Storage adapter
import { DrizzleStorageAdapter } from '@sylphx/rosetta-drizzle';

const storage = new DrizzleStorageAdapter({
  db,                    // Drizzle database instance
  sources: rosettaSources,     // Sources table from schema
  translations: rosettaTranslations, // Translations table from schema
});

Supported Databases

The @sylphx/rosetta-drizzle package supports:

• PostgreSQL - createRosettaSchema()
• SQLite - createRosettaSchemaSQLite()
• MySQL - createRosettaSchemaMySQL()

Next.js Sync (@sylphx/rosetta-next/sync)

The sync module provides build-time string extraction for Next.js projects:

// next.config.ts
import { withRosetta } from '@sylphx/rosetta-next/sync';

export default withRosetta({
  // your next config
});

// scripts/sync-rosetta.ts (run after build)
import { syncRosetta } from '@sylphx/rosetta-next/sync';
import { storage } from '../src/lib/rosetta-storage';

await syncRosetta(storage, { verbose: true });

Distributed Lock Behavior

syncRosetta() uses a file-based lock to prevent multiple processes from syncing simultaneously.

✅ Works Well For

• Single-server deployments - Traditional Node.js servers
• CI/CD pipelines - Single build runner syncing to DB
• Development - Local development workflows
• Docker single-instance - One container syncing at a time

⚠️ Limitations

| Environment | Issue | Recommendation | |------------|-------|----------------| | Kubernetes multi-pod | Pods have isolated filesystems, lock not shared | Sync from CI/CD only, not at runtime | | Vercel/Lambda | Ephemeral filesystems don't persist | Use forceLock: true or sync in build step | | Docker Swarm/ECS | Each container has own filesystem | Sync from single deployment task | | NFS/shared filesystem | O_EXCL may not be atomic | Use database-level locking instead |

Recommended Patterns

Pattern 1: CI/CD Sync (Recommended)

# In your CI/CD pipeline, after build
bun run sync-rosetta.ts

This ensures only one process syncs, regardless of deployment target.

Pattern 2: Force Lock for Serverless

// For environments where file-based locking doesn't work
await syncRosetta(storage, {
  forceLock: true,  // Skip lock acquisition
  verbose: true,
});

⚠️ May cause duplicate sync operations in concurrent scenarios.

Pattern 3: Custom Database Lock

// Implement your own distributed lock with Redis/database
const lockAcquired = await acquireRedisLock('rosetta-sync');
if (lockAcquired) {
  try {
    await syncRosetta(storage, { forceLock: true });
  } finally {
    await releaseRedisLock('rosetta-sync');
  }
}

Caching

For serverless environments where DB latency matters, use the cache adapters:

import { Rosetta, ExternalCache } from '@sylphx/rosetta-next/server';
import { Redis } from '@upstash/redis';

const redis = new Redis({ url, token });
const cache = new ExternalCache(redis, { ttlSeconds: 60 });

const rosetta = new Rosetta({
  storage,
  cache,  // Optional: reduces DB queries in serverless
  defaultLocale: 'en',
});

// Invalidate cache after admin updates translations
await rosetta.invalidateCache('zh-TW');

Available cache adapters:

• InMemoryCache - LRU cache for traditional servers
• ExternalCache - Redis/Upstash for serverless (cross-pod)
• RequestScopedCache - Request-level deduplication

License

MIT