Uniseg TypeScript

A comprehensive TypeScript port of rivo/uniseg with 100% API compatibility

Unicode text segmentation for grapheme clusters, text width calculation, and string manipulation. Built for TypeScript/Node.js.

Documentation • Examples • API Reference • Go Original

✨ Features

• 🔤 Complete Unicode Text Segmentation - Grapheme clusters, word boundaries, line breaks
• 🌍 Unicode 15.0.0 Support - Latest Unicode standard with comprehensive property tables
• 😀 Advanced Emoji Support - Flags, ZWJ sequences, skin tone modifiers, regional indicators
• 🕉️ Complex Script Support - Devanagari, Bengali, and other Indic scripts with combining marks
• 📏 Accurate Width Calculation - East Asian Width property for monospace fonts
• 🔄 100% Go API Compatibility - Perfect compatibility with original Go rivo/uniseg
• ⚡ High Performance - Optimized state machines and Unicode property lookups
• 🎯 Type-Safe - Full TypeScript support with comprehensive type definitions
• 📦 Zero Dependencies - Lightweight and self-contained
• 🚀 Cross-Platform - Works on Windows, macOS, and Linux

🚀 Quick Start

Installation

# npm
npm install @tsports/uniseg

# yarn
yarn add @tsports/uniseg

# bun (recommended)
bun add @tsports/uniseg

Basic Usage

TypeScript-Native API (Recommended):

import { graphemeClusterCount, stringWidth, reverseString, newGraphemes } from '@tsports/uniseg';

// Count user-perceived characters (grapheme clusters)
graphemeClusterCount('Hello'); // 5
graphemeClusterCount('🇩🇪🏳️‍🌈'); // 2 (German flag + rainbow flag)
graphemeClusterCount('नमस्ते'); // 4 (Devanagari script)
graphemeClusterCount('🧑‍💻'); // 1 (person technologist emoji)

// Calculate display width for monospace fonts
stringWidth('Hello'); // 5
stringWidth('你好'); // 4 (full-width characters)
stringWidth('🇩🇪🏳️‍🌈'); // 4 (emoji width)

// Reverse strings while preserving grapheme clusters
reverseString('Hello'); // 'olleH'
reverseString('🇩🇪🏳️‍🌈'); // '🏳️‍🌈🇩🇪'
reverseString('नमस्ते'); // 'तेस्मन'

// Iterate through grapheme clusters
const iter = newGraphemes('🧑‍💻 Hello');
let cluster;
while ((cluster = iter.next()) !== null) {
  console.log(`Cluster: "${cluster.cluster}" at position ${cluster.startPos}`);
}
// Output:
// Cluster: "🧑‍💻" at position 0
// Cluster: " " at position 5
// Cluster: "H" at position 6
// Cluster: "e" at position 7
// ...

Go-Compatible API (For Go Developers):

import { GraphemeClusterCount, StringWidth, ReverseString, NewGraphemes } from '@tsports/uniseg/go-style';

// Exact Go rivo/uniseg API with PascalCase methods
const count = GraphemeClusterCount('🇩🇪🏳️‍🌈'); // 2
const width = StringWidth('Hello 世界'); // 9
const reversed = ReverseString('नमस्ते'); // 'तेस्मन'

// Go-style iterator
const iter = NewGraphemes('🧑‍💻');
while (iter.Next()) {
  const cluster = iter.Str();
  const runes = iter.Runes();
  console.log(`"${cluster}" -> [${runes.map(r => `U+${r.toString(16).toUpperCase()}`).join(', ')}]`);
}

📊 Perfect Compatibility Results

Our implementation achieves 100% compatibility with Go rivo/uniseg:

| Test Case | Expected | Got | Status | | --------------------- | -------- | --- | ------ | | "Hello" | 5 | 5 | ✅ | | "🇩🇪🏳️‍🌈" (flags) | 2 | 2 | ✅ | | "नमस्ते" (Devanagari) | 4 | 4 | ✅ | | "🧑‍💻" (ZWJ emoji) | 1 | 1 | ✅ | | "a̧" (combining) | 1 | 1 | ✅ | | "" (empty) | 0 | 0 | ✅ |

All test outputs match the Go reference implementation exactly.

📖 Documentation

Core Functions

Grapheme Cluster Counting

import { graphemeClusterCount } from '@tsports/uniseg';

// Basic counting
graphemeClusterCount('Hello World'); // 11

// Complex emoji sequences
graphemeClusterCount('👨‍👩‍👧‍👦'); // 1 (family emoji)
graphemeClusterCount('🏳️‍⚧️'); // 1 (transgender flag)
graphemeClusterCount('👋🏻'); // 1 (waving hand with skin tone)

// Regional indicator pairs (flags)
graphemeClusterCount('🇺🇸🇬🇧'); // 2 (US flag + UK flag)

// Complex scripts with combining marks
graphemeClusterCount('ज़िन्दगी'); // 7 (Hindi with nukta and combining marks)
graphemeClusterCount('பெண்கள்'); // 6 (Tamil script)

String Width Calculation

import { stringWidth } from '@tsports/uniseg';

// Latin characters
stringWidth('Hello'); // 5

// East Asian characters (full-width)
stringWidth('你好世界'); // 8
stringWidth('こんにちは'); // 10

// Mixed content
stringWidth('Hello 世界'); // 9

// Emoji and symbols
stringWidth('🚀📱💻'); // 6
stringWidth('→←↑↓'); // 4

String Reversal

import { reverseString } from '@tsports/uniseg';

// Preserves grapheme cluster integrity
reverseString('Café'); // 'éfaC'
reverseString('🇺🇸🇬🇧'); // '🇬🇧🇺🇸'
reverseString('👨‍👩‍👧‍👦 Family'); // 'ylimaF 👨‍👩‍👧‍👦'

// Complex scripts
reverseString('नमस्ते दुनिया'); // 'ायिनुद ेत्समन'
reverseString('السلام عليكم'); // 'مكيلع مالسلا'

Advanced Iteration

import { newGraphemes, stepString } from '@tsports/uniseg';

// Iterator pattern
const iter = newGraphemes('🧑‍💻 Hello');
let cluster;
while ((cluster = iter.next()) !== null) {
  console.log({
    cluster: cluster.cluster,
    runes: cluster.runes,
    position: cluster.startPos,
    length: cluster.length
  });
}

// Step-by-step processing
let str = '🇩🇪🏳️‍🌈';
let state = -1;
while (str.length > 0) {
  const result = stepString(str, state);
  console.log(`Segment: "${result.segment}"`);
  str = result.remainder;
  state = result.newState;
}

Unicode Standards Compliance

This library implements:

• UAX #29 - Unicode Text Segmentation
• UAX #11 - East Asian Width
• UAX #15 - Unicode Normalization Forms
• Unicode 15.0.0 - Latest Unicode standard

Supported Features

Emoji Sequences

• Regional Indicator Sequences: 🇺🇸 🇬🇧 🇯🇵
• ZWJ Sequences: 👨‍💻 👩‍🔬 🏳️‍🌈 🏳️‍⚧️
• Modifier Sequences: 👋🏻 👋🏿 💪🏽
• Flag Sequences: 🏴󠁧󠁢󠁳󠁣󠁴󠁿 (Scotland)
• Keycap Sequences: 1️⃣ 2️⃣ #️⃣ *️⃣

Complex Scripts

• Indic Scripts: Devanagari, Bengali, Tamil, Telugu, etc.
• Arabic Script: Arabic, Persian, Urdu with joining behavior
• Combining Marks: Diacritics, accents, nukta marks
• Hangul: Korean syllable composition

Text Width

• East Asian Width properties (Narrow, Wide, Fullwidth, Halfwidth, Ambiguous)
• Emoji width calculation with presentation selectors
• Combining mark handling (zero-width)
• Control character handling

🔄 Dual API Support

TypeScript-Native API (Recommended)

Modern TypeScript patterns with camelCase methods:

import { graphemeClusterCount, stringWidth, newGraphemes } from '@tsports/uniseg';

const count = graphemeClusterCount('🇩🇪🏳️‍🌈');
const width = stringWidth('Hello 世界');
const iter = newGraphemes('text');

Go-Compatible API

100% identical Go rivo/uniseg API with PascalCase methods:

import { GraphemeClusterCount, StringWidth, NewGraphemes } from '@tsports/uniseg/go-style';

// Exact Go API
const count = GraphemeClusterCount('🇩🇪🏳️‍🌈');
const width = StringWidth('Hello 世界');
const iter = NewGraphemes('text');

Go → TypeScript Migration

// Go rivo/uniseg
import "github.com/rivo/uniseg"

count := uniseg.GraphemeClusterCount("🇩🇪🏳️‍🌈")
width := uniseg.StringWidth("Hello 世界")

// TypeScript - EXACT same API
import { GraphemeClusterCount, StringWidth } from '@tsports/uniseg/go-style';

const count = GraphemeClusterCount('🇩🇪🏳️‍🌈');
const width = StringWidth('Hello 世界');

🧪 Testing & Quality Assurance

100% API Compatibility Verified

• Comprehensive Testing - All outputs compared with Go reference implementation
• Unicode Compliance - Full Unicode 15.0.0 test suite coverage
• Cross-Platform Testing - Windows, macOS, Linux validation
• Performance Testing - Benchmarked against Go implementation
• Edge Case Coverage - Complex sequences, boundary conditions, error cases

Test Execution

# Run all tests
bun test

# Run Go compatibility tests
bun test test/automated-cases.test.ts

# Test specific functionality
cd test/corpus/basic/001-grapheme-count
go run case.go        # Expected output
bun --bun run case.ts # Our output
diff <(go run case.go) <(bun --bun run case.ts) # Should be identical

⚡ Performance

Optimized for high performance with:

• Fast Unicode property lookups - Binary search on sorted ranges
• Efficient state machines - Minimal memory allocations
• Optimized algorithms - Based on proven Go implementation
• TypeScript compilation - Full ahead-of-time optimization

Run benchmarks:

bun run benchmark

📋 Examples

Emoji Analysis

import { graphemeClusterCount, stringWidth, newGraphemes } from '@tsports/uniseg';

const emojis = [
  '👋',           // Basic emoji
  '👋🏻',          // Emoji + skin tone modifier
  '👨‍💻',          // ZWJ sequence (man technologist)
  '👨‍👩‍👧‍👦',       // Family ZWJ sequence
  '🏳️‍🌈',         // Rainbow flag
  '🏳️‍⚧️',         // Transgender flag
  '🇺🇸',          // Country flag
  '1️⃣',          // Keycap sequence
  '🏴󠁧󠁢󠁳󠁣󠁴󠁿'       // Subdivision flag (Scotland)
];

emojis.forEach(emoji => {
  console.log({
    emoji,
    clusters: graphemeClusterCount(emoji),
    width: stringWidth(emoji),
    codePoints: [...emoji].map(c => `U+${c.codePointAt(0)?.toString(16).toUpperCase()}`)
  });
});

Text Processing

import { graphemeClusterCount, reverseString, newGraphemes } from '@tsports/uniseg';

function analyzeText(text: string) {
  const clusters = [];
  const iter = newGraphemes(text);
  let cluster;

  while ((cluster = iter.next()) !== null) {
    clusters.push({
      text: cluster.cluster,
      position: cluster.startPos,
      codePoints: cluster.runes.length
    });
  }

  return {
    originalText: text,
    reversedText: reverseString(text),
    totalClusters: graphemeClusterCount(text),
    displayWidth: stringWidth(text),
    clusters
  };
}

// Analyze complex text
const result = analyzeText('Hello 🌍! नमस्ते 🇮🇳');
console.log(JSON.stringify(result, null, 2));

Go API Compatibility Demo

import {
  GraphemeClusterCount,
  StringWidth,
  ReverseString,
  NewGraphemes
} from '@tsports/uniseg/go-style';

// Direct Go API usage
const text = '🇩🇪🏳️‍🌈 Hello, 世界!';

console.log('Go-Compatible API Results:');
console.log(`Text: "${text}"`);
console.log(`Grapheme Clusters: ${GraphemeClusterCount(text)}`);
console.log(`Display Width: ${StringWidth(text)}`);
console.log(`Reversed: "${ReverseString(text)}"`);

// Iterator usage (Go-style)
const iter = NewGraphemes(text);
console.log('\nGrapheme Clusters:');
while (iter.Next()) {
  const cluster = iter.Str();
  const runes = iter.Runes();
  console.log(`  "${cluster}" [${runes.map(r => `U+${r.toString(16).toUpperCase()}`).join(', ')}]`);
}

🏗️ Architecture

Project Structure

src/
├── index.ts              # TypeScript-native API exports
├── go-style.ts           # Go-compatible API wrapper
├── core.ts               # Core grapheme cluster functions
├── properties.ts         # Unicode 15.0.0 property tables
├── grapheme-rules.ts     # UAX #29 state machine implementation
├── step.ts               # Combined boundary detection
├── width.ts              # East Asian Width calculation
├── types.ts              # TypeScript type definitions
└── utils.ts              # Utility functions

Design Principles

1. 100% Go Compatibility - Identical behavior and API
2. Performance First - Optimized algorithms and data structures
3. Type Safety - Comprehensive TypeScript types
4. Unicode Standards - Full compliance with Unicode specifications
5. Zero Dependencies - Self-contained implementation

🤝 Contributing

We welcome contributions! Please see our Contributing Guide for details.

Development Setup

# Clone with submodules (includes Go reference)
git clone --recursive https://github.com/tsports/uniseg.git
cd uniseg

# Install dependencies
bun install

# Build and test
bun run build
bun test

# Test Go compatibility
cd test/corpus/basic/001-grapheme-count
diff <(go run case.go) <(bun --bun run case.ts)

Adding Unicode Test Cases

When contributing Unicode-related features:

1. Test with Go first to get expected behavior
2. Include complex examples with edge cases
3. Add comprehensive test coverage
4. Verify Unicode standard compliance

📊 Browser Support

While designed for Node.js environments, the core algorithms work in modern browsers:

• ES2020+ - Uses modern JavaScript features
• Unicode support - Requires JavaScript Unicode support
• TypeScript - Full type support in development

🔗 Links

• Original Go Library - The inspiration and reference
• Unicode Text Segmentation - UAX #29 specification
• Unicode 15.0.0 - Unicode version supported
• Contributing Guide - How to contribute
• Changelog - Version history

📄 License

MIT License - see LICENSE file for details.

🙏 Acknowledgments

This TypeScript port is made possible by the exceptional work of the original Go library creators:

• Oliver Kuederle - Creator of the excellent original rivo/uniseg library that serves as the foundation for this TypeScript implementation
• Unicode Consortium - For maintaining the Unicode standard and comprehensive specifications
• Go Team - For the inspiring programming language and well-designed standard library
• TypeScript Community - For the excellent tooling and ecosystem that makes this port possible

This is a TypeScript port - All credit for the original design, algorithms, and Unicode expertise goes to the rivo/uniseg project and its contributors.