Suzume

Japanese tokenization for browser and Node.js applications. Suzume is not a full morphological analyzer like MeCab: it prioritizes useful search units while still providing part-of-speech tags, lemmas, and keyword extraction in one WebAssembly package.

📖 Documentation & getting started  ·  🧪 Live Demo

See the MeCab comparison for concrete examples of token boundaries, lemmatization, and trade-offs.

Installation

npm install @libraz/suzume

Quick Start

import { Suzume } from '@libraz/suzume'

const suzume = await Suzume.create()

const tokens = suzume.analyze('すもももももももものうち')
const tags = suzume.generateTags('東京の公園に行きました')

suzume.destroy() // optional immediate cleanup

Filtering generated tags by part of speech

Use posFilter with any combination of noun, verb, adjective, and adverb:

const tags = suzume.generateTags('東京の公園に行きました', {
  posFilter: ['noun', 'verb'],
})

Unknown names throw an Error. An empty array (posFilter: []) means all content words, matching the default native filter. The former pos option remains available as a deprecated alias; when both are provided, posFilter takes precedence.

Loading the .wasm file

Bundlers that do not resolve the WebAssembly asset automatically can pass its URL explicitly:

import wasmUrl from '@libraz/suzume/wasm?url' // Vite

const suzume = await Suzume.create({ wasmPath: wasmUrl })

For CDN usage, user dictionaries, and the full API, see the JavaScript / TypeScript guide.

Error handling

A failing native call throws an Error carrying the message from the WebAssembly module, so ordinary failures — a rejected user dictionary, use after destroy() — are catchable:

try {
  suzume.loadUserDictionaryOrThrow(csv)
} catch (error) {
  // message comes from the module
}

Running out of memory is the exception. The module is compiled without C++ exceptions, so an allocation failure aborts the WebAssembly instance rather than returning an error, and no catch can recover from it. The instance is unusable afterwards. Analyze long documents in chunks rather than relying on error handling to survive a large input.

Also available

pip install suzume  # Native Python bindings

The C and C++ library is documented at suzume.libraz.net/docs/cpp.

License

Apache License 2.0