@chonkiejs/core

Core chunking library for Chonkie - lightweight and efficient text chunking with zero dependencies.

Features

✨ Simple & Clean API - Easy to use OOP design ⚡ Zero Dependencies - Minimal, lightweight, fast 🔤 Character-based - Simple tokenization (1 char = 1 token) 🎯 Recursive Chunking - Smart hierarchical text splitting 📦 TypeScript First - Full type safety with TypeScript

Installation

Install with npm:

npm i @chonkiejs/core

Install with pnpm:

pnpm add @chonkiejs/core

Install with yarn:

yarn add @chonkiejs/core

Install with bun:

bun add @chonkiejs/core

Quick Start

import { RecursiveChunker } from '@chonkiejs/core';

// Create a chunker
const chunker = await RecursiveChunker.create({
  chunkSize: 512,
  minCharactersPerChunk: 24
});

// Chunk your text
const chunks = await chunker.chunk('Your text here...');

// Use the chunks
for (const chunk of chunks) {
  console.log(chunk.text);
  console.log(`Tokens: ${chunk.tokenCount}`);
}

Available Chunkers

| Name | Description | |------|-------------| | RecursiveChunker | Recursively splits text using hierarchical rules (paragraphs → sentences → punctuation → words → characters). Each level only activates if chunks exceed the configured size. | | TokenChunker | Splits text into fixed-size token chunks with optional overlap. Uses character-based tokenization by default, or HuggingFace models with @chonkiejs/token. |

For detailed API documentation, configuration options, and advanced usage, see DOCS.md.

Contributing

Want to help grow Chonkie? Check out CONTRIBUTING.md to get started! Whether you're fixing bugs, adding features, improving docs, or simply leaving a ⭐️ on the repo, every contribution helps make Chonkie a better CHONK for everyone.

Remember: No contribution is too small for this tiny hippo!

Acknowledgements

Chonkie would like to CHONK its way through a special thanks to all the users and contributors who have helped make this library what it is today! Your feedback, issue reports, and improvements have helped make Chonkie the CHONKIEST it can be.

And of course, special thanks to Moto Moto for endorsing Chonkie with his famous quote:

"I like them big, I like them chonkie in TypeScript" ~ Moto Moto... definitely did not say this

Citation

If you use Chonkie in your research, please cite it as follows:

@software{chonkie2025,
  author = {Bhavnick Minhas and Shreyash Nigam},
  title = {Chonkie: A no-nonsense fast, lightweight, and efficient text chunking library},
  year = {2025},
  publisher = {GitHub},
  howpublished = {\url{https://github.com/chonkie-inc}},
}