ctxsplit

Split large contexts intelligently. Semantic chunking. Overlap control.

Install

npm install -g ctxsplit

Usage

# Split file into chunks
ctxsplit large-doc.txt

# Specify chunk size (tokens)
ctxsplit doc.txt --size 4000
ctxsplit doc.txt -s 4000

# Set overlap between chunks
ctxsplit doc.txt --overlap 200
ctxsplit doc.txt -o 200

# Split by sentences (semantic)
ctxsplit doc.txt --semantic

# Split by paragraphs
ctxsplit doc.txt --by paragraph

# Split by headers (markdown)
ctxsplit doc.md --by header

# Output to files
ctxsplit doc.txt --output chunks/

# Output as JSON
ctxsplit doc.txt --json

# Show chunk info only
ctxsplit doc.txt --info

# Read from stdin
cat doc.txt | ctxsplit -s 2000

# Help
ctxsplit --help

Strategies

• size (default): Split by token count with overlap
• sentence: Split at sentence boundaries
• paragraph: Split at paragraph boundaries
• header: Split markdown by headers

Output

$ ctxsplit doc.txt --info
  ctxsplit - pnkd.dev
  ───────────────────
  Input:     doc.txt
  Size:      45,678 tokens
  Strategy:  size (4000 tokens)
  Overlap:   200 tokens

  Chunks:    12
  Range:     3,800 - 4,000 tokens
  Average:   3,806 tokens

$ ctxsplit doc.txt --output chunks/
  ✓ Created chunks/chunk-01.txt (3,923 tokens)
  ✓ Created chunks/chunk-02.txt (4,001 tokens)
  ✓ Created chunks/chunk-03.txt (3,856 tokens)

  3 chunks written to chunks/

License

MIT

pnkd.dev - glitch the system