@azghr/foreshorten

Trim a chat message array to fit a token/size budget — keep the system prompt, pin the most recent turns, never split a message.

The problem

When working with LLMs, chat message arrays can grow unbounded, exceeding context window limits and increasing costs. You need to trim conversation histories while preserving system prompts and maintaining conversation context. Manual trimming is error-prone and can split messages or lose important context.

Install

npm install @azghr/foreshorten
# or
pnpm add @azghr/foreshorten
# or
yarn add @azghr/foreshorten

Use

import { foreshorten } from "@azghr/foreshorten";

const messages = [
  { role: "system", content: "You are a helpful assistant." },
  { role: "user", content: "Hello!" },
  { role: "assistant", content: "Hi there!" },
  { role: "user", content: "How are you?" },
  { role: "assistant", content: "I'm doing well, thanks!" }
];

// Simple character count measure
const countChars = (m: Message) => m.content.length;

// Trim to fit within 50 characters
const trimmed = foreshorten(messages, 50, countChars);

Preserve system prompt and pin recent messages

const longConversation = [
  { role: "system", content: "You are a coding assistant." },
  // ... many messages ...
  { role: "user", content: "What about memoization?" },
  { role: "assistant", content: "Memoization caches computed results..." }
];

// Keep the 2 most recent messages (after system)
const trimmed = foreshorten(longConversation, 50, estimateTokens, {
  keepRecent: 2
});

Handle overflow

import { foreshorten, BudgetTooSmall } from "@azghr/foreshorten";

// Option 1: truncate-oldest (default)
const trimmed = foreshorten(messages, 30, estimateTokens, {
  overflow: "truncate-oldest"
});

// Option 2: throw if budget exceeded
try {
  const trimmed = foreshorten(messages, 30, estimateTokens, {
    overflow: "throw"
  });
} catch (error) {
  if (error instanceof BudgetTooSmall) {
    console.log("Budget too small:", error.message);
  }
}

Custom measure function

// Use your own tokenizer
const tokenizer = /* your tiktoken or similar */;
const countTokens = (m: Message) => tokenizer.encode(m.content).length;

const trimmed = foreshorten(messages, 4096, countTokens);

API

foreshorten(messages, budget, measure, options?)

Trim a chat message array to fit a token/size budget.

Parameters:

• messages (readonly MessageType[]) — The chat history to trim
• budget (number) — Maximum total cost allowed (e.g., tokens, bytes)
• measure (MeasureFn<MessageType>) — Function to measure cost of a single message
• options? (ForeshortenOptions) — Optional configuration

Returns: MessageType[] — A new array with messages fitting the budget

Throws: BudgetTooSmall — If overflow is "throw" and required messages exceed budget

ForeshortenOptions

Configuration options for foreshorten.

Properties:

• overflow — How to handle overflow when budget is exceeded
  • "throw" — Throw BudgetTooSmall error
  • "truncate-oldest" — Remove oldest non-required messages (default)
• keepRecent — Number of most recent messages to always keep (default: 0)

BudgetTooSmall

Error thrown when required messages exceed budget and overflow is "throw".

class BudgetTooSmall extends Error {
  constructor(deficit: number);
}

How it works

1. Preserve system messages — All messages with role: "system" are kept
2. Pin recent messages — The N most recent messages (after system) are pinned
3. Fill remaining budget — Add older messages from newest to oldest while budget allows
4. Apply overflow policy — Either throw or stop adding when budget is exhausted

Original: [system, user1, assistant1, user2, assistant2, user3, assistant3]
           ↓                        ↓                    ↓
        System          keepRecent: 2          Fill from newest
           │                        │                    │
Result:   [system, assistant2, user3, assistant3, user2]
           └───────────────────────────────────────────────────
           System + Recent + Older (until budget exhausted)

Non-goals

• Token counting — This package doesn't count tokens; you provide the measure function
• Message splitting — Messages are never split; they're kept whole or removed entirely
• Conversation state management — This package only trims arrays; it doesn't manage conversation state
• LLM API integration — This is a utility package; it doesn't make API calls

Related Packages

• @azghr/filterkit — Framework-agnostic, type-safe filtering for TypeScript
• @azghr/shorn — Truncate strings by byte budget without breaking graphemes
• @azghr/singlet — Deduplicate concurrent async calls
• congeal — Data structure utilities
• decant — Extract and transform utilities
• expunge — Remove or exclude items from collections
• extricate — Extract and separate utilities
• forbar — Read server rate-limit instructions from HTTP responses
• forestall — Delay execution until a condition is met
• obviate — Render operations unnecessary through caching
• occlude — Hide or mask data and functionality
• placemark — Geographic location and mapping utilities
• quiesce — Ordered, timeboxed graceful shutdown for Node
• seriatim — Sequential processing utilities
• sortition — Deterministic percentage rollouts and A/B bucketing
• specie — Currency and financial calculations
• stanch — Stop flows or operations based on conditions
• staleness — Stale-while-revalidate caching for async functions
• wend — Polling utilities with exponential backoff

Note: This list should be kept in sync with the packages in pnpm-workspace.yaml. When adding a new package to the monorepo, update this list to include all sibling packages.

License

MIT