ctx-compact

Trim a plain OpenAI-shaped message array down to a token budget without ever orphaning a tool result.

The problem

Every agent framework ships its own conversation compaction (LangGraph, Inspect, MS Agent Framework, the Claude SDK all have one), and each is welded to that framework's message type. If you are working with a plain array of { role, content, ... } messages, people tend to hand-roll a "drop the oldest N messages" loop. That works until an assistant message with tool_calls gets dropped but its matching tool result messages do not (or the reverse). Most providers reject that shape outright, so the trim silently turns into an API error on the next call. ctx-compact is a small, framework-neutral compactor that keeps tool-call and tool-result messages paired and dropped or kept as a unit.

Install

npm i ctx-compact

Usage

import { compact, compactWithSummary } from 'ctx-compact';

const messages = [
  { role: 'system', content: 'You are a helpful assistant.' },
  { role: 'user', content: 'What is the weather in Denver?' },
  {
    role: 'assistant',
    content: null,
    tool_calls: [{ id: 'call_1', type: 'function', function: { name: 'get_weather', arguments: '{"city":"Denver"}' } }],
  },
  { role: 'tool', tool_call_id: 'call_1', content: '{"tempF":72}' },
  { role: 'assistant', content: 'It is 72F in Denver.' },
  { role: 'user', content: 'What about Austin?' },
  {
    role: 'assistant',
    content: null,
    tool_calls: [{ id: 'call_2', type: 'function', function: { name: 'get_weather', arguments: '{"city":"Austin"}' } }],
  },
  { role: 'tool', tool_call_id: 'call_2', content: '{"tempF":88}' },
  { role: 'assistant', content: 'It is 88F in Austin.' },
  { role: 'user', content: 'And tomorrow in Denver?' },
];

const result = compact(messages, { maxTokens: 150, keepHead: 1, keepTail: 2 });
// result.tokensBefore -> 195
// result.tokensAfter  -> 124
// result.fits         -> true (124 <= 150)
// result.dropped      -> the 3 oldest droppable messages: the first "What is
//                        the weather in Denver?" turn and its whole
//                        assistant/tool_calls + tool group

// Async variant: summarize whatever got dropped and splice a note back in.
const withSummary = await compactWithSummary(messages, {
  maxTokens: 150,
  keepHead: 1,
  keepTail: 2,
  summarize: async (dropped) => `Earlier in this conversation: ${dropped.length} messages were removed.`,
});
// withSummary.summary      -> 'Earlier in this conversation: 3 messages were removed.'
// withSummary.tokensAfter  -> 145 (the 124 kept after dropping, plus the inserted summary message)
// withSummary.fits         -> true (145 <= 150)

API

compact(messages, options) -> CompactResult

Synchronous. Drops messages from the middle of messages until the estimated token count fits the budget.

• messages: array of { role, content, tool_calls?, tool_call_id?, name? }. role is one of 'system' | 'user' | 'assistant' | 'tool'.
• options.maxTokens (required, number) - the budget. Throws TypeError if missing or not a positive number.
• options.countTokens - (message) => number. Default: Math.ceil(JSON.stringify(message).length / 4).
• options.keepHead - number of leading messages always kept. Default 1.
• options.keepTail - number of trailing messages always kept. Default 4.

Returns:

{
  messages: Message[],   // the compacted array
  dropped: Message[],    // what was removed, in original order
  tokensBefore: number,  // summed estimated tokens of the input array
  tokensAfter: number,   // summed estimated tokens of the output array
  fits: boolean          // tokensAfter <= maxTokens
}

If the input already fits, it is returned unchanged with dropped: [].

compactWithSummary(messages, options) -> Promise<CompactResult & { summary: string | null }>

Async. Same as compact, then, if anything was dropped and options.summarize is provided, calls await options.summarize(dropped) and inserts the returned string as a message { role: options.summaryRole, content: <summary> } immediately after the head-kept messages.

• options.summarize - (dropped: Message[]) => Promise<string> | string. If omitted, behaves exactly like compact and returns summary: null.
• options.summaryRole - default 'user'. Some providers reject a second system message, which is why the default is 'user' rather than 'system'.

The inserted summary message counts toward the budget: after insertion the result is re-checked, and if it no longer fits, additional whole groups are dropped (oldest first) to make room. fits is reported false if it is still over budget after that.

estimateTokens(message, countTokens?) -> number

Runs countTokens (or the default heuristic) against a single message. Exported so callers can reuse the same estimator compact/compactWithSummary use, e.g. to pre-check a message before appending it.

How it works

1. Group first. Before anything is dropped, the whole array is split into groups: an assistant message carrying tool_calls plus every immediately-following tool message whose tool_call_id matches one of that assistant's tool_calls[].id forms one group. Every other message (including a tool message with no matching assistant) is its own group. Groups are always dropped or kept whole, so a tool result is never left without its assistant call, or vice versa.
2. Snap keepHead/keepTail to group boundaries. keepHead and keepTail are counted in messages, but if the boundary would land inside a group, it expands outward to keep that whole group.
3. Drop oldest-first. Whatever is left in the middle is droppable. Groups are dropped oldest first until the running token total fits maxTokens or nothing droppable is left.
4. Token counts are an estimate (~length / 4 by default, or your own countTokens), not a real tokenizer. There is no LLM call, no tokenizer library, and no streaming. If your countTokens is inaccurate, fits will be inaccurate too. Pass a countTokens backed by your provider's real tokenizer if you need exact numbers.
5. compactWithSummary never re-summarizes after dropping additional groups to make room for the summary itself; it just drops more of the already-dropped-eligible messages. If you need every dropped message reflected in the summary text, make sure maxTokens leaves enough headroom for the summary you expect summarize to produce.

Related

Small, single-purpose packages for the same problem space. Each one has zero dependencies and does one thing.

• prompt-cache-fit - Reorder prompt blocks least-variable-first for prefix cache reuse, and measure the hit rate.
• cmd-risk - Classify how destructive a shell command is, so an agent knows when to ask a human.
• apply-edit-block - Apply LLM search/replace edit blocks that do not match the source exactly.
• cassette-fn - Record and replay LLM calls at the function boundary, so your tools still run on replay.

License

MIT