Provider-independent model contracts, prompt caching and explicit model registration.

Issue Reporting and Security

For reporting bugs, issues, or security vulnerabilities, please visit community.foss.global/. This is the central community hub for all issue reporting. Developers who sign and comply with our contribution agreement and go through identification can also get a code.foss.global/ account to submit Pull Requests directly.

Part of the FlexHarness toolbox. The toolbox has four package names: flexharness, flexharness-agent, flexharness-models, and flexharness-providers. Install the packages your project uses. Provider adapters are registered explicitly; the agent and harness do not install or register vendor SDKs.

pnpm add @modelprofile.com/flexharness-models

See the toolbox usage and migration guide for composition examples and the SmartAI/SmartAgent migration map.

ModelLimitError is the one typed error for provider usage and rate limits; its limit (IModelLimitInfo) carries kind, provider, and when known resetsAt (epoch milliseconds), retryAfterMs, planType and window. createModelLimitMiddleware({ provider, classify }) raises it for the failures a provider dialect recognizes, from thrown calls and from stream error parts, keeping the provider failure as cause. For an AI SDK provider stream error the classifier receives the provider's frame (data) as the body and the status the provider states for it. A classifier receives the failure's observedAt time for relative reset delays; a delay or reset time outside the range a Date can represent is dropped from the limit. createModelLimitInfo, isModelLimitError, isModelLimitInfo, describeModelLimit, readRetryAfterMs, readEpochSeconds and readJsonRecord are exported for dialects and hosts.

describeModelFailure(error) returns an IModelFailureDescription that names why a model call failed and never holds provider response text, headers or credentials. Its kind (TModelFailureKind) is limit (with the limit), http (with the answer's status), stream (an error a successful response stream reported before or after output began, with the providerCode the provider states, its error code or, for an error with none such as Anthropic's, its error type (overloaded_error), and no status: the answer succeeded, and the HTTP-equivalent status the AI SDK derives from the code, such as 500 for server_error, is no HTTP answer), response (an answer that could not be processed, with its status), network (no answer arrived), aborted (an abort or timeout signal) or unknown. A provider code is kept only in a shape providers name errors in (snake_case words, optionally dotted, such as insufficient_quota; upper-case status names such as RESOURCE_EXHAUSTED; numeric codes) of at most 64 characters, and never when it looks like a credential: a key or token prefix such as sk-, sk_, pk_, ghp_ or eyJ, or a long run of letters and digits. The description follows cause and the last attempt of a retried call.

createTextFilePartMiddleware() sends the text files of user messages as text parts, for providers that read only images and PDFs as files. A file part with inline text, or whose media type names text (isTextMediaType: text/*, JSON, XML, YAML, CSV, script types, +json, +xml and +yaml types, and any type with a charset), becomes one text part with the file name, the media type and the content decoded in the stated charset (UTF-8 by default), in a fence longer than any backtick run in the content. Images, PDFs, other binary files, URLs and file references pass unchanged; a file that names text but does not decode fails before the request is sent.

Model call prices

priceModelCall(price, usage) turns one call's usage into its cost. An IModelPrice holds integer rates in micro-USD per million tokens ($4/MTok is 4_000_000) per service tier the response reports, each tier as prompt-size bands (maxPromptTokens, ascending, the unlimited band last; the prompt counts uncached input, cache reads and every cache write), cache-write rates by lifetime (5m, 1h, 30m, 12h), tool-call fees in micro-USD per call, per-mille surcharges on the token cost (1100 is 1.1×) and its source (url, checkedAt). An IModelPricedUsage, as a provider's usage reader states it, names the tier, the uncached, cached-read and written tokens, output, tool calls and surcharges; writes the provider did not split by lifetime go to unknownLifetimeCacheWriteTokens.

The result is { priced: true, microUsd, upperBound }, summed exactly and rounded up once; upperBound is set when writes of unknown lifetime were charged at the band's highest write rate. A call the price cannot cover is never priced at zero: { priced: false, reason } names unknownService, noBand, unpricedCategory (tokens in a category without a rate), unpricedToolCall, unknownSurcharge, invalidPrice or invalidUsage. Readers refuse an incomplete provider usage with ModelUsageReadError.

Prompt caching

TSmartAiCacheSetting is 'auto' (the default of a FlexHarness session), false, or an ISmartAiCacheOptions object. It decides the cache hints a request carries:

• Anthropic (and the message-level dialects of OpenRouter, Bedrock and OpenAI-compatible servers): applySmartAiRequestCaching({ prompt, tools }, options) places at most four breakpoints, counting markers the caller set itself. The first two system messages are the stable ones; the second, or the only one, ends the stable prefix and caches the tool definitions rendered before it. Without a system message the last tool definition ends the prefix, so sessions share it. The last message and the one before it carry the rolling breakpoints of a growing conversation, then the first system message. On Anthropic the stable prefix's breakpoints last an hour (prefixRetention, default '1h'), so sessions that start minutes apart still read it, while the conversation keeps five-minute breakpoints; a one-hour write costs twice the input price, a five-minute write 1.25 times, and on steady traffic the prefix is written once an hour. prefixRetention: 'ephemeral' writes the prefix for five minutes; retention: '1h' makes every breakpoint last an hour. A request that carries markers of the caller's own keeps the prefix at retention, since a one-hour breakpoint may not follow a five-minute one. createSmartAiCachingMiddleware(options) applies it to every call of a model. Each model has a minimum cacheable prefix (512 to 4,096 tokens); a breakpoint below it neither writes nor reads and costs nothing, so the planner places breakpoints without counting tokens it cannot count before the request is sent.
• OpenAI (Responses and the ChatGPT Codex backend): store: false and a prompt_cache_key. deriveSmartAiPromptCacheKey({ system, tools, scope }) derives the key from the instructions and tool definitions every turn repeats, so sessions with one prefix reach the machine that caches it; any change to the prefix changes the key. keyScope keeps tenants apart, and key replaces the derived key. OpenAI suggests about 15 requests a minute per key (OPENAI_PROMPT_CACHE_KEY_REQUESTS_PER_MINUTE); keyBuckets spreads a prefix over that many keys, each session keeping one by its session id, on models that route by key; requests without a session id take the first. From GPT-5.6 on OpenAI routes by itself and the key only separates cache accounting (openAiRoutesPromptCacheAutomatically). retention: 'in_memory' | '24h' is sent as prompt_cache_retention to earlier models; without it the provider's default applies.

getSmartAiCacheProviderOptions and applySmartAiCacheProviderOptions return the provider options of a setting; a caller's own options win.

License and Legal Information

This repository contains open-source code licensed under the MIT License. A copy of the license can be found in the repository license file.

Please note: The MIT License does not grant permission to use the trade names, trademarks, service marks, or product names of the project, except as required for reasonable and customary use in describing the origin of the work and reproducing the content of the NOTICE file.

Trademarks

This project is owned and maintained by Task Venture Capital GmbH. The names and logos associated with Task Venture Capital GmbH and any related products or services are trademarks of Task Venture Capital GmbH or third parties, and are not included within the scope of the MIT license granted herein.

Use of these trademarks must comply with Task Venture Capital GmbH's Trademark Guidelines or the guidelines of the respective third-party owners, and any usage must be approved in writing. Third-party trademarks used herein are the property of their respective owners and used only in a descriptive manner, e.g. for an implementation of an API or similar.

Company Information

Task Venture Capital GmbH Registered at District Court Bremen HRB 35230 HB, Germany

For any legal inquiries or further information, please contact us via email at [email protected].

By using this repository, you acknowledge that you have read this section, agree to comply with its terms, and understand that the licensing of the code does not imply endorsement by Task Venture Capital GmbH of any derivative works.