@modelprofile.com/flexharness-agent

v9.4.0

Published

5 hours ago

Standalone event-driven agent runtime with generation transactions, context and tool execution contracts.

0High
0Medium
0Low

FlexHarness Agent runs model-and-tool loops inside your application. Give it a language model, a task and your tools; it streams progress, executes tool calls, maintains conversation events and returns the answer with usage and tool-call records. Use runAgent for a task with automatic cleanup, or AgentSession when your application owns an ongoing conversation.

Issue Reporting and Security

For reporting bugs, issues, or security vulnerabilities, please visit community.foss.global/. This is the central community hub for all issue reporting. Developers who sign and comply with our contribution agreement and go through identification can also get a code.foss.global/ account to submit Pull Requests directly.

When to use it

| Use case | How this package helps | | --- | --- | | A CLI command or background worker that investigates a task | runAgent runs several model/tool steps and closes its session when it finishes or fails. | | A support assistant or application copilot with follow-up questions | AgentSession keeps the conversation, serializes generations and exposes events for your UI or application. | | A job that must return an application-validated answer | validateCompletion can accept an answer or request a bounded correction. | | A workflow that reviews a draft before accepting it into future context | Transactional generations separate model execution from the host's acceptance decision. | | An application with its own storage, permissions or execution infrastructure | Supply tools, an event store and execution-context contracts while retaining control of those services. | | A browser assistant using an application-controlled model transport | The /runner entrypoint provides the same task loop without Node session dependencies. |

This is the standalone agent layer of the FlexHarness toolbox. Choose the full @modelprofile.com/flexharness package when you also want managed session discovery, scope/model/tool resolution, permission workflows, prompt queues and subagents. For a single inference request, generateText or streamText from @modelprofile.com/flexharness-models may be sufficient.

Install and choose an entrypoint

pnpm add @modelprofile.com/flexharness-agent

The root entrypoint requires Node.js 24 or newer. It exports runAgent, AgentSession, tool helpers, event utilities and storage contracts. Browser code must import @modelprofile.com/flexharness-agent/runner directly; that entrypoint exports runAgent and its option/result types, with the session lifecycle owned internally.

The agent accepts an AI SDK LanguageModelV3 or LanguageModelV4. It does not select a provider, install provider SDKs or register providers globally. If your application already has a compatible model, pass it directly. To use the toolbox's provider adapters, also install:

pnpm add @modelprofile.com/flexharness-models @modelprofile.com/flexharness-providers

All four toolbox packages share a release version. Keep packages used together on the same release.

Quick start: a task with a tool

This Node example uses an explicitly registered provider. Set OPENAI_API_KEY and MODEL_ID to a tool-capable model available to your account. Other adapters are available through the providers package.

import { runAgent, tool, z } from '@modelprofile.com/flexharness-agent';
import { ModelRegistry } from '@modelprofile.com/flexharness-models';
import { createOpenAiModelProvider } from '@modelprofile.com/flexharness-providers/openai';

const apiKey = process.env.OPENAI_API_KEY;
const modelId = process.env.MODEL_ID;
if (!apiKey || !modelId) {
  throw new Error('Set OPENAI_API_KEY and MODEL_ID before running this example.');
}

const models = new ModelRegistry().register(createOpenAiModelProvider());
const setup = models.getModelSetup({ provider: 'openai', model: modelId, apiKey });
const tools = {
  convert_distance: tool({
    description: 'Convert a distance in kilometres to miles.',
    inputSchema: z.object({ kilometres: z.number().nonnegative() }),
    execute: async ({ kilometres }) => ({ miles: kilometres / 1.609344 }),
  }),
};

const result = await runAgent({
  ...setup,
  system: 'Use the supplied tools for calculations and explain the result briefly.',
  prompt: 'Use convert_distance to convert a 42.195 km marathon to miles.',
  tools,
  maxSteps: 5,
  abort: AbortSignal.timeout(60_000),
  onToken: (delta) => { process.stdout.write(delta); },
  onToolCallFinish: (event) => {
    if (!event.success) console.error(event.toolName, event.error);
  },
});

console.log();
console.log(result.text);
console.log(result.usage, result.toolCalls);

runAgent creates a session, adds the prompt, generates the answer and closes the session in a finally block. There is no session object for the caller to close. Model, tool, validation and cleanup failures reject its promise.

The following Node examples reuse setup and, where needed, tools from the quick start.

Read the result and continue a conversation

| Result field | Meaning | | --- | --- | | text | Text from the final generation. | | messages | Current AI SDK message history after projection or compaction. Pass it into another run to continue. | | steps | Completed model steps, including steps from validation-triggered attempts. A step can call several tools. | | finishReason | The model's final finish reason; inspect this together with application validation. | | usage | Input, output and total tokens, plus cache-read and cache-write tokens, summed over every model call of the run the provider reported: its model steps, retried calls included, and the reported calls of a compaction its context overflow caused. | | toolCalls | Tool-call IDs, names and inputs, with available outputs or errors. |

const followUp = await runAgent({
  ...setup,
  tools,
  messages: result.messages,
  prompt: 'Now convert a half marathon of 21.0975 km.',
  maxSteps: 5,
});
console.log(followUp.text);

messages is a conversation projection. Use session events and an event store when you need the canonical history, transaction outcomes or tool-execution reconciliation records.

Keep a session across turns

Use AgentSession.create() when the host needs to add messages or runtime events, observe committed changes, control cancellation or save and restore a session.

import { AgentSession } from '@modelprofile.com/flexharness-agent';

const session = await AgentSession.create({
  ...setup,
  tools,
  sessionId: 'distance-assistant',
  maxSteps: 5,
});
const unsubscribe = session.subscribe((change) => {
  console.log('Session change:', change.type, change.events.length);
});

try {
  await session.pushUserMessage('Convert 10 km to miles.');
  const first = await session.generate({ abort: AbortSignal.timeout(60_000) });
  console.log(first.text);

  await session.pushUserMessage('And 25 km?');
  const second = await session.generate({ abort: AbortSignal.timeout(60_000) });
  console.log(second.text);

  console.log(session.getEvents());
  console.log(session.getModelMessages());
} finally {
  unsubscribe();
  await session.close();
}

generate() operates on the events already in the session; it does not take a prompt. It returns the run-result fields plus events. Generations on a session are serialized. Per-generation model, system, tools, providerOptions, cache and maxSteps override the session defaults.

For bursts of application events, scheduleGenerate({ key, debounceMs }) coalesces calls with the same key and matching generation options while they are debouncing or queued. It returns their shared result promise. The default quiet period is 50 ms; cancelScheduledGeneration(key) cancels pending work for that key. An actively executing generation is controlled through its abort signal or abortCurrentGeneration().

Streaming and observation

Pass these callbacks to runAgent or AgentSession.create():

| Callback | Delivered data | | --- | --- | | onToken(delta) | Each streamed answer-text delta. | | onReasoningStart, onReasoningDelta, onReasoningEnd | Provider-supplied streamed reasoning summaries, when available. | | onToolCallStart(event) | toolCallId, toolName, input. | | onToolCallUpdate(event) | The call identity and a transient streamed output. | | onToolCallFinish(event) | The call identity plus either success: true, output or success: false, error. | | onRetry(event) | Before each wait to retry a model call: attempt, maxAttempts, delayMs and reason (rate_limit, overloaded, unavailable). | | onUsage(event) | Once per model call, as soon as its usage is known; see Count the usage of every run. |

Tool updates are transient; the finish callback carries the authoritative final output. Use subscribe() for committed session changes (committed, updated, archived), and retain its returned unsubscribe function. Session listeners are delivered in order per listener, have bounded queues and timeouts, and are removed on failure. Streaming callbacks and session-change listeners serve different purposes.

Count the usage of every run

A run that throws or is aborted has used tokens too, and its promise carries no result. onUsage reports each model call once, as soon as its usage is known, whatever the run's outcome. Sum it to count a run's usage:

import type { IAgentUsage } from '@modelprofile.com/flexharness-agent';

const used: IAgentUsage = { inputTokens: 0, outputTokens: 0, totalTokens: 0, cacheReadTokens: 0, cacheWriteTokens: 0 };
let usageComplete = true;
try {
  await runAgent({
    ...setup,
    tools,
    prompt: 'Convert 10 km to miles.',
    abort: AbortSignal.timeout(60_000),
    onUsage: (event) => {
      if (event.status === 'unreported') {
        usageComplete = false;
        return;
      }
      for (const key of Object.keys(used) as (keyof IAgentUsage)[]) used[key] += event.usage[key];
    },
  });
} finally {
  console.log(used, usageComplete ? 'complete' : 'lower bound');
}

| Event field | Meaning | | --- | --- | | source | generation: a model step of the generation. compaction: a model call of a context compaction. | | generationId | The generation the call belongs to. A compaction carries the generation whose context overflow caused it; a compaction by compact() or event retention has none. | | provider, requestedModelId | The model the call was made with: the language model's provider and modelId. Every event carries both, whatever its status, so key usage caps on them. | | status: 'reported', usage, responseModelId | The provider reported the call's usage. A count the provider leaves out is zero. responseModelId is the model id the provider's response named (a provider may answer with a dated model version), or requestedModelId when it named none. | | status: 'unreported', reason | The call ended before the provider reported its usage: aborted (the call was aborted), failed (the call or its response failed) or missing (the response carried no usage). The provider may still have consumed tokens for it; their number is unknown. |

When a run returns, its reported calls sum to result.usage; count one or the other, not both. A call is reported when the provider's response ends, before its tool calls run, so a run aborted during a tool call still reports the call that requested it. Retried calls and validation retries are included. onUsage must not throw; an error it throws is reported like a session listener error and does not change the run's outcome. runAgent delivers every event of the run before its promise settles. AgentSession.create() accepts the same callback for every generation and compaction of the session; generate() and scheduleGenerate() without a transaction reject as soon as their abort signal fires and may report the interrupted call afterwards; close() waits for that report. With a transaction, they settle only after every call of the generation has been reported.

What is counted

Every model step of a generation, whatever its outcome, including retried calls and validation retries.
The model calls of a context compaction, when the compactor reports them. contextCompactor and onContextOverflow receive reportUsage in their options; it reports into onUsage with source: 'compaction'. compactMessages() from @modelprofile.com/flexharness/compaction reports each attempt of its model call when it receives reportUsage, so pass the handler's options through:
```
import { compactMessages } from '@modelprofile.com/flexharness/compaction';

const session = await AgentSession.create({
  ...setup,
  contextCompactor: (messages, _events, options) => compactMessages(setup.model, messages, options),
  onContextOverflow: (messages, options) => compactMessages(setup.model, messages, options),
  onUsage: (event) => console.log(event),
});
```
A handler that makes its own model calls reports each one through reportUsage before its promise settles, a call that fails or is aborted as unreported. AgentModelCallUsageRecorder does the bookkeeping for one language model: call start() when a call begins, end(responseModelId, usage) with the AI SDK's onLanguageModelCallEnd values, and settle(reason) when a call ends otherwise.

Not counted: model calls a handler makes without reporting them through reportUsage, and model calls outside the session, such as tools that call models themselves. Report those in your own accounting.

Validate an answer and request corrections

validateCompletion returns void to accept a result or a string to add a corrective user message and generate again. maxValidationRetries defaults to 0: a failed validation throws unless retries are configured.

const answerSchema = z.object({
  summary: z.string().min(1),
  actionItems: z.array(z.string()),
});

const validated = await runAgent({
  ...setup,
  prompt: 'Return only JSON with summary and actionItems for planning a team meeting.',
  maxSteps: 3,
  maxValidationRetries: 2,
  abort: AbortSignal.timeout(60_000),
  validateCompletion: (candidate) => {
    try {
      answerSchema.parse(JSON.parse(candidate.text));
    } catch {
      return 'Return valid JSON only: a nonempty summary string and an actionItems array of strings.';
    }
  },
});
console.log(answerSchema.parse(JSON.parse(validated.text)));

Validation retries retain the conversation and accumulate usage, steps and tool-call records. maxSteps applies to each generation, so validation retries can increase the overall step count. Validation checks the answer; it does not undo effects of tools already executed.

Review a generation before accepting it

A transactional generation gives the host an explicit acceptance boundary. This is useful when an application must review or durably stage an answer before including it in future model context.

export async function createReviewedDraft(
  review: (text: string) => Promise<boolean>,
) {
  const reviewSession = await AgentSession.create({ ...setup, sessionId: 'draft-review' });
  try {
    const transaction = await reviewSession.beginGeneration(
      'Draft a short invitation to a team meeting.',
      { generationId: 'meeting-draft-1' },
    );
    const candidate = await reviewSession.generate({
      transaction,
      abort: AbortSignal.timeout(60_000),
    });
    const outcome = await review(candidate.text) ? 'accepted' : 'rejected';
    await reviewSession.finalizeGeneration(transaction, outcome);
    return { outcome, text: candidate.text, events: reviewSession.getEvents() };
  } finally {
    await reviewSession.close();
  }
}

The review callback belongs to your application. While awaiting acceptance, candidate.messages includes the candidate, but getModelMessages() excludes it from ordinary future context. Acceptance includes the generation; rejection keeps it in the canonical event history while excluding it from future context. Generation IDs identify transactions within a session; use a distinct ID for each new transaction.

Closing a session or restoring a stored session interrupts unfinished transactions. Restoration does not automatically rerun them. Persisted transactions require an IAgentEventStoreV2 store with eventSchemaVersion: 2; without a store, transactions are process-local.

A rejected generation does not roll back external tool effects. The runtime records transactional tool-execution intents and results. If a crash leaves an intent without a known result, inspect listUncertainToolExecutions() and reconcile the external operation through reconcileToolExecution() before treating its outcome as known. This is not a general exactly-once guarantee for external services.

Events, storage and context

Canonical events retain user and assistant messages, tool calls and results, runtime events, compaction records and transaction outcomes. getEvents() returns this history; getModelMessages() projects the model-visible conversation through the configured contextBuilder, which defaults to buildModelMessages.

Restore a stored session

Supply a stable sessionId and an event store to AgentSession.create(). It loads the stored snapshot and saves subsequent event changes. Do not use the constructor directly with a store, and do not also supply initial events or messages when a snapshot already exists.

import type { IAgentEventStoreV2 } from '@modelprofile.com/flexharness-agent';

export function openStoredSession(eventStore: IAgentEventStoreV2, sessionId: string) {
  return AgentSession.create({ ...setup, tools, eventStore, sessionId });
}
// The caller owns the returned session and must await session.close() when done.

Use an application-owned durable adapter for persistent sessions. Its save(sessionId, events, expectedRevision) must atomically compare the revision and return the new revision, or reject a conflict. Preserve the schema and supported values in canonical events; the package exports schema validators and AgentEventStoreConflictError for adapter implementations. Keep one active owner per session; compare-and-swap detects conflicting writes rather than merging competing sessions.

InMemoryAgentEventStore is useful for tests and sessions that only need process-local storage. The Node root also exports the explicit FileAgentEventStore adapter. Neither storage selection nor a durable backend is implicit.

Bound model context and active events

contextBuilder({ events }) controls the model-message projection.
contextCompactor(messages, events, { reason, abortSignal, reportUsage }) returns replacement model messages. Provide it to use session.compact() or automatic event retention. Report the usage of its model calls through reportUsage; see What is counted.
eventRetention: { maxEvents } triggers compaction and archival when the active event count exceeds the threshold. It also requires an event store with archive() support.
Context overflow invokes the configured compactor, or the onContextOverflow(messages, { abortSignal, reportUsage }) handler. Without either, generation throws ContextOverflowError. maxContextOverflowRetries defaults to 3.
Open transactions and uncertain tool intents constrain when compaction and archival can proceed. Resolve them before manual compaction.

Compaction changes the active model context; archival moves covered events out of the active event set. Implement archive retention in your chosen store when you need a complete audit history.

Tools and execution ownership

The root exports the AI SDK tool, jsonSchema and stepCountIs helpers, plus Zod as z. Pass a tool object directly, as in the quick start, or register definitions with new ToolRegistry().register(name, definition).getTools(). Tool names must be unique; invalid is reserved for tool-call repair.

Your tool implementations own access to application services and external effects. IToolExecutionContext defines optional filesystem, shell, browser, background-job and permission interfaces for host integrations. Supplying a context does not install tools or create isolation. Reusable tools live in @modelprofile.com/flexharness/tools; Node execution integrations live in /tools/node. Permission checks and isolation must be implemented by the chosen host/tool integration.

For generation-scoped resources, generate({ prepare }) accepts a callback that returns an IAgentGenerationLease. Its model/tool/settings overrides take precedence over per-generation options and session defaults. The runtime calls the lease's close() after execution. The host must clean up partial acquisition if prepare throws; lease cleanup must tolerate repeated or overlapping attempts after a failure or timeout.

Cancellation, retries and cleanup

| Operation | Effect | | --- | --- | | runAgent({ abort }) / generate({ abort }) | Cancel that run or generation cooperatively. | | session.abortCurrentGeneration(reason) | Abort the active generation while retaining the session for later work. | | session.abortSession(reason, { abortBackgroundJobs: true }) | Stop the session's generations and request cancellation of its running background jobs. Job cancellation is opt-in. | | await session.close() | Close admission, cancel queued/active generations, settle persistence and tracked cleanup, and release listeners. Call it from the host, outside active tool, preparation, cleanup or compactor callbacks. |

Model transports, tools and resource callbacks must observe their supplied abort signals. Background jobs have separate lifecycles; closing a session does not automatically terminate them. Cleanup failures are observable. closeCleanupCompleted reports whether the session has released its retryable cleanup ownership.

The runtime retries a model call that failed with a rate limit (429), an overloaded provider (529) or an unavailable one (503) at most 8 times, waiting for the provider's retry-after delay but at least the backoff that doubles from 2 s to 30 s. A requested delay beyond 60 s, or one that would take the call's retries past 150 s in total, fails the generation: a rate limit (429) as a ModelLimitError of kind rate_limit carrying that retry time, a 503 or 529 with the provider error. A ModelLimitError of kind usage_limit, which the provider adapters raise for a spent quota, is never retried. Completed tool results are preserved during a generation's retry sequence. A ledger reuses an already observed execution for the same tool-call identity. Applications still need their own idempotency and recovery rules for external side effects.

maxSteps defaults to 20 and limits completed model steps per generation. Prompt-cache handling defaults to cache: 'auto'; use cache: false to disable the runtime's cache defaults, or pass an explicit cache policy and a stable sessionId for supported provider affinity.

Portable runner in the browser

Import /runner directly in browser code. It uses the same generation engine, completion validation and tool ledger, and always closes its internally owned session. It requires a secure context with Web Crypto, Web Streams and structured cloning. It exposes no AgentSession, Node file-store implementation or external session lifecycle controls.

import {
  runAgent as runPortableAgent,
  type IAgentRunOptions,
} from '@modelprofile.com/flexharness-agent/runner';

export function runInBrowser(
  model: IAgentRunOptions['model'],
  prompt: string,
  onToken: (delta: string) => void,
  abort?: AbortSignal,
) {
  return runPortableAgent({ model, prompt, onToken, abort, maxSteps: 5 });
}

The caller supplies a browser-compatible model and tools. A browser model can use an authenticated transport to your own server; keep provider secrets on that server. For ready-made conversation state and UI components, see @modelprofile.com/flexharness/chat and /chat/web.

Migrating from SmartAgent

Replace imports from @push.rocks/smartagent with @modelprofile.com/flexharness-agent. Install provider adapters from @modelprofile.com/flexharness-providers and create models through your application's ModelRegistry. Built-in tools and compaction helpers now use @modelprofile.com/flexharness subpaths; they are not exported by the agent package root. See the toolbox migration map for the full import mapping.

License and Legal Information

This repository contains open-source code licensed under the MIT License. A copy of the license can be found in the repository license file.

Please note: The MIT License does not grant permission to use the trade names, trademarks, service marks, or product names of the project, except as required for reasonable and customary use in describing the origin of the work and reproducing the content of the NOTICE file.

Trademarks

This project is owned and maintained by Task Venture Capital GmbH. The names and logos associated with Task Venture Capital GmbH and any related products or services are trademarks of Task Venture Capital GmbH or third parties, and are not included within the scope of the MIT license granted herein.

Use of these trademarks must comply with Task Venture Capital GmbH's Trademark Guidelines or the guidelines of the respective third-party owners, and any usage must be approved in writing. Third-party trademarks used herein are the property of their respective owners and used only in a descriptive manner, e.g. for an implementation of an API or similar.