@tealbrick/vision

Configurable image analysis for Teal Brick. Node 24+, with optional native Eve 0.55.0 adapters. The core is host-independent; Codex/Claude MCP mounting remains separate adapter work.

Supports OpenAI-compatible Chat Completions and Responses endpoints using supplied PNG, JPEG or WebP images. Configure the full URL, protocol, model, credential reference, output-token budget and image limits. No default provider is selected. Protocol reference: OpenAI image inputs.

import {createVisionHandler} from '@tealbrick/vision';

const handle = createVisionHandler({
  auth: {issuer:'https://portal.example', org:'my-org', agent:'helper'},
  endpoints: [{
    url:'https://models.example/v1/chat/completions',
    credentialRefs:['vision-production'],
  }],
  // Implement in the trusted runtime's secret store, scoped to this binding.
  resolveCredential: async ({reference, org, agent, url}) =>
    secretStore.resolve({reference, org, agent, url}),
  // Read verified persisted settings on each request. UI/storage is host-owned.
  getConfig: async ({org, agent}) => ({
    version:1, enabled:true, displayName:'Vision',
    endpoint:{
      url:'https://models.example/v1/chat/completions',
      protocol:'openai-chat-completions', model:'your-vision-model',
      credentialRef:'vision-production',
    },
    maxImages:4, maxImageBytes:4194304, maxOutputTokens:1024,
    detail:'auto',
  }),
});
// Route authenticated GET to handle('manifest', request)
// and authenticated POST to handle('analyze', request).

secretStore above is an application-provided dependency. For Responses, configure protocol:'openai-responses' with the provider's full Responses URL. Chat servers requiring the older token field can set chatTokenParameter:'max_tokens'; the default is max_completion_tokens. Arbitrary custom provider protocols require a separately implemented adapter.

Analysis body:

{"prompt":"Read the visible text","images":[{"mediaType":"image/png","data":"BASE64_IMAGE_BYTES"}]}

Response: {ok:true,text,model,truncated}. Supplied image headers must match their MIME type; full decoding is delegated to the provider. The package does not fetch image URLs, read paths, capture screens, persist images or execute provider tool calls. Capture belongs to Local Runtime Bridge; durable documents belong to Knowledge. Model output is untrusted content, not authority to act.

Eve mounting

// agent/channels/vision.ts
import {visionChannel} from '@tealbrick/vision/eve';
import {options} from '../vision-options.js';
export default visionChannel(options);

Exposes GET /eve/v1/vision/manifest and POST /eve/v1/vision/analyze.

// agent/tools/vision.ts
import {visionTool} from '@tealbrick/vision/eve';
import {options, turnAuthorization} from '../vision-options.js';
export default visionTool({...options, getAuthorization: turnAuthorization});

The host must implement turnAuthorization(context) to supply signed Portal credentials for the current turn. Human access requires identity plus x-tealbrick-bundle with agent-use grants; agent access requires a callee/audience-bound A2A token. The same existing Portal verifier runs for HTTP and native tools. Labels and ambient owner credentials are not an authorization substitute. Native cancellation is propagated.

Endpoint and data boundaries

Voice and Vision reuse @tealbrick/provider-transport for independent endpoint/credential approval, secret resolution, redirect refusal, cancellation and bounded reads. Endpoints require HTTPS; operator-approved loopback HTTP is opt-in via allowLoopback:true. Anonymous endpoints require allowAnonymous:true. Card configuration alone cannot authorize an endpoint or credential. Browser callers cannot override the configured model or endpoint, and never receive provider credentials.

Defaults: four images, 4 MiB per image, 12 MiB total decoded input, 1,024 output tokens, 30-second deadline. Hard ceilings: eight images, 8 MiB per image, 12 MiB total, 8,192 output tokens and 120 seconds. Prompt limit: 16,000 characters. Provider response limit: 256 KiB and 64,000 text characters. Responses requests set store:false; a third-party provider's retention behavior remains its own contract.

Tests use disposable signed tokens and synthetic provider responses. Package discovery/build is separate from real-provider acceptance. Portal card UI/persistence and shipped Codex/Claude adapters are not included here.