llm

Package to connect and trace LLM calls.

Vision utilities

This package also contains utilities for vision.

Query

Ask a question against the image (e.g. to extract some info, make a decision) and get the answer.

import { query } from "@empiricalrun/llm/vision";

// With Appium
const data = await driver.saveScreenshot("dummy.png");
const instruction =
  "Extract number of ATOM tokens from the image. Return only the number.";

const text = await query(data.toString("base64"), instruction);
// Example response: "0.01"