freesolo-sdk

TypeScript SDK surface for source applications that need Freesolo tracing or custom evaluations.

This npm package intentionally contains only:

• tracing helpers for exporting OpenTelemetry spans to Freesolo
• evaluation primitives and EvaluationClient

It does not include Freesolo training, datasets, GEPA, Tinker, or generated Python training-repo helpers.

Tracing

import { setupTracing, traceApiCall } from "freesolo-sdk/tracing";

setupTracing({
  serviceName: "linkd-search",
  scorerBundleId: "scorer-bundle-id",
});

const response = await traceApiCall(
  "openai.chat.completions",
  {
    input: { messages },
    provider: "openai",
    model: "gpt-4o-mini",
  },
  async (trace) => {
    const result = await fetch("https://api.openai.com/v1/chat/completions", {
      method: "POST",
      body: JSON.stringify({ model: "gpt-4o-mini", messages }),
    }).then((r) => r.json());
    trace.setOutput(result);
    return result;
  },
);

setupTracing(...) is no-op safe when FREESOLO_API_KEY is absent.

Evaluation

import { BinaryResponse, CustomScorer, EvaluationClient } from "freesolo-sdk/evaluation";

class NonEmpty extends CustomScorer<BinaryResponse> {
  name = "non_empty";

  score(row: Record<string, unknown>) {
    return new BinaryResponse({
      value: Boolean(String(row.actual_output ?? "").trim()),
      reason: "actual_output is non-empty",
    });
  }
}

await new EvaluationClient().run({
  name: "local-check",
  data: [{ actual_output: "hello" }],
  scorers: [new NonEmpty()],
});