bare-needle

Bare addon wrapping Needle 3 for local completion, tool calling, and embeddings.

It runs under Bare only, not Node, on these platforms:

| Platform | Minimum | | ---------------------------------- | ------------------- | | darwin-arm64 | macOS 26 | | ios-arm64, ios-arm64-simulator | iOS 27 | | android-arm64 | Android 10 (API 29) | | linux-x64, linux-arm64 | glibc 2.35 | | win32-x64, win32-arm64 | Windows 10 |

Needle 3 ships no engine for darwin-x64, ios-x64-simulator, android-x64 or android-ia32. Its android-armv7 engine needs a libc++ symbol no Android NDK provides.

Do not use the generic Cactus / Unsloth convert path. Needle has its own engine, .cact archive, and needle CLI.

Install

npm install bare-needle

The package ships a prebuild for each platform above, so installing it needs no toolchain. The weights are separate: download needle3.cact as in Building from source, or use a tuned archive from needle build.

Usage

Every call returns a promise and runs on a worker thread, one at a time, because Needle holds a single process-global model:

const needle = require('bare-needle')

await needle.loadFile('needle3.cact')
await needle.init({
  system: 'You are a helpful assistant.',
  tools: [
    {
      name: 'get_weather',
      description: 'Get the current weather for a city.',
      parameters: {
        type: 'object',
        properties: { city: { type: 'string' } },
        required: ['city']
      }
    }
  ]
})

const result = await needle.complete("what's it like in Lagos right now?")

result is the engine's JSON, parsed:

{
  "type": "call",
  "success": true,
  "error": null,
  "error_code": null,
  "reason": null,
  "function_calls": [{ "name": "get_weather", "arguments": { "city": "Lagos" } }],
  "suppressed_calls": [],
  "reasoning": "'Lagos' -> city 'Lagos'. No other tool needed.",
  "confidence": 0.8504,
  "prefill_tps": 3165.1,
  "decode_tps": 1147.8,
  "peak_ram_mb": 109.5,
  "validation": { "ungrounded": [], "negation": false }
}

A message no tool fits comes back with an empty function_calls, for example "tell me a joke about cats" gives "reasoning": "No tool available for joke generation.".

API

await needle.loadFile(path)

Maps a .cact archive and loads it. Rejects naming the path when it cannot be opened.

await needle.load(buffer)

Loads a .cact already in memory, from an ArrayBuffer, Buffer or typed array.

const prefix = await needle.init({ system, tools, toolIndexPath })

Sets the system prompt and the tools, as an array or a JSON string, and returns the tokenized prefix length. Rejects when the prompt and tools do not fit the context window. Call it again to change the tools. toolIndexPath is optional and passed through to needle_init.

const result = await needle.complete(text, { maxNewTokens = 512, bufferSize = 65536 })

Answers one message against the tools from init. bufferSize is the bytes reserved for the result JSON.

const vector = await needle.embed(text)

Returns the text's embedding as a Float32Array (3072 floats for needle3.cact). Tuned archives built without the embedding head reject.

await needle.reset()

Resets the engine's state between unrelated messages.

Calls made before load or loadFile reject with call load() or loadFile() first, and errors carry the engine's own reason.

Building from source

Needs Node, Bare (>= 1.32) and a C toolchain. CMake comes in via npm (cmake-bare). Linux needs clang and libc++ 21, since Needle's engine is built against them and the addon links libc++ statically. Android needs -D ANDROID_STL=c++_static. Needle's Windows engine is a MinGW build, so on Windows the build links it into needle.dll with llvm-mingw (found on PATH or under LLVM_MINGW_ROOT), and the addon loads that DLL from prebuilds/win32-<arch>/bare-needle/. Cross-compile with bare-make generate --platform <platform> --arch <arch>, adding --simulator for the iOS simulator.

npm run build downloads needle.h and the target's libneedle.a from Hugging Face (Cactus-Compute/needle3, pinned by revision and SHA-256 in CMakeLists.txt). Point -DNEEDLE_SDK_DIR=… at a local SDK for the target to build against that instead.

The weights are not part of the build. The tests read test/fixtures/needle3.cact, and the example and eval read ../needle3-hf/macos-arm64/needle3.cact:

npm install
pip install cactus-needle
needle download macos-arm64 --out ../needle3-hf
cp ../needle3-hf/macos-arm64/needle3.cact test/fixtures/

npm run build
npm test
npm run example

Example and eval

example.js loads that needle3.cact and runs a get_weather complete. Eval against a JSONL:

npm run eval -- --model ../needle3-hf/macos-arm64/needle3.cact

Default --data is ../data/create-agent/eval.jsonl (sibling of this repo in the action-runner tree).

Desktop QVAC (qvac-app branch experiment/action-runner) packs a tuned archive next to the Needle sidecar (NEEDLE_CACT_PATH, default ../action-runner/tuned.cact). Sidecar usage is loadFile then init with the create-agent tools JSON, then reset + complete per utterance.

Fine-tune

Local LoRA + 4-bit export: docs/finetune.md.

License

Apache-2.0. The prebuild bundles the Needle 3 engine from Cactus Compute, also Apache-2.0; see NOTICE.md.