clipkit-embeddings

Fast CLIP text and image embedding toolkit for Node.js and browser using ONNX.

clipkit-embeddings is a TypeScript library and CLI that enables high-performance CLIP embeddings for both text and images, powered by onnxruntime-node. It allows you to use CLIP models offline for semantic search, similarity computation, or zero-shot classification — both programmatically and via the command line.

Features

• Embed text using clip-text.onnx
• Embed images using clip-image.onnx
• Cosine similarity for comparing embeddings
• Fast inference with ONNXRuntime
• CLI tool for quick use in scripts
• Works in Node.js and browser (with bundler)

Installation

As a library

npm install clipkit-embeddings

As a CLI tool

npm install -g clipkit-embeddings

Usage

Text Embedding

const { loadTextModel, embedText } = require("clipkit-embeddings");

await loadTextModel("onnx/clip-text.onnx");
const embedding = await embedText("a photo of a dog");
console.log(embedding);

Image Embedding

const { loadImageModel, embedImage } = require("clipkit-embeddings");
const { preprocessImage } = require("clipkit-embeddings/utils");

await loadImageModel("onnx/clip-image.onnx");
const tensor = await preprocessImage("test/cat.jpg");
const embedding = await embedImage(tensor);
console.log(embedding);

Cosine Similarity

const { cosineSimilarity } = require("clipkit-embeddings/utils");

const similarity = cosineSimilarity(imageEmbedding, textEmbedding);
console.log("Cosine similarity:", similarity.toFixed(4));

CLI Usage

Embed text:

clipkit text "a photo of a cat"

Embed image:

clipkit image ./path/to/image.jpg

Project Structure

clipkit-embeddings/
├── cli/              # CLI logic
├── src/              # Core logic (text/image/utils)
├── test/             # Sample test files and images
├── onnx/             # Model files (.onnx)
│   ├── clip-text.onnx
│   └── clip-image.onnx
├── package.json
├── tsconfig.json
├── README.md
└── LICENSE

Contributing

Contributions, issues and feature requests are welcome!

Please open an issue to discuss any major changes beforehand.

References

• CLIP Paper: https://openai.com/research/clip
• ONNX Runtime: https://onnxruntime.ai/
• onnxruntime-node: https://www.npmjs.com/package/onnxruntime-node
• Sharp image processing: https://www.npmjs.com/package/sharp

License

This project is licensed under the MIT License.

Author

Divyansh Saraswat

• GitHub: https://github.com/divyanshsaraswat
• LinkedIn: https://linkedin.com/in/imdivyanshmv
• Twitter: https://twitter.com/imdivyanshmv