MediaToolkit-JS is the JavaScript counterpart of Python media-toolkit: a unified API for loading, converting, and serializing images, audio, video, and arbitrary files across browser and Node environments.

It is the file layer behind socaity-js — API inputs and media results flow through these types automatically.

Perfect for: Socaity/APIPod frontends, generative-AI web apps, upload pipelines, and any project that needs one media abstraction instead of ad-hoc Blob/base64 handling.

Installation

npm install @socaity/media-toolkit

No native binaries. Two small runtime deps (magic-bytes.js, mime-types) for content detection.

Quickstart

One factory for all media types — load from paths, URLs, bytes, base64, Blobs, or API FileModel JSON:

import { MediaFileFactory, ImageFile, AudioFile } from '@socaity/media-toolkit';

// Smart content detection picks ImageFile, AudioFile, VideoFile, or MediaFile
const image = await MediaFileFactory.create('https://example.com/photo.jpg');
const audio = await MediaFileFactory.create('./welcome.mp3');

console.log(image.getInfo()); // { fileName, contentType, size, extension }
await image.save('output/photo.jpg');        // Node: write to disk
const b64 = image.toBase64();                // data URI for <img src="...">

Or construct a typed class directly when you know the kind:

const img = await ImageFile.create('./avatar.png');
const clip = await AudioFile.create('https://example.com/voice.wav');

Why MediaToolkit-JS

| | Manual Blobs / base64 | media-toolkit-js | |---|---|---| | Load from | You wire each source | Path, URL, base64, Blob, File, Buffer, FileModel JSON | | Type detection | Guess from extension | Magic bytes + MIME inference | | API payloads | Hand-build multipart/JSON | toJson() → Socaity FileModel; pass MediaFile to socaity-js | | Browser display | Create object URLs yourself | toImageElement(), toBlob(), toBase64() | | Batch inputs | Arrays of glue code | MediaList, MediaDict |

The Python library adds FFmpeg/OpenCV depth (frame streams, numpy, re-encoding). The JS library focuses on I/O, serialization, and web integration — the operations generative-AI clients actually need in the browser.

Loading files

The same entry points work in browser and Node (paths are Node-only):

import { MediaFileFactory } from '@socaity/media-toolkit';

// Local path (Node.js)
await MediaFileFactory.create('./assets/logo.png');

// URL (browser or Node — fetched automatically)
await MediaFileFactory.create('https://example.com/image.jpg');

// Base64 / data URI
await MediaFileFactory.create('data:image/jpeg;base64,/9j/4AAQ...');

// Binary
await MediaFileFactory.create(uint8Array);
await MediaFileFactory.create(arrayBuffer);

// Browser File / Blob from <input type="file">
await MediaFileFactory.create(fileFromInput);

// Socaity API FileModel (from a job result or request schema)
await MediaFileFactory.create({ file_name: 'out.png', content_type: 'image/png', content: '...' });

Export and serialize

Convert to whatever the next layer expects:

const mf = await MediaFileFactory.create('./photo.jpg');

mf.toBase64();           // data URI (default) or raw base64
mf.toArrayBuffer();
mf.toUint8Array();
mf.toBlob();             // browser
mf.toBuffer();           // Node.js
mf.toJson();             // Socaity FileModel for JSON/multipart APIs

mf.fileSize('mb');
mf.getInfo();
mf.setFileName('renamed.png');

await mf.save('./output/renamed.png'); // Node

Embed in the browser

import { ImageFile } from '@socaity/media-toolkit';

const generated = await ImageFile.create(apiResult);

// HTML string with inline base64 src
document.getElementById('preview')!.innerHTML = generated.toImageElement({ alt: 'Result' });

// Or wire up an existing <img>
const img = document.getElementById('myImg') as HTMLImageElement;
img.src = generated.toBase64();

Typed subclasses add media-specific helpers (ImageFile.toImageElement(), AudioFile.toAudioElement(), VideoFile.toVideoElement()).

Socaity integration

socaity-js re-exports this package and uses it end-to-end:

• Inputs — pass MediaFile, Blob, URL, base64, or path; the SDK formatter serializes to JSON FileModel or multipart upload as required.
• Outputs — FileModel payloads in job results are parsed back into typed ImageFile / AudioFile / VideoFile.

import { connect } from 'socaity';
import { MediaFileFactory } from '@socaity/media-toolkit';

const client = await connect('https://api.socaity.ai/services/v1/face2face');
const source = await MediaFileFactory.create('./face.jpg');
const target = await MediaFileFactory.create('./target.jpg');

const swapped = await client.submitJob('/swap-img-to-img', {
  source_img: source,
  target_img: target,
});

await swapped.save('swapped.jpg');

You rarely call toJson() yourself when using socaity-js — but it is there when you talk to the API directly.

Containers

MediaList — batch inputs

Lazy-loading list with batch convert/save:

import { MediaList, ImageFile } from '@socaity/media-toolkit';

const images = new MediaList<ImageFile>({
  files: ['./a.png', 'https://example.com/b.jpg', blobFromInput],
});

for (const img of images) {
  console.log(img.getInfo());
}

await images.save('./output'); // Node: save all with deduplicated names
const allBase64 = images.toBase64();

MediaDict — keyed media maps

import { MediaDict } from '@socaity/media-toolkit';

const assets = new MediaDict();
assets.set('profile', './profile.jpg');
assets.set('banner', 'https://example.com/banner.png');

const json = assets.toJson(); // Record<string, FileModel>
await assets.save('./export');

Typed media classes

| Class | When it is chosen | Extra helpers | |---|---|---| | ImageFile | image/* content | toImageElement(), dimension probing (browser) | | AudioFile | audio/* content | toAudioElement() | | VideoFile | video/* content | toVideoElement() | | Asset3DFile | model/*, glTF, etc. | Base MediaFile operations | | MediaFile | Everything else | Universal fallback |

MediaFileFactory.create() picks the class from magic bytes and MIME type.

Format support

| Category | Examples | Integration | |---|---|---| | Images | jpg, png, gif, webp, avif, svg, … | Deep — ImageFile | | Audio | wav, mp3, ogg, flac, aac, … | Deep — AudioFile | | Video | mp4, webm, mov, mkv, … | Deep — VideoFile | | 3D / docs / archives | glb, pdf, zip, … | Shallow — MediaFile (load, save, convert bytes) |

Deep = typed subclass with content validation and DOM helpers.
Shallow = universal file handle; no codec processing (that stays on the server / in Python media-toolkit).

Browser and Node.js

| | Browser | Node ≥ 20 | |---|---|---| | Import | import { MediaFileFactory } from '@socaity/media-toolkit' | Same | | UMD | <script src="media-toolkit.umd.js"> → globals | — | | File paths | Not available | create('./path'), save() | | URL fetch | fetch | fetch |

Pure web-platform APIs internally; Node paths use feature detection, not separate builds.

Example: examples/node_usage/main.js.

Key features

• Universal input — paths, URLs, bytes, base64, Blob/File, FileModel JSON, FileReader objects
• Automatic type detection — magic bytes + MIME inference → typed subclass
• Socaity-native serialization — toJson() / fromAny(FileModel) round-trip
• Batch containers — MediaList, MediaDict with lazy loading
• Lightweight — no FFmpeg/OpenCV; small bundle for frontend use
• Paired with socaity-js — same types the SDK imports and re-exports

Ecosystem

| Package | Role | |---|---| | media-toolkit | Python media processing (FFmpeg, OpenCV, numpy) | | media-toolkit-js (this repo) | JS media I/O for browser and Node | | socaity-js | Socaity/APIPod client; uses this as its file layer | | APIPod | Services accept and return FileModel / media types |

Contribute

Issues and pull requests welcome.

git clone https://github.com/SocAIty/media-toolkit-js.git
cd media-toolkit-js
npm install
npm run prod
npm run node_usage

License

MIT. See LICENSE.