volcanic-tools

Tools for the volcanic (minds) backend. This library provides a collection of modular utilities designed to be tree-shakeable.

Installation

npm install @volcanicminds/tools

Requirements

• Node.js >= 24.x
• ESM project ("type": "module")

How to upgrade packages

npm run upgrade-deps

Documentation

For detailed information about hardcoded limits, default configuration values (e.g., AI concurrency, chunk sizes, token window), and advanced setup, please refer to the docs/ folder:

• Configuration & Limits

Usage

This package supports both root imports and sub-path imports to optimize bundle size and tree-shaking.

Import specific features (Recommended)

import * as mfa from '@volcanicminds/tools/mfa'
import { Mailer } from '@volcanicminds/tools/mailer'
import * as logger from '@volcanicminds/tools/logger'
import { StorageManager } from '@volcanicminds/tools/storage'
import { TransferManager } from '@volcanicminds/tools/transfer'
import { createEmbedder, embedText, PgVectorStore } from '@volcanicminds/tools/ai'

Features

MFA (Multi-Factor Authentication)

Utilities for generating secrets, QR codes, and verifying TOTP tokens based on otpauth.

import * as mfa from '@volcanicminds/tools/mfa'

// 1. Generate a generic base32 secret (Optional, useful if you need to store it before setup)
const secret = mfa.generateSecret()

// 2. Generate Setup Details for the User (returns secret, otpauth URI, and QR Code Data URL)
// If secret is omitted, a new one is generated automatically.
const setup = await mfa.generateSetupDetails('MyApp', '[email protected]', secret)

console.log(setup.secret) // Save this to DB
console.log(setup.qrCode) // Send this to Frontend to display QR

// 3. Verify a Token provided by the user
const userToken = '123456' // From input
const isValid = mfa.verifyToken(userToken, setup.secret)

if (isValid) {
  // Proceed with login/action
}

// 4. Generate a valid token (Useful for testing or recovery codes)
const currentToken = mfa.generateToken(setup.secret)

Mailer

A wrapper around nodemailer designed for simplicity and configuration injection. It automatically handles HTML-to-Text conversion if the text body is missing.

Configuration & Initialization

import { Mailer } from '@volcanicminds/tools/mailer'

// Initialize with a config object (not bound to process.env)
const mailer = new Mailer({
  host: 'smtp.example.com',
  port: 587,
  secure: false, // true for 465, false for other ports
  auth: {
    user: 'my-user',
    pass: 'my-password'
  },
  defaultFrom: '"My Service" <[email protected]>', // Optional: used if not specified in send()
  defaultReplyTo: '[email protected]' // Optional
})

// Optional: Verify connection on startup
const isConnected = await mailer.verifyConnection()
if (isConnected) console.log('SMTP Ready')

Sending Emails

try {
  const info = await mailer.send({
    // Optional if defaultFrom is set in config, otherwise Mandatory
    from: '"Support Team" <[email protected]>',

    to: '[email protected]', // Can be a string or array of strings
    cc: ['[email protected]'],
    subject: 'Welcome to Volcanic Tools',

    // Text version is automatically generated from HTML if omitted,
    // converting <br> to newlines and stripping tags.
    text: 'Hello, World! Welcome aboard.',
    html: '<p>Hello, <strong>World</strong>!<br/>Welcome aboard.</p>',

    attachments: [
      {
        filename: 'license.txt',
        content: 'MIT License...'
      }
    ]
  })

  console.log('Message sent: %s', info.messageId)
} catch (error) {
  console.error('Error sending email:', error)
}

Logging

Use Pino logger wrapper if in your project you have a global.log with a valid instance.

import * as log from '@volcanicminds/tools/logger'

log.info('Application started')
log.error({ err: new Error('Oops') }, 'Something went wrong')

Storage (S3 / Minio)

A robust wrapper around the minio client to handle file operations on S3-compatible storage.

import { StorageManager } from '@volcanicminds/tools/storage'

const storage = new StorageManager({
  endPoint: 'minio.example.com',
  port: 9000,
  useSSL: true,
  accessKey: 'minioadmin',
  secretKey: 'minioadmin',
  bucket: 'my-bucket',
  region: 'us-east-1'
})

// Check connection
const isConnected = await storage.verifyConnection() // true/false

// Upload a file (Stream, Buffer, or Path)
const info = await storage.uploadFile('folder/image.png', fileBuffer, {
  contentType: 'image/png',
  metadata: { userId: '123' }
})
console.log('ETag:', info.etag)

// Generate Presigned URLs (for frontend direct access)
const downloadUrl = await storage.getFileUrl('folder/image.png', 3600) // Expires in 1h
const uploadUrl = await storage.getUploadUrl('folder/new-image.png', 3600)

// Check existence
const exists = await storage.fileExists('folder/image.png')

// Delete
await storage.deleteFile('folder/image.png')

Transfer (Resumable Uploads - Tus.io)

A wrapper around @tus/server to implement resumable file uploads (standard protocol). Supports both local filesystem and S3 backends.

Initialization

import { TransferManager } from '@volcanicminds/tools/transfer'

const transfer = new TransferManager({
  driver: 'local', // or 's3'
  path: '/files', // The HTTP endpoint path (e.g. http://localhost:3000/files)
  maxSize: 10 * 1024 * 1024 * 1024, // 10 GB

  // If driver is 'local'
  local: {
    directory: './uploads'
  },

  // If driver is 's3'
  s3: {
    bucket: 'uploads',
    endPoint: 'minio.example.com',
    accessKey: '...',
    secretKey: '...',
    partSize: 8 * 1024 * 1024 // 8MB chunks
  }
})

Integration with Fastify/Node Key

The transfer instance exposes a standard Node.js request handler (handle). You can use it within a Fastify route or raw Node server.

// Fastify Example
fastify.all('/files/*', async (req, reply) => {
  // Pass the raw Node.js request/response objects to Tus
  await transfer.handle(req.raw, reply.raw)
  // Prevent Fastify from sending a response, Tus handles it
  reply.sent = true
})

Events

You can listen for upload lifecycle events:

transfer.onUploadCreate((upload, req, res) => {
  console.log('Upload started:', upload.id)
})

transfer.onUploadFinish((upload, req, res) => {
  console.log('Upload finished:', upload.id)
})

AI Module (New)

The AI module provides a standardized way to create AI models and agents, wrapping the Vercel AI SDK and Mastra.

Features:

• Unified Model Factory: Create models with createModel supporting OpenAI, Mistral, Ollama, Anthropic, Google.
• Environment Variable Fallback: Automatically uses AI_PROVIDER, OPENAI_API_KEY, etc. if no config is provided.
• Mastra Agent Wrapper: createAgent simplifies Mastra agent creation with Volcanic configuration.
• Concurrency Guard: Manage concurrent AI requests per provider to avoid rate limits.
• Embeddings: createEmbedder / embedText / embedTexts produce vectors (OpenAI, Mistral, Google, Ollama), parametrizable via config or env.
• Vector store: PgVectorStore, an engine‑agnostic pgvector helper that runs identically on real Postgres and embedded PGlite.

Usage:

import { createModel, createAgent, ConcurrencyGuard } from '@volcanicminds/tools/ai'

// 1. Create a Model (uses Env vars by default)
const model = await createModel()

// 2. Create an Agent
const agent = await createAgent({
  name: 'Auditor',
  instructions: 'You are an auditor...',
  model: model // or config object
})

// 3. Concurrency Control
const guard = new ConcurrencyGuard()
await guard.run('openai', async () => {
  // critical section
})

Installation:

You must install the peer dependencies:

npm install ai @mastra/core

And the provider SDKs you need.

Example for OpenAI:

npm install @ai-sdk/openai

Example for Anthropic:

npm install @ai-sdk/anthropic

Example for Google (Gemini):

npm install @ai-sdk/google

Example for Ollama:

npm install ai-sdk-ollama

Embeddings & Vector Search (pgvector)

Generate embeddings and run similarity search. The vector store is engine‑agnostic: give it any pg‑compatible query executor — a TypeORM dataSource.query, an embedded PGlite instance, or a node‑postgres pool — and the same code runs on a real Postgres (with pgvector installed) and on PGlite (with vector: true). Everything is parametrizable via config: provider/model for embeddings, table name, vector dimensions, and distance (cosine | l2 | ip).

import { createEmbedder, embedText, PgVectorStore } from '@volcanicminds/tools/ai'

// 1. Embeddings — provider/model resolved from config or env
//    (AI_EMBEDDING_PROVIDER falls back to AI_PROVIDER; e.g. OPENAI_EMBEDDING_MODEL).
const vector = await embedText('hello world')               // number[]
// const embedder = await createEmbedder({ provider: 'openai', model: 'text-embedding-3-small' })

// 2. Vector store — works on Postgres or embedded PGlite unchanged
const store = new PgVectorStore({
  query: (sql, params) => dataSource.query(sql, params),     // any pg-compatible executor
  table: 'documents',
  dimensions: 1536,                                          // match your embedding model
  distance: 'cosine',
  // schema: 'tenant_a'                                      // optional, for multi-tenant isolation
})

await store.init()                                          // CREATE EXTENSION vector + table (idempotent)
await store.upsert('doc-1', 'the text', vector, { source: 'docs' })
const matches = await store.search(await embedText('a query'), 5)
//   -> [{ id, content, metadata, distance }, ...] nearest first

Installation: the store itself needs only your DB executor. For embeddings install ai + a provider SDK (see below). To back it with embedded PGlite use the backend's type: 'pglite' engine (vector: true) — see @volcanicminds/backend docs/PGLITE.md.

Advanced Model Examples

1. Anthropic (Claude 3.5 Sonnet)

Configured via Environment Variables:

AI_PROVIDER=anthropic
ANTHROPIC_API_KEY=sk-ant-...
ANTHROPIC_MODEL=claude-3-5-sonnet-20240620

Or explicit configuration:

const model = await createModel({
  provider: 'anthropic',
  apiKey: 'sk-ant-...',
  model: 'claude-3-5-sonnet-20240620'
})

2. Google (Gemini 1.5 Pro)

Configured via Environment Variables:

AI_PROVIDER=google
GOOGLE_API_KEY=AIza...
GOOGLE_MODEL=models/gemini-1.5-pro-latest

Or explicit configuration:

const model = await createModel({
  provider: 'google',
  apiKey: 'AIza...',
  model: 'models/gemini-1.5-pro-latest'
})

3. Ollama (Llama 3 Local)

Configured via Environment Variables:

AI_PROVIDER=ollama
OLLAMA_BASE_URL=http://localhost:11434/api
OLLAMA_MODEL=llama3

Or explicit configuration:

const model = await createModel({
  provider: 'ollama',
  baseUrl: 'http://localhost:11434/api',
  model: 'llama3'
})