@kadi.build/file-sharing

File sharing service with tunneling, authentication, and a local S3-compatible interface.
Integrates @kadi.build/file-manager and @kadi.build/tunnel-services into one turnkey solution.

Features

• HTTP File Server — Static file serving with directory listing, range requests, CORS, and multi-scheme authentication
• Local S3-Compatible API — Emulates AWS S3 endpoints locally so you can use @aws-sdk/client-s3 against your local filesystem
• Extensible HTTP Pipeline — Register custom middleware and routes (e.g. Docker Registry v2 endpoints) alongside built-in file serving
• Tunnel Integration — Expose your local server publicly via KĀDI (default), ngrok, serveo, localtunnel, or pinggy
• Authentication — Basic Auth, Bearer Token, and API Key (header / query param) for HTTP; AWS SigV4 / SigV2 / Bearer / Basic for S3
• Temporary Credentials — Generate time-limited S3 credentials for secure sharing
• Secrets Management — Automatic .env loading (walks up parent directories for monorepo support) + environment variables + explicit config
• Download Monitoring — Track active downloads, progress, speed, and completion statistics
• Graceful Shutdown — Priority-ordered shutdown with timeout and force-kill
• Webhook Notifications — Event-driven notifications to external endpoints
• Monitoring Dashboard — Console-based real-time dashboard

Installation

npm install @kadi.build/file-sharing

Quick Start

Basic File Sharing

import { FileSharingServer } from '@kadi.build/file-sharing';

const server = new FileSharingServer({
  staticDir: '/path/to/files',
  port: 3000
});

await server.start();
console.log(`Serving files at ${server.getInfo().localUrl}`);
await server.stop();

With Public Tunnel (KĀDI)

const server = new FileSharingServer({
  staticDir: '/path/to/files',
  port: 3000,
  tunnel: {
    enabled: true
    // KĀDI is the default — reads token from KADI_TUNNEL_TOKEN env var or .env
  }
});

await server.start();
console.log(`Public URL: ${server.tunnelUrl}`);

With Authentication

const server = new FileSharingServer({
  staticDir: '/path/to/files',
  port: 3000,
  auth: { apiKey: 'my-secret-key' },  // or { username: 'admin', password: 'secret' }
  tunnel: { enabled: true }
});

await server.start();
// Clients must send X-API-Key header, Bearer token, or ?apiKey= query param

With Local S3 API

import { FileSharingServer } from '@kadi.build/file-sharing';
import { S3Client, ListObjectsV2Command, PutObjectCommand } from '@aws-sdk/client-s3';

const server = new FileSharingServer({
  staticDir: '/path/to/files',
  port: 3000,
  enableS3: true,
  s3Port: 9000
});

await server.start();

// Use the standard AWS SDK against your LOCAL filesystem
const s3 = new S3Client({
  endpoint: 'http://localhost:9000',
  region: 'us-east-1',
  credentials: { accessKeyId: 'minioadmin', secretAccessKey: 'minioadmin' },
  forcePathStyle: true
});

const response = await s3.send(new ListObjectsV2Command({ Bucket: 'local' }));
console.log(response.Contents);

await s3.send(new PutObjectCommand({
  Bucket: 'local',
  Key: 'uploaded.txt',
  Body: 'Hello from S3!'
}));

Quick Share (One-Liner)

import { createQuickShare } from '@kadi.build/file-sharing';

const { server, localUrl, publicUrl } = await createQuickShare('./my-files', {
  tunnel: true,
  auth: { apiKey: 'share-key-123' }
});

console.log(`Share this link: ${publicUrl}`);

Secrets & Authentication

How Secrets Are Loaded

As of v1.3.0, secrets are resolved using a 3-tier vault pattern with fallback:

| Priority | Source | What it provides | |----------|--------|------------------| | 1 (highest) | process.env | Direct overrides at startup | | 2 | Constructor config | Values passed to FileSharingServer | | 3 | Vault (secrets.toml) | Encrypted tokens via secret-ability | | 4 | config.yml | Non-secret settings (walks up from CWD) | | 5 (fallback) | .env file | Legacy fallback (walks up from CWD) |

The caller (typically deploy-ability or kadi-deploy) must have secret-ability installed via kadi install for vault access. If no vault is found, the system gracefully falls back to .env files.

Vault Setup

Store tunnel tokens in the tunnel vault and file-sharing secrets in the file-sharing vault:

# Tunnel tokens (shared with tunnel-services)
kadi secret set -v tunnel KADI_TUNNEL_TOKEN <your-token>
kadi secret set -v tunnel NGROK_AUTH_TOKEN <your-token>

# File-sharing secrets (optional)
kadi secret set -v file-sharing KADI_AUTH_API_KEY <your-api-key>
kadi secret set -v file-sharing KADI_S3_ACCESS_KEY <your-key>
kadi secret set -v file-sharing KADI_S3_SECRET_KEY <your-secret>

config.yml

Place a config.yml in your project root (or any parent directory):

# Tunnel config (shared across tunnel-services consumers)
tunnel:
  server_addr: broker.kadi.build
  tunnel_domain: tunnel.kadi.build
  server_port: 7000
  ssh_port: 2200
  mode: frpc
  transport: wss
  wss_control_host: tunnel-control.kadi.build
  agent_id: kadi

# File-sharing specific config
file-sharing:
  port: 3000
  host: 0.0.0.0
  enable_directory_listing: true
  cors: true
  enable_s3: false

Environment Variables (overrides)

| Variable | Description | Default | |----------|-------------|---------| | Tunnel — KĀDI | | | | KADI_TUNNEL_TOKEN | KĀDI authentication token | (required for KĀDI) | | KADI_TUNNEL_SERVER | KĀDI broker address | broker.kadi.build | | KADI_TUNNEL_DOMAIN | Tunnel domain | tunnel.kadi.build | | KADI_TUNNEL_PORT | KĀDI frps server port | 7000 | | KADI_TUNNEL_SSH_PORT | KĀDI SSH gateway port | 2200 | | KADI_TUNNEL_MODE | Connection mode: ssh, frpc, or auto | auto | | KADI_TUNNEL_TRANSPORT | Transport protocol: wss or tcp | wss | | KADI_TUNNEL_WSS_HOST | WSS gateway hostname | — | | KADI_AGENT_ID | Agent identifier for proxy naming | kadi | | Tunnel — Ngrok | | | | NGROK_AUTHTOKEN | Ngrok auth token (also accepts NGROK_AUTH_TOKEN) | — | | HTTP Authentication | | | | KADI_AUTH_API_KEY | API key for Bearer / X-API-Key auth | — | | KADI_AUTH_USERNAME | HTTP Basic auth username | — | | KADI_AUTH_PASSWORD | HTTP Basic auth password | — | | S3 Authentication | | | | KADI_S3_ACCESS_KEY | S3 access key ID | minioadmin | | KADI_S3_SECRET_KEY | S3 secret access key | minioadmin |

HTTP Authentication Schemes

When auth is configured (via config, env vars, or .env), the HTTP server supports three authentication schemes:

| Scheme | How to Authenticate | |--------|---------------------| | Basic Auth | Authorization: Basic <base64(user:pass)> | | Bearer Token | Authorization: Bearer <apiKey> | | API Key | X-API-Key: <apiKey> header or ?apiKey=<key> query param |

If auth.apiKey is set, all three token-based methods are accepted.
If auth.username + auth.password are set, only Basic Auth is accepted.

S3 Authentication

When custom S3 credentials are set (anything other than the default minioadmin/minioadmin), the S3 server validates requests:

| Method | How It Works | |--------|-------------| | AWS SigV4 | Standard Authorization: AWS4-HMAC-SHA256 Credential=<accessKeyId>/... | | AWS SigV2 | Legacy Authorization: AWS <accessKeyId>:signature | | Bearer | Convenience Authorization: Bearer <accessKeyId> | | Basic | Docker-style Authorization: Basic base64(accessKeyId:secretAccessKey) | | Pre-signed URL | ?X-Amz-Credential=<accessKeyId>/... query param |

With default credentials (minioadmin/minioadmin), auth is skipped for backward compatibility unless you set enforceAuth: true in the S3 config.

API Reference

FileSharingServer

Main orchestrating class.

const server = new FileSharingServer({
  // File serving
  staticDir: process.cwd(),        // Directory to serve
  port: 3000,                      // HTTP port
  host: '0.0.0.0',                 // Bind address
  enableDirectoryListing: true,    // Show directory listing
  cors: true,                      // Enable CORS

  // Authentication (or use env vars / .env)
  auth: null,                      // { apiKey: 'key' } or { username: 'u', password: 'p' }

  // S3 API
  enableS3: false,                 // Enable S3-compatible API
  s3Port: 9000,                    // S3 API port
  s3Config: {                      // Passed to S3Server
    accessKeyId: 'minioadmin',
    secretAccessKey: 'minioadmin',
    bucketName: 'local',
    region: 'us-east-1',
    enforceAuth: false             // Force auth even with default creds
  },

  // Tunnel
  tunnel: {
    enabled: false,                // Start tunnel on server.start()
    service: 'kadi',               // 'kadi' | 'ngrok' | 'serveo' | 'localtunnel' | 'pinggy'
    autoFallback: true,            // Fall back to other services on failure
    autoReconnect: true,           // Auto-reconnect on tunnel drop
    reconnectDelay: 5000,          // ms between reconnect attempts
    // KĀDI-specific (or use env vars / .env)
    kadiToken: undefined,
    kadiServer: undefined,
    kadiDomain: undefined,
    kadiPort: undefined,           // frps server port (NOT the local port)
    kadiSshPort: undefined,
    kadiMode: undefined,           // 'ssh' | 'frpc' | 'auto'
    kadiAgentId: undefined,
    kadiTransport: undefined,       // 'wss' (default) | 'tcp'
    kadiWssControlHost: undefined,  // WSS gateway hostname
    // Ngrok-specific
    ngrokAuthToken: undefined,
    // Advanced
    managerOptions: {}             // Extra options passed to TunnelManager
  },

  // Shutdown
  shutdown: {
    gracefulTimeout: 30000,
    finishActiveDownloads: true,
    forceKillTimeout: 60000
  },

  // Monitoring
  monitoring: {
    enabled: true,
    dashboard: false,              // Enable console dashboard
    webhooks: []                   // ['https://hooks.example.com/events']
  }
});

Methods

| Method | Returns | Description | |--------|---------|-------------| | start() | Promise<ServerInfo> | Start HTTP server (+ S3 and tunnel if enabled) | | stop() | Promise<void> | Gracefully stop all servers and tunnels | | enableTunnel(options?) | Promise<TunnelInfo> | Enable public tunnel after start | | disableTunnel() | Promise<void> | Close active tunnel | | enableS3(options?) | Promise<{endpoint, port}> | Enable S3 API dynamically | | disableS3() | Promise<void> | Disable S3 API | | getInfo() | ServerInfo | Get server info, URLs, and tunnel status | | getStats() | DownloadStats | Get download statistics | | listFiles(subPath?) | Promise<FileEntry[]> | List files in served directory | | addWebhook(url, events?) | void | Register a webhook endpoint | | removeWebhook(url) | void | Remove a webhook |

Properties

| Property | Type | Description | |----------|------|-------------| | isRunning | boolean | Whether the server is running | | port | number | Current HTTP port | | tunnelUrl | string \| null | Current public tunnel URL | | staticDir | string | Directory being served | | tunnel | object \| null | Active tunnel info | | tunnelManager | TunnelManager | Underlying tunnel manager |

Events

| Event | Payload | When | |-------|---------|------| | started | ServerInfo | Server fully started | | stopping | — | Shutdown initiated | | stopped | — | Server fully stopped | | download:start | {id, file, ...} | File download begins | | download:complete | {id, file, duration, ...} | Download finished | | download:error | {id, error} | Download failed | | upload:complete | {file, size} | S3 upload finished | | s3:started | {port, endpoint} | S3 server started | | s3:get | {bucket, key} | S3 GetObject | | s3:put | {bucket, key} | S3 PutObject | | s3:delete | {bucket, key} | S3 DeleteObject | | bucket:removed | {bucket} | Bucket programmatically removed | | credentials:generated | {accessKey, expiresAt} | Temporary credentials created | | middleware:added | {name, priority} | HTTP middleware registered | | middleware:removed | {name} | HTTP middleware removed | | route:added | {method, path, priority} | Custom HTTP route registered | | route:removed | {method, path} | Custom HTTP route removed | | tunnel:created | TunnelInfo | Tunnel established | | tunnel:closed | — | Tunnel shut down | | tunnel:error | Error | Tunnel failed (non-fatal) | | http:started | {port, host} | HTTP server listening | | http:error | Error | HTTP error |

HttpServerProvider

Low-level HTTP file server with authentication and extensibility.

import { HttpServerProvider } from '@kadi.build/file-sharing/http';

const httpServer = new HttpServerProvider({
  port: 3000,
  staticDir: './files',
  enableDirectoryListing: true,
  cors: true,
  auth: { apiKey: 'my-key' }       // or { username: 'u', password: 'p' }
});

await httpServer.start();

Middleware (Extensibility)

Register middleware that runs before built-in auth and static file handling. Higher priority runs first.

// Add authentication middleware (priority 10 = runs before default handlers)
httpServer.addMiddleware('dockerAuth', (req, res, next) => {
  if (req.url.startsWith('/v2')) {
    const auth = req.headers.authorization;
    if (!auth || !isValidToken(auth)) {
      res.writeHead(401);
      res.end(JSON.stringify({ errors: [{ code: 'UNAUTHORIZED' }] }));
      return;
    }
    req.user = { token: auth };
  }
  next();
}, { priority: 10 });

// List registered middleware
httpServer.getMiddleware(); // [{ name: 'dockerAuth', priority: 10 }]

// Remove middleware
httpServer.removeMiddleware('dockerAuth');

Custom Routes (Extensibility)

Register custom route handlers with Express-style :param path patterns. Custom routes match before built-in static file serving.

// Docker Registry v2 ping
httpServer.addCustomRoute('GET', '/v2/', (req, res) => {
  res.writeHead(200, { 'Content-Type': 'application/json' });
  res.end('{}');
});

// Route with path parameters — req.params is populated automatically
httpServer.addCustomRoute('GET', '/v2/:name/manifests/:reference', (req, res) => {
  console.log(req.params); // { name: 'myapp', reference: 'latest' }
  res.writeHead(200, { 'Content-Type': 'application/json' });
  res.end(JSON.stringify(getManifest(req.params.name, req.params.reference)));
});

// HEAD method support
httpServer.addCustomRoute('HEAD', '/v2/:name/blobs/:digest', (req, res) => {
  res.writeHead(200, { 'Docker-Content-Digest': req.params.digest });
  res.end();
});

// List / remove routes
httpServer.getCustomRoutes();                   // [{ method, path, priority }]
httpServer.removeCustomRoute('GET', '/v2/');     // true if found

Request Pipeline Order

Request → CORS → Middleware Chain → Custom Routes → Built-in Auth → Static Files

S3Server

Local S3-compatible API server. Files are stored on disk, not on AWS.

import { S3Server } from '@kadi.build/file-sharing/s3';

const s3 = new S3Server({
  port: 9000,
  rootDir: './data',
  bucketName: 'local',
  region: 'us-east-1',
  accessKeyId: 'my-key',           // default: 'minioadmin'
  secretAccessKey: 'my-secret',    // default: 'minioadmin'
  enforceAuth: true                // validate even with default creds
});

const info = await s3.start();
console.log(info.serverId);         // 'kadi-s3-9000'

Supported S3 Operations:

• ListBuckets, CreateBucket, DeleteBucket, HeadBucket
• ListObjects, ListObjectsV2
• GetObject (with range requests)
• PutObject
• DeleteObject
• HeadObject
• CreateMultipartUpload, UploadPart, CompleteMultipartUpload

Programmatic Authentication Validation

Validate incoming requests against configured credentials. Returns a structured result — useful for layering custom auth (e.g. Docker Registry) on top of S3.

const result = s3.validateAuthentication(req);
// Success: { success: true, user: { accessKey: 'AKIA...' } }
// Failure: { success: false, error: 'Invalid credentials' }

Supports AWS SigV4, SigV2, Bearer, Basic Auth (Docker clients), and pre-signed URL query params.

Temporary Credentials

Generate time-limited credentials for secure sharing:

const creds = s3.generateTemporaryCredentials({ ttl: 3600 }); // 1 hour
console.log(creds.accessKey);    // 'AKIA...' (random)
console.log(creds.secretKey);    // random hex string
console.log(creds.expiresAt);    // '2026-02-16T...' (ISO string)
console.log(creds.expiry);       // Date object (backward compat)

// Temp credentials are automatically validated by _checkS3Auth and validateAuthentication

Programmatic Bucket Removal

Force-delete a bucket directory and all its contents (for cleanup):

await s3.removeBucket('container-id-123');
// Emits 'bucket:removed' event. Idempotent (no error if bucket doesn't exist).

Properties

| Property | Type | Description | |----------|------|-------------| | serverId | string | Stable server identifier (e.g. kadi-s3-9000) | | isRunning | boolean | Whether the server is running |

createQuickShare

One-liner to share a directory with optional tunnel and auth.

import { createQuickShare } from '@kadi.build/file-sharing';

const { server, localUrl, publicUrl } = await createQuickShare('./my-files', {
  port: 3000,                       // HTTP port (default: 3000)
  tunnel: true,                     // Enable tunnel (default: false)
  tunnelService: 'kadi',            // Tunnel service (default: 'kadi')
  kadiToken: 'token',               // Or set KADI_TUNNEL_TOKEN env
  ngrokAuthToken: 'token',          // Or set NGROK_AUTHTOKEN env
  auth: { apiKey: 'share-key' },    // Protect downloads
  tunnelOptions: {}                 // Extra TunnelManager options
});

console.log(`Local:  ${localUrl}`);
console.log(`Public: ${publicUrl}`);

DownloadMonitor

Track download progress and statistics.

import { DownloadMonitor } from '@kadi.build/file-sharing';

const monitor = new DownloadMonitor();

monitor.on('download:start', (dl) => console.log(`Started: ${dl.file}`));
monitor.on('download:progress', (dl) => console.log(`${dl.progress.toFixed(1)}%`));
monitor.on('download:complete', (dl) => console.log(`Done in ${dl.duration}ms`));

monitor.startDownload('id', { file: 'test.txt', totalSize: 1000 });
monitor.updateProgress('id', 500);
monitor.completeDownload('id');

console.log(monitor.getStats());
// { activeCount, completedCount, totalBytes, peakConcurrent, ... }

ShutdownManager

Graceful shutdown with priority-ordered callbacks.

import { ShutdownManager } from '@kadi.build/file-sharing';

const sm = new ShutdownManager({ gracefulTimeout: 10000 });

sm.register(async () => console.log('Close DB'), 1);     // priority 1 = first
sm.register(async () => console.log('Close HTTP'), 10);   // priority 10 = later

await sm.shutdown();

Sub-path Exports

// Main export — all classes and utilities
import { FileSharingServer, createQuickShare } from '@kadi.build/file-sharing';

// S3 server only
import { S3Server } from '@kadi.build/file-sharing/s3';

// HTTP server only
import { HttpServerProvider } from '@kadi.build/file-sharing/http';

// Re-exports from dependencies (convenience)
import { FileManager, createFileManager, TunnelManager } from '@kadi.build/file-sharing';

Tunnel Services

KĀDI is the default and recommended tunnel service. It supports two connection modes:

| Mode | How It Works | Requirements | |------|-------------|--------------| | SSH (default) | ssh -R through KĀDI's SSH gateway — zero-dependency | SSH client in $PATH | | frpc | Uses the frp client binary for higher reliability | frpc binary in $PATH | | auto | Prefers frpc if available, falls back to SSH | — |

Set KADI_TUNNEL_MODE=ssh (or frpc) in your .env to force a specific mode.

Transport Protocol

When using frpc mode, the control channel transport can be configured:

| Transport | Description | Default | |-----------|-------------|---------| | wss | Routes the frpc control channel through a WSS gateway on port 443. Works reliably on enterprise/campus networks that block non-standard ports. | ✅ Default | | tcp | Direct TCP connection to the frps server port (typically 7000). | — |

Set KADI_TUNNEL_TRANSPORT=wss and KADI_TUNNEL_WSS_HOST=tunnel-control.kadi.build in your .env to use WSS transport. TCP mode can be forced with KADI_TUNNEL_TRANSPORT=tcp.

If KĀDI credentials are not provided, the tunnel will automatically fall back to free services (serveo → localtunnel → pinggy → localhost.run) unless autoFallback: false is set.

Container Usage — Installing frpc

When running inside a Docker container (e.g. via kadi deploy), the KĀDI tunnel requires the frpc binary to be available at build time. Alpine-based images (like node:22-alpine) do not include it by default.

Add the following lines to the run array in your agent.json build config:

"run": [
  "apk add --no-cache curl openssh-client",
  "FRPC_VERSION=0.61.1 && wget -qO /tmp/frpc.tar.gz https://github.com/fatedier/frp/releases/download/v${FRPC_VERSION}/frp_${FRPC_VERSION}_linux_amd64.tar.gz && tar -xzf /tmp/frpc.tar.gz -C /tmp && mv /tmp/frp_${FRPC_VERSION}_linux_amd64/frpc /usr/local/bin/frpc && chmod +x /usr/local/bin/frpc && rm -rf /tmp/frpc.tar.gz /tmp/frp_${FRPC_VERSION}_linux_amd64"
]

What these lines do:

1. apk add --no-cache curl openssh-client — Installs curl and the SSH client (needed for SSH-mode fallback).
2. FRPC_VERSION=0.61.1 && wget … — Downloads the frpc binary (v0.61.1) from the official fatedier/frp releases, extracts it to /usr/local/bin/frpc, and cleans up temp files.

Note: For non-Alpine images (Debian/Ubuntu-based), replace apk add with apt-get update && apt-get install -y curl openssh-client.

See arcadedb-ability/agent.json and backup-ability/agent.json for real-world examples.

Troubleshooting Tunnels

If your deployed container fails to establish a tunnel, the most common cause is a missing frpc binary. Symptoms include:

• Tunnel creation silently fails or times out
• Logs show frpc: not found or the service falls back to SSH mode unexpectedly
• The agent connects to the broker but is not reachable via its tunnel URL

Fix: Ensure the frpc installation lines above are in your agent.json build.*.run array, then rebuild:

kadi build

Verify frpc is installed inside the container:

frpc --version
# Expected: frpc version 0.61.1

| Symptom | Likely Cause | Fix | |---------|-------------|-----| | frpc: not found | frpc not installed in the container image | Add the run lines above and rebuild | | Tunnel times out | Network/firewall blocking port 7000 | Set KADI_TUNNEL_TRANSPORT=wss to use port 443 | | SSH fallback instead of frpc | frpc binary missing from $PATH | Install frpc (see above) | | Permission denied on frpc | Binary not executable | Ensure chmod +x /usr/local/bin/frpc is in your build |

Dependencies

| Package | Purpose | |---------|---------| | @kadi.build/file-manager | File management utilities | | @kadi.build/tunnel-services | Multi-provider tunnel management | | express | HTTP framework | | cors | CORS middleware | | mime-types | MIME type detection | | xml2js | XML generation for S3 responses | | chalk | Console styling |

Requirements

• Node.js ≥ 18.0.0

License

MIT