@~lyre/file

File/asset layer for Drizzle + Postgres (Node/TypeScript). Pluggable storage drivers (local / S3 / remote), md5-dedup uploads, responsive image variants, remote-URL ingestion, polymorphic attachments, and a turnkey stream + upload route provider.

TypeScript port of the Laravel lyre/file package. (That PHP package lives at packages/file in this repo; this is the TS sibling.)

Install

npm install @~lyre/file drizzle-orm

drizzle-orm is the only required peer. Two optional extras unlock features when present, and are never imported otherwise:

| Package | Enables | | --- | --- | | sharp | Image dimensions + sm/md/lg responsive variants | | @aws-sdk/client-s3 | The s3 storage driver | | @~lyre/model | The ./models entry point (File CRUD model) |

No assumptions about your schema

files.tenant_id, files.owner_user_id and attachments.attachable_id are plain uuids with no foreign keys by default, so the package imports none of your tables. It works in an app that has no tenants/users at all — one keyed by app slug, workspace, or anything else. Ownership is enforced by your application.

Register before the schema is imported to change either the column type or the table each column references:

// db/schema/file-owners.ts — its own module, so it is evaluated first
import { registerFileOwners } from '@~lyre/file/owners';
import { tenants } from './tenants';
import { users } from './users';

// An app with uuid keys: add real foreign keys.
registerFileOwners({ tenant: { column: tenants.id }, owner: { column: users.id } });

// An app whose ids are NOT uuids — e.g. tenant ids like '82', and owner rows
// spread across tables with different key types. Postgres rejects '82'::uuid,
// so the type must be declared.
registerFileOwners({ tenant: { type: 'text' }, attachable: { type: 'text' } });

// db/schema/files.ts
import './file-owners.js';        // MUST precede the export below
export * from '@~lyre/file/schema';

The separate module matters. ESM hoists every import above the module body, so a registerFileOwners(...) call sitting next to export * from '.../schema' would run after the table was already built.

attachable_id is polymorphic and never carries a foreign key, so only its type is configurable. When one attachable table has a bigserial id and another has a text slug, 'text' is the only type that spans both.

File lifecycle

A row created with referenceOnly points at someone else's host, so it can rot without anything here changing. files.status (active | unreachable | unlinked) plus last_checked_at make that an indexed column rather than a guess:

import { refreshRemoteFile, markFileViewed } from '@~lyre/file';

await refreshRemoteFile(db, files, fileId);   // probes the URL, records the result
await markFileViewed(db, files, fileId);      // stamps viewed_at when served

refreshRemoteFile HEADs the URL and falls back to a ranged GET (some CDNs answer HEAD with 405 while serving GET fine). Network failures record unreachable rather than throwing, so a sweep over many rows completes. unlinked is never inferred — set it deliberately when you know the original is gone.

Resolving file references

resolveFileTokens walks any object and substitutes <prefix>:<id> strings via a lookup you supply. Use it when a producer should be handed opaque ids instead of URLs — the ids are stable, and a URL that changes (or was never meant to be public) is resolved at the boundary rather than copied around:

import { resolveFileTokens } from '@~lyre/file';

const urls = new Map([['7f3a', 'https://cdn.example.com/a.png']]);
resolveFileTokens(payload, { prefix: 'media', lookup: (id) => urls.get(id) });
// { attachments: ['media:7f3a', 'media:ghost'] } → { attachments: ['https://…/a.png'] }

An id the lookup cannot resolve is removed — dropped from arrays, nulled on object properties — so a dangling reference never survives. The prefix and lookup are yours; this knows nothing about what produced the value.

Storage

import { configureStorage, S3StorageDriver } from '@~lyre/file';

configureStorage(new S3StorageDriver({ bucket: 'assets', region: 'eu-west-1' }));

Or via env: STORAGE_DRIVER=local|s3|remote, plus STORAGE_LOCAL_ROOT or S3_BUCKET / S3_REGION / S3_ENDPOINT / S3_FORCE_PATH_STYLE and standard AWS credentials. S3_ENDPOINT + S3_FORCE_PATH_STYLE cover R2, MinIO and Spaces.

An unknown or unconfigured driver throws. It does not fall back to another disk. An earlier version silently wrote to local disk when S3 was requested, while the file row still recorded the requested backend — rows that pointed at objects which were never uploaded. A misconfiguration should fail at startup.

Uploading

import { uploadFile } from '@~lyre/file';
import { files } from './db/schema';

const record = await uploadFile(db, files, {
  tenantId,
  bytes,
  filename: 'logo.png',
  mimeType: 'image/png',
  publicBase: '/api/v1/files'   // where you mounted the stream route
});

Uploads are md5-deduplicated per owner: re-uploading identical bytes bumps usage_count and returns the existing row instead of storing again.

Attachments

One file can hang off many rows, and a row can hold an ordered list of files — without either side knowing the other's schema.

import { attach, listAttachments, detach } from '@~lyre/file';

await attach(db, { attachments, files }, {
  attachableType: 'prompt_block',   // any app-defined discriminator
  attachableId: blockId,
  fileId: record.id,
  order: 0
});

const assets = await listAttachments(db, { attachments, files }, {
  attachableType: 'prompt_block',
  attachableId: blockId
});

attach is idempotent — re-attaching the same file updates its position rather than duplicating.

Route provider

filesProvider contributes GET <mount>/[file] (stream) and POST <mount> (upload) to a @~lyre/model platform hook. It ships from its own subpath — @~lyre/file/provider, not the root entry — so mounting HTTP routes is always a deliberate opt-in.

Storage and both authorization gates are injected; that DI is what keeps the package portable, so they cannot be defaulted:

import { filesProvider } from '@~lyre/file/provider';

filesProvider({
  // Reads are authorized FIRST, and the grant scopes the lookup.
  authorizeRead: async (event, { slug }) => {
    const session = await sessionFor(event);
    if (!session) return null;                       // 403
    return { tenantId: session.tenantId, viewerId: session.userId };
  },
  // MUST filter on grant.tenantId — a slug is guessable, not a capability.
  getFile: (slug, grant) => getFileForStream(slug, grant.tenantId),
  recordView: (fileId, viewerId) => files.recordView(fileId, viewerId),
  upload: (input) => uploadFileToStorage(input),
  authorizeUpload: async (event, fields) => {
    if (!isAdmin(event)) return null;                // 403
    return { tenantId: fields.tenantId!, ownerUserId: currentUserId(event) };
  },
  ownerField: 'tenantId'   // multipart field carrying the owner id
});

Slugs derive from the filename (logo.png → logo, with a random suffix only on collision), so they are guessable and are not a capability. authorizeRead is where access is decided, and getFile must constrain the query to the granted scope — returning a file outside it is a cross-tenant leak the package cannot detect.

Streamed bytes are sent private, max-age=3600 with X-Content-Type-Options: nosniff and an explicit Content-Disposition. image/svg+xml is not in the default mime allowlist: an SVG served inline from your own origin is a stored-XSS vector. Opt in via allowedMime only if you serve it from a separate origin.

Media tokens

Agents and stored content reference files as opaque media:<id> tokens, never URLs — a moved asset is then a data fix, not a content rewrite. Resolve them at the last moment:

const reply = await repo.resolveMediaUrls(modelOutput, {
  prefix: 'media',
  dropObjectsMissing: ['url']   // prune attachments whose url didn't resolve
});

One batched findManyByIds per call, not a query per token. Only active rows with a url resolve — unlinked (source dropped the original) and unreachable (last probe failed) are withheld, and anything unresolved is dropped rather than shipped as a dangling media:abc string.

dropObjectsMissing exists because an unresolved token is removed from an array but nulled on an object property — correct in general, since the walker cannot know which properties are load-bearing, but wrong for an attachment list where {url: null, type: 'image'} is malformed. Pass the property names an object cannot survive without.

collectFileTokenIds(value, prefix) is available separately when you need the ids without substituting.

Fetching remote URLs

Remote ingestion and the refresh sweep validate every URL through assertPublicHttpUrl / safeFetch: http(s) only, no embedded credentials, no private / loopback / link-local / metadata addresses, and every redirect hop re-checked. Validation happens at store time as well as fetch time, because a reference-only ingestion records a URL that the background sweep fetches later.

Both are exported for hosts that fetch media themselves. Residual risk: DNS rebinding is not closed — pass allowedHosts where the host set is known.

Notes

Node-only — it uses node:fs, and both sharp and the AWS SDK are loaded lazily at runtime.