@molecule/api-content-moderation-openai

Auto-generated, AI-first package reference for the molecule.dev ecosystem. It is written to be read by coding agents as much as by people, and is generated from this package's source — edit src/index.ts JSDoc, not this file.

OpenAI content classifier for @molecule/api-content-moderation.

Scores text and images with OpenAI's moderation endpoint (omni-moderation-latest), which OpenAI does not charge for.

Quick Start

import { setClassifier, requireClassifier } from '@molecule/api-content-moderation'
import { classifier } from '@molecule/api-content-moderation-openai'

setClassifier(classifier) // reads OPENAI_API_KEY on first use

const verdict = await requireClassifier().check(post.body)
if (verdict.flagged) {
  // Reject, or hold for review, BEFORE saving.
}
const image = await requireClassifier().checkImage!(bytes, { mimeType: 'image/png' })

Type

provider

Installation

npm install @molecule/api-content-moderation-openai @molecule/api-content-moderation @molecule/api-secrets

API

Interfaces

OpenaiModerationConfig

Options for {@link createClassifier}. Every field falls back to an env var or a default, so the zero-argument classifier export works from .env alone.

interface OpenaiModerationConfig {
  /** OpenAI API key. Defaults to `OPENAI_API_KEY`. */
  apiKey?: string
  /** API origin. Defaults to `OPENAI_BASE_URL`, then `https://api.openai.com`. */
  baseUrl?: string
  /** Moderation model. Default `omni-moderation-latest` (text + images). */
  model?: string
  /** Per-request timeout in milliseconds. Default 30000. */
  timeoutMs?: number
}

ProcessEnv

Environment variables this classifier reads.

interface ProcessEnv {
  OPENAI_API_KEY?: string
  OPENAI_BASE_URL?: string
}

Classes

OpenaiContentClassifier

Classifier backed by OpenAI's moderation endpoint.

OpenaiModerationError

Error thrown when the moderation endpoint refuses or fails.

Functions

createClassifier(config)

Create an OpenAI content classifier.

function createClassifier(config?: OpenaiModerationConfig): ContentClassifierProvider

• config — Options; each falls back to its env var or default.

Returns: A ContentClassifierProvider.

toModerationResult(entry, options)

Turn one moderation entry into the core's result, applying the caller's category filter and threshold.

• No threshold: a category is flagged exactly when OpenAI flagged it.
• With threshold: a category is flagged iff its score ≥ threshold.
• categories limits the returned categories AND the overall flagged.

function toModerationResult(
  entry: OpenaiModerationEntry,
  options?: ModerationOptions,
): ModerationResult

• entry — The endpoint's result for one input.
• options — Category filter and threshold.

Returns: The normalized result.

Constants

classifier

The classifier implementation (wire with setClassifier).

const classifier: ContentClassifierProvider

contentModerationOpenaiSecretDefinitions

Secret definitions required by the OpenAI moderation bond.

const contentModerationOpenaiSecretDefinitions: SecretDefinition[]

Core Interface

Implements @molecule/api-content-moderation interface.

Injection Notes

Requirements

Peer dependencies:

• @molecule/api-content-moderation >=1.1.0
• @molecule/api-secrets ^1.0.1

Environment Variables

• OPENAI_API_KEY (required) — OpenAI API key
  • Setup: Create a secret key on the OpenAI platform (API keys page).
  • Get it here: https://platform.openai.com/api-keys
  • Example: sk-proj-...

Runtime Dependencies

• @molecule/api-content-moderation

• @molecule/api-secrets

• Wire with setClassifier — NOT setProvider: this is a classifier (scores only), not the full ContentModerationProvider with user reports.

• Config: OPENAI_API_KEY (SERVER-side only); optional OPENAI_BASE_URL.

• Categories are OpenAI's names: harassment, harassment/threatening, hate, hate/threatening, illicit, illicit/violent, self-harm, self-harm/intent, self-harm/instructions, sexual, sexual/minors, violence, violence/graphic. Results are sorted highest score first.

• Without threshold the verdict is OpenAI's own. With threshold, a category is flagged iff its score ≥ threshold — lower it to be stricter. Pass categories to decide on a subset (e.g. only sexual/minors).

• Errors are OpenaiModerationError with status; nothing is retried. Decide per surface whether a failed check blocks the content or holds it for review — never publish silently on an exception.

• No OpenAI account? @molecule/api-content-moderation-molecule exposes the same classifier through molecule.dev, billed to the molecule project.

E2E Tests

Integration checklist — drive the real UI (live preview, no mocks), adapt each item to this app's actual screens/flows, and check every box off one by one. A box you can't check is an integration bug to fix — not a skip:

• [ ] Every surface that accepts user-generated content (post, comment, image upload, bio — whatever this app has) runs it through check() / checkImage() SERVER-SIDE in the create/update handler, before persisting. Confirm the moderation call is on the write path, not client-side or skipped.
• [ ] Assert BOTH directions with real samples: clearly-violating content returns flagged: true and the UI rejects it with a visible reason; benign content returns flagged: false and publishes normally. A moderator that flags everything or nothing is broken.
• [ ] The decision gates persistence: a blocked item is NOT stored and is absent when you view the feed as a second user. Tighten ModerationOptions.threshold and a borderline item flips allowed → blocked.
• [ ] The failure mode is deliberate: when the moderation/AI call errors the request does NOT 500 — content is either published + logged (fail-open) or held for review (fail-closed) per the surface's intent.
• [ ] If the app has reporting, the round-trip works: a user's report() creates a 'pending' Report, it appears in the moderator queue via getReports(), and resolveReport() (approve/reject/dismiss) visibly changes its status and clears it from the pending queue.
• [ ] Moderator surfaces are privileged: a non-admin can't reach getReports() / resolveReport() (403) and can't see other users' flagged or pending content.