@sema-agent/primitives

Low-level definitions of the sema engine that every layer may import: no policy of any layer, no state, no dependencies. It holds two members: the credential reference, and the secret-shape detection definition (the one table of credential shapes, its matcher and its renderer). The engine depends on this package and reads it by name, so in a project that installs both there is one copy: an acceptance gate, a settings validator or an admin form reads the same closed set through the same function object as the engine does. Runs in Node and in browsers; ships its own types.

Install

npm install @sema-agent/primitives

ES modules only ("type": "module"), Node >= 20.18.1 where Node is used. One entry, .; there are no deep paths.

Version

@sema-agent/[email protected] is cut with @sema-agent/[email protected]: the same version number, built from the same commit. A package version moves with every core release whether or not the primitives changed.

@sema-agent/[email protected] depends on @sema-agent/[email protected] exactly. A project that also imports this package names the same version as its core (or a range npm can resolve to it); then npm ls @sema-agent/primitives shows one copy. Naming another version installs two — npm keeps core's own beside it — and the two answer for different versions.

The credential reference

A declaration names a credential by reference instead of carrying its value. The reference is one of a closed set of kinds; whoever resolves it (a deployment's seat, not this package) decides what the name means and whether this declaration may use it.

| kind | Members | Request surface (CREDENTIAL_REF_TRUST[kind].request) | | --- | --- | --- | | env | headers?: header name → variable name · env?: child variable → variable name (at least one) | refused | | vault | entry: a stored credential entry | own_entry — only an entry that belongs to the submitting principal | | handle | id: an opaque id the deployment minted | refused | | assertion | audience: what a short-lived assertion is requested for | refused |

readCredentialRef(value) answers { ok: true, ref } with a fresh frozen copy (forward the copy, never the value you read), or { ok: false, problem }. Each fault is its own reason, and the first one found is the one reported, in this order: not a record; kind absent, not text or unknown; an extra member; a missing member; a bad value. The reader does not throw: a value whose members cannot be read (a revoked or throwing Proxy) is not_a_record (or, inside a name map, that map's bad_value). With no stack left a valid value may be refused, under any reason, and never an invalid one accepted; the call itself may throw a RangeError before it is entered, which no function can catch.

| reason | Meaning | | --- | --- | | not_a_record | not an ordinary record (a primitive, null, an array, a class instance, a Map, a Date), or one whose members cannot be read; an object whose prototype is none or itself has none — the Object.prototype of any realm is one — is a record | | unknown_kind | kind is text outside CREDENTIAL_REF_KINDS (every spelling the set does not hold: another case, padded, zero-width, full-width, a look-alike letter) | | extra_field | a member the kind does not have (a member spelled in another case is one) | | missing_field | kind, or a member the kind needs, is absent (for env: neither headers nor env, field headers\|env) | | bad_value | a member of the wrong form (below) |

A member is an own enumerable string-keyed data property. A getter is refused and never called, nothing is read through a prototype, a key present with the value undefined is a bad value (not an absent member), and a non-enumerable or symbol-keyed member is not read and not copied.

bad_value is: for kind, anything but text; for a text member (entry, id, audience), anything but 1 to CREDENTIAL_REF_TEXT_MAX (2048) UTF-16 code units with no control, format or line-separator character (C0 and C1, zero-width, bidi override, soft hyphen, BOM, U+2028 / U+2029), no default-ignorable, noncharacter or private-use character, no lone surrogate and no leading or trailing white space; for a name map (headers, env), anything but a record of 1 to CREDENTIAL_REF_MAP_MAX (64) entries whose names are within the grammar (a header name is an HTTP token; a child variable name and every variable name is [A-Za-z_][A-Za-z0-9_]*) and within the text bound, and — for headers, whose names are case-insensitive — in which no two names differ only by case. The maps come back frozen and without a prototype: an absent name reads undefined, and an own __proto__ name is a plain key. Text is compared as written; nothing is normalised (two spellings of one visible string are two entries; visually confusable text — homoglyphs, a braille blank, stacked combining marks — is not judged, and an emoji presentation selector or a zero-width joiner is refused, because these members are identifiers, not prose). The text bound counts UTF-16 code units (up to 6144 UTF-8 bytes), and the character classes are the host's Unicode tables, so two surfaces on engines with different Unicode versions can differ for a recently added character.

import { readCredentialRef } from "@sema-agent/primitives";

readCredentialRef({ kind: "vault", entry: "github-main" }); // { ok: true, ref: { kind: "vault", entry: "github-main" } }
readCredentialRef({ kind: "vault2", entry: "x" });          // { ok: false, problem: { reason: "unknown_kind", kind: "vault2" } }
readCredentialRef({ kind: "handle" });                      // { ok: false, problem: { reason: "missing_field", kind: "handle", field: "id" } }
readCredentialRef({ kind: "handle", id: "h1", note: "x" }); // { ok: false, problem: { reason: "extra_field", kind: "handle", field: "note" } }

A kind or field the reader echoes back is the caller's text, cut to 64 characters with control, format and line-separator characters and lone surrogates replaced; fence it before rendering it where markup is interpreted. A Proxy's own traps (on the value, its maps or its prototype) are the caller's code and run as they would for any property read.

The reader judges shape only: it does not know whether a variable exists, whether an entry belongs to anyone, which header names a deployment forbids (the protocol's own names, a principal header), which child variables a declaration may set, or whether a reference may reach a destination. Those are the policy of the layer that reads the reference.

CREDENTIAL_REF_TRUST has a row for each kind and for nothing else (it has no prototype, so no inherited member answers); look a kind up in it only after readCredentialRef has accepted the value.

Secret-shape detection

CREDENTIAL_VOCABULARY is the one table of what a credential looks like: value families (each row with a wide form, and for some a strict form, a fragment form — what is left of the credential when a text was cut inside its body — and the form and token head an environment value is read by), the credential words a name is read by, the names that look like credentials but are none, and names: the names that mark a credential in an assignment (places: ["assignment"]) or as a URL query or fragment parameter ("url-param"). Every layer that looks for a secret reads this table, so a shape added to it is found everywhere at once. The table holds no layer's policy: which form is read, how a text is read first and how a URL's username is treated are the caller's, passed in whole on every call. Nothing here has a default. Each value family row names the mark its hit is written as (mark, below).

interface SecretScanPolicy {
  grade: "strict" | "wide" | "fragment";         // the strict forms (narrow: a hit may refuse a write), the wide ones (broad: a hit is withheld) or the fragment ones (a cut residue)
  readings: SecretReading[];                     // at least one of { kind: "raw" } (as written) / { kind: "format-free" } (format characters removed),
                                                 // and any of { kind: "percent-layers", depth: 1..8 } / { kind: "percent-boundary" } / { kind: "form-plus" }
  urlUserinfo: "keep-username" | "whole";        // a URL userinfo: keep the user and withhold the password, or withhold all of it
  onUnresolvedReading: "secret" | "ignore";      // what a reading that cannot read a text to its end answers
}

| Function | Answers | | --- | --- | | scanSecrets(text, policy) | { outcome: "spans", spans } — every hit as { start, end, family, kind } in text's own coordinates, sorted (empty = none found) — or { outcome: "whole", reason } with reason one of scan-budget (more than 64 places where format characters stood), reading-unresolved (the percent layers ran out, under onUnresolvedReading: "secret"), mapping-failed; a hit in a derived text is reported over the written units it was read from | | isSecretShaped(text, policy) | scanSecrets found a span or answered the whole text | | userinfoHolds(userinfo) | a URL userinfo (no trailing @) holds a user or a password — an empty one and the bare separator : hold neither (the reading the keep-username rendering withholds by; a mark is one token, its : splits nothing) | | renderSecrets(text, policy, { passes?, report? }?) | text with every hit replaced by its row's mark (the fragment grade's by that mark's :partial member; a text no reading can afford, whole, by «redacted:oversized»); passes are the caller's own rules, run after the table's in the same pipeline, each marker a member of REDACTION_MARKS; report.findings receives one finding per hit (report.preexistingMarkers is the caller's to set) | | renderWithheld(text, spans) | text with each { start, end, mark } span withheld by the same law renderSecrets writes its own hits by — for a layer that finds its regions itself | | isSensitiveRecordKey(key) / isSensitiveHeaderName(name) | a record member / an HTTP header name names a credential | | envNameView() | the readings of environment variable names (exact names, credential words, a namespace's names anySegmentUnder(namespace)) | | envValueView() | the readings of environment values, off the rows: forms (each row's env value form, in table order), tokenHead (each row's token head, one anchored alternation), flagWords | | secretReadingViews(text, readings) | the texts the readings derive — the written text first, each once — each with toOriginalSpan(start, end) (the whole written units a span was read from), and unresolved (the percent layers ran out while one more would still decode) |

A policy that is not whole (a missing or unknown member), a caller's pass whose marker is no member of the marks table, or whose string replace writes anything but that member and $n references, throws a TypeError; so does a renderWithheld span that is not a range of the text or whose mark is no member. The policy is read on every call; what it reads as picks a compilation kept across calls. scanSecrets answers where its forms match, a mark the text holds included (password=«redacted:key» is a span); renderSecrets leaves a hit whose withheld part is exactly a mark the text already holds (below). URL_OF_ANY_SCHEME is the URL grammar (a global regex: read it with replace / matchAll), for a layer that withholds whole URLs. CREDENTIAL_MIN_CHARS is the fewest characters a string can have and still be read as a credential.

Redaction marks

REDACTION_MARKS is the one table of what stands in a text where a layer withheld part of it — fifteen members, frozen:

| Name | Member | Written for | | --- | --- | --- | | bare | «redacted» | a value whose place in the text already says what it was: a credential name's value, an assignment, a header value, a scheme word's credential (Authorization: «redacted»), the password after a kept username (https://alice:«redacted»@host), a URL query or fragment value | | key | «redacted:key» | a credential read by its shape alone (sk-…, AIza…, a JWT, …) | | gh-token / aws-key / slack-token / private-key | «redacted:gh-token» … | a credential read by its shape alone, of that vendor or kind | | <kind>:partial | «redacted:key:partial» … | the same five kinds, found by the fragment grade (a cut residue) | | userinfo | «redacted:userinfo» | a URL userinfo withheld whole | | url / path | «redacted:url» / «redacted:path» | a whole URL / an absolute path, withheld by a layer that hides where things are | | oversized | «redacted:oversized» | a stretch withheld because it could not be read |

isRedactionMark(s), redactionMarkerSpans(s) and eachRedactionMarkerSpan(s) (lazy) are the one reader: they recognize the members exactly — «redacted:evil» and square-bracket spellings ([redacted], [redacted-url]) are ordinary text. A mark is one token to the renderer: a hit whose withheld part is a mark the text already holds is no hit (password=«redacted:key» stays as written), a hit reaching into or inside a mark takes the whole mark, and hits that overlap or touch write one mark — the widest hit's (of two over one region, the later pass's). Rendering a rendered text again changes nothing.

Every count in the table follows one character or one character class and is closed — "n or more" is written as n of them and then a plain loop, and a group's count is written out — so a reading keeps no regex backtrack state per character of one value run, and a run of millions of characters is scanned and rendered to its end (pinned at 8 million code units) instead of throwing RangeError. A credential header value made of millions of = / , / quoted-parameter pieces is still read one group pass per piece and throws from about two million code units on.

The table and its rows are frozen; its regexes are not (replace writes a global regex's lastIndex), and the functions here never run them: the package runs its own copies, made as it loads, so a regex a caller recompiles or moves on the exported table changes only that caller's reading.

import { renderSecrets, scanSecrets } from "@sema-agent/primitives";

const policy = Object.freeze({ grade: "wide", readings: Object.freeze([Object.freeze({ kind: "raw" })]), urlUserinfo: "keep-username", onUnresolvedReading: "secret" });
renderSecrets("clone https://me:[email protected]/r", policy);           // "clone https://me:«redacted»@git.example/r"
scanSecrets("clone https://me:[email protected]/r", policy);             // { outcome: "spans", spans: [{ start: 17, end: 25, family: "url-userinfo", kind: "url-credentials" }] }

Detection is best effort over known shapes: it never proves a text holds no credential.

Exports

Each name below is a runtime value or a type exported from the package root. The root's declarations are the contract; a removed or re-kinded export is named as breaking in the @sema-agent/core release notes.

| Group | Names | | --- | --- | | Credential reference | readCredentialRef CREDENTIAL_REF_KINDS CREDENTIAL_REF_TRUST CREDENTIAL_REF_TEXT_MAX CREDENTIAL_REF_MAP_MAX CREDENTIAL_REF_PROBLEM_REASONS | | Types | CredentialRef CredentialRefKind CredentialRefRequestAdmission CredentialRefProblem CredentialRefProblemReason CredentialRefReading | | Secret-shape detection | CREDENTIAL_VOCABULARY CREDENTIAL_MIN_CHARS URL_OF_ANY_SCHEME SECRET_READING_KINDS scanSecrets isSecretShaped renderSecrets renderWithheld isSensitiveRecordKey isSensitiveHeaderName envNameView envValueView secretReadingViews splitCamelCase | | Redaction marks | REDACTION_MARKS isRedactionMark redactionMarkerSpans eachRedactionMarkerSpan | | Types | CredentialVocabulary ValueFamilyRow CredentialWordRow CredentialWordPosition CredentialNameRow CredentialNamePlace EnvNameView EnvValueView RecordKeyView RedactionConfidence RedactionSource SecretGrade SecretReading SecretReadingKind SecretReadingView SecretReadingViews SecretScanPolicy SecretScan SecretSpan RenderOptions RedactionPass RedactionFinding RedactionReport SummarizableFinding WithheldSpan RedactionMark RedactionMarkName |