auditfile-js

Streaming reader for Dutch tax audit files (XAF and related), following auditfile-spec. The JavaScript counterpart of auditfile-dotnet, with the same API in JavaScript style.

Status: draft (0.x). Not affiliated with the Belastingdienst, SRA or any software vendor.

npm install @xml-auditfile/auditfile

The package ships TypeScript declarations, generated from the JSDoc. Every version is published from GitHub Actions with provenance: npm audit signatures verifies that it was built from this repository.

import { createReadStream } from 'node:fs';
import { Auditfile, AuditfileSource, MemorySink } from '@xml-auditfile/auditfile';

const sink = new MemorySink();
const source = AuditfileSource.fromFactory(() => createReadStream('administratie.xaf'), 'administratie.xaf');
const report = await Auditfile.read(source, sink);

for (const diagnostic of report.diagnostics) {
    console.log(String(diagnostic));
}

for (const line of sink.rows('tr_line')) {
    console.log(`${line.getString('acc_id')} ${line.getDecimal('amnt')} ${line.getString('amnt_tp')}`);
}

Rows have typed accessors (getString, getDecimal, getInteger, getDate, getDateTime, getTime, getBoolean) that check the column type and name the problem; has() tells whether a column exists in this version. getDecimal also reads an integer column, since a column can be integer in one version and decimal in another. In a loop over many rows, look the column up once (table.findColumn('amnt')) and pass it instead of the name.

Values are read in XSD notation only: a date-time such as 03/01/2017 10:00:00 is refused (XAF-S004) rather than guessed.

What it does

• Determines the version: forced in code, official namespace or alias from the registry, optionally derived from the namespace (useResolver). Anything else stops with a clear diagnostic.
• Reads in two phases: scan (validate against the built-in XSD, count rows per table) and load (stream rows in batches to a sink). Verified mode loads only a valid file.
• Turns every repeating element into a table following the manifest generated by auditfile-spec. Rows know their id, parent, position and source part; values are typed.
• Checks required and expected tables, keys, references and the XAF business rules (balanced transactions, header counts and totals).
• Reads split audit files (audit.xaf, audit-2.xaf, ...) as one file; you pass the parts in order.
• Reports everything as diagnostics with stable codes, in Dutch or English.

Entry points

| Call | Use | | ---- | --- | | await Auditfile.read(source, sink, options) | Read into a sink: MemorySink or your own sink | | await Auditfile.validate(source, options) | Validate without storing anything; mode: 'unverified' skips the XSD | | await Auditfile.scan(source, options) | Version, XSD validation and row counts only | | (await Auditfile.open(source, options)).readRows() | Stream rows yourself (for await); the rows can be iterated once | | AuditfileVersion.Financial.V3_2 | Manifest, XSD, definitions, dialects and views of a version | | Auditfile.dialects.add(version, json) / Auditfile.views.add(json) | Add your own dialects and views next to the built-in ones (see below) | | schema.toJson() / AuditfileSchema.fromJson() | Store and restore the schema of an import |

Sources: the reader never opens files; you supply the streams. A stream is a Node Readable, a web ReadableStream, a Blob, a Uint8Array/ArrayBuffer, or any (async) iterable of Uint8Array chunks.

| Call | Use | | ---- | --- | | AuditfileSource.fromFactory(open, name) | Fresh stream per call: can be read twice (scan, then load) | | AuditfileSource.fromFactories([name, open], ...) | Same, for the parts of a split audit file | | AuditfileSource.fromStream(stream, name) | Read once; validation then happens during loading. The stream is not destroyed | | AuditfileSource.fromStreams([name, stream], ...) | Same, for the parts of a split audit file |

Options are a plain object: version (an AuditfileVersion or "xaf/3.2"), mode ('verified' or 'unverified'), useResolver, xsd, applyDialects, batchSize, language ('nl' or 'en'), maxDiagnostics, maxDepth, maxCharactersInDocument, maxValueLength, onProgress and signal (an AbortSignal, checked in the scan phase as well as the load phase). Unknown names and values out of range are refused. When a sink fails and its abort fails too, the read rejects with an AggregateError holding both errors.

A sink is any object with begin(context), write(table, rows), complete(report) and abort(report); each may return a promise. Set capabilities = SinkCapabilities.NeedsRowCounts to get row counts before the first row.

Custom dialects and views

Dialects and views come from auditfile-spec and are built into the package. You can add your own at runtime, in the same format as dialects.json and views/<type>.json of the spec, without a fork or a custom build. Pass JSON text or the parsed object; the reader opens no files, so read them yourself.

import { readFileSync } from 'node:fs';
import { Auditfile, AuditfileVersion } from '@xml-auditfile/auditfile';

// { "formatVersion": 1, "dialects": [ { "name": "my-package", "match": { ... }, "corrections": [ ... ] } ] }
Auditfile.dialects.add(AuditfileVersion.Financial.V3_2, readFileSync('dialects.json', 'utf8'));   // or 'xaf/3.2'

// { "formatVersion": 1, "type": "xaf", "views": [ { "name": "lines_per_account", ... } ] }
Auditfile.views.add(readFileSync('views.json', 'utf8'));

const report = await Auditfile.read(source, sink);           // recognizes and applies the added dialect
const result = sink.run(sink.schema.version.findView('lines_per_account'));

• Added definitions come after the built-in ones in version.dialects and version.views, so findView, version resolution and dialect corrections use them like the built-in ones. A view is added for every version of its type; whether it fits a version is decided when it runs (XAF-Q001), as for built-in views. applyDialects: false ignores added dialects too.
• A file is checked when it is added and refused as a whole when anything is wrong: invalid JSON (SyntaxError), a formatVersion other than 1, an unknown audit file type or version, an unknown dialect action, view aggregate or operator, a correction code that is not a dialect diagnostic (XAF-C001–C003), or a name the version or type already has (RangeError). Built-in definitions cannot be replaced or removed.
• Auditfile.dialects.remove(version, name) and Auditfile.views.remove(type, name) remove one added definition; clear() removes all of them. The definitions are shared by every read in the process; a read that has started keeps the dialects it found.

Values

| Column type | JavaScript value | | ----------- | ---------------- | | string | string | | decimal | Decimal: exact, with the scale as written ('100.00'); never a floating-point number | | integer | number; values outside the safe integer range are reported (XAF-S004), never rounded | | date | Date at midnight UTC; a time zone in the file is ignored | | datetime | Date; a value without time zone is taken as UTC | | time | string "HH:MM:SS" | | boolean | boolean |

Security

The reader is built for files from untrusted sources:

• DTDs are refused, so there is no entity expansion and nothing is fetched (XXE, billion laughs).
• XSD imports resolve only to the definitions embedded in the package; never to disk or network.
• Nesting deeper than maxDepth (64) stops reading; the root element must appear within the first megabyte.
• Memory grows with the number of keys (accounts, customers and suppliers, ...), not with the number of lines: only open elements, key values, unresolved references (at most 100,000 per reference, XAF-R017 beyond that) and the totals of transactions being read are kept, and the diagnostics list is capped at maxDiagnostics.
• Text from the file is never interpreted: messages are filled without replacement patterns, and lookups use Map, not plain objects.
• A stream you pass with fromStream is never destroyed; streams from a factory are closed when reading stops.

Diagnostics

Every finding is a diagnostic with a stable code from diagnostics.json of auditfile-spec. The report lists at most maxDiagnostics (default 1000), followed by XAF-X900 when more were found. A full list makes room for errors by leaving out infos and warnings, and fatal diagnostics are always included. isValid, hasErrors, hasFatal and count(severity) count every diagnostic, also the ones left out; isTruncated says whether some were.

| Layer | Codes this reader reports | | ----- | ------------------------- | | XML | X001–X004 (X004 also for a part of a split file in another encoding than the first part), X900 | | Version | V001–V006 | | Structure | S001–S005, S008; S006 in unverified mode (in verified mode the XSD reports a missing field as D001) | | XSD | D001, D002 | | Rules | R001–R004, R017 | | Dialect | C001–C003, when a dialect applies | | Views | Q001, Q002 |

Defined in the spec but not checked yet: S007 and S009 (in verified mode the XSD reports too many occurrences and values outside a code list as D001), R005–R016 (ascending numbers, periods, eleven test, IBAN, personal data), X005 (the XML parser reports an invalid character as X001) and E001–E004 (envelopes).

XSD validation

Validation uses a streaming validator for the XSD subset that auditfile-spec uses: global and local elements, sequences with occurrence bounds, attributes, simple types with facets (enumeration, pattern, length, digits, ranges, white space) and key/keyref/unique. Every XSD of the registry compiles; a custom XSD (xsd option) that uses anything else (such as xs:choice or xs:any) is refused when it is compiled, never validated halfway. Like the .NET reader, identity constraints follow XPath 1.0: an unprefixed step selects elements without namespace.

Platforms

Node 20 or later. The library itself uses no file system and no Node built-ins (SHA-256 uses Node's implementation when present, its own otherwise), so it also runs in the browser through a bundler. Its one dependency is saxes, a strict, well-formedness-checking XML parser.

Layout

| Path | Contents | | ---- | -------- | | src/ | The reader; src/memory/ holds MemorySink (also exported as @xml-auditfile/auditfile/memory) | | src/xsd/ | The streaming XSD validator | | test/ | Tests, using the XAF 3.2 sample in test/data/xaf-3.2-sample.xaf (two intended findings: XAF-R001, XAF-R003) | | spec.lock.json | The pinned auditfile-spec release | | tools/ | fetch-spec.ps1 (copies generated/ of the pinned spec into spec/generated/) , embed-spec.mjs and generate-constants.mjs (version constants from the registry) |

Example programs live in auditfile-js-samples.

Definitions

Everything in generated/ of a pinned auditfile-spec release (XSD, manifests, definitions, dialects, registry, diagnostics, views) is embedded in the package as src/generated/spec.js. The spec is not committed: spec.lock.json pins a commit, tools/fetch-spec.ps1 copies only generated/ of that commit into spec/generated/ (git-ignored; a plain folder, not a git checkout), and npm test / npm pack embed it.

pwsh tools/fetch-spec.ps1        # after cloning and whenever spec.lock.json changes
npm install
npm test

To use a local spec checkout instead: SPEC_ROOT=../auditfile-spec npm test.

The version constants (AuditfileVersion.Financial.V3_2, ...) are written from the registry of the pinned spec by node tools/generate-constants.mjs; run it after spec.lock.json changes. CI checks that they match. spec-update.yml does both when auditfile-spec has a new release, and opens a pull request.

Workflows

| Workflow | What it does | | -------- | ------------ | | ci.yml | Push to main and pull requests: fetch the pinned spec, install, test on Node 20, 22 and 24, pack (dry run), audit dependencies | | release.yml | Manual, with a version such as 1.0.1. From main the version must be higher than every release; from release/X.Y (maintenance of an older line) it must be in line X.Y and higher than the latest release in that line. Then it fetches the pinned spec, tests, packs, creates the tag and GitHub release, and stages the version on npm through Trusted Publishing when the NPM_PUBLISH variable is true (environment npm). A staged version goes live only after a maintainer approves it with 2FA on npmjs.com (Staged Packages) or with npm stage approve <stage-id>. Dist-tags: latest for the highest stable release, next for a pre-release, release-X.Y for maintenance | | dependabot.yml | Weekly updates of npm packages and GitHub Actions |

License

Apache License 2.0.