posti

Finnish postal data loader — fetches Basic Address File (BAF), Postal Code File (PCF), and Postal Code Changes (POM) from Posti's open data services and loads them into your database.

Supports PostgreSQL, MySQL, MariaDB, and SQLite.

Requirements

• Node.js >= 22.12 (the maintained LTS lines: 22, 24 and 26; CI tests all three)
• One of PostgreSQL, MySQL, MariaDB or SQLite (the database driver is an optional dependency)

Installation

# As a CLI tool
pnpm add -g posti

# As a library
pnpm add posti

Quick Start

CLI

# Load into PostgreSQL
posti --dialect postgres --host localhost --port 5432 --database posti --user posti --password posti

# Load into MySQL
posti --dialect mysql --host localhost --port 3306 --database posti --user posti --password posti

# Load into SQLite (no server needed)
posti --dialect sqlite --database ./posti.db

# Force re-process even if files haven't changed
posti --force

# Only load specific tables
posti --dialect postgres --database posti --tables postalcodes,addresses

CLI Options

posti [command] [options]

Commands:
  run                   Fetch and load postal data (default)
  migrate               Run database migrations only

Options:
  --config <path>       Path to config file
  --force               Force re-processing even if files haven't changed
  --no-migrate          Skip automatic database migrations on startup
  --dialect <dialect>   Database dialect (postgres, mysql, mariadb, sqlite)
  --host <host>         Database host
  --port <port>         Database port
  --database <name>     Database name (or file path for SQLite)
  --user <user>         Database user
  --password <pass>     Database password
  --tables <list>       Comma-separated tables (addresses,postalcodes,postalcode_changes)
  --help                Show this help
  --version             Show version

Config File

Instead of CLI flags, you can use a config file. Create posti.config.js:

export default {
  dialect: "postgres",       // postgres, mysql, mariadb, sqlite
  host: "localhost",
  port: 5432,
  user: "posti",
  password: "posti",
  database: "posti",
  tablePrefix: "posti_",
  migrate: true,             // Auto-run migrations on startup (disable with false)

  process: {
    chunkSize: 1000,         // Rows per bulk insert
    deleteOnComplete: true,  // Remove temp files after processing
  },

  // Which tables to process (default: all three)
  tables: ["addresses", "postalcodes", "postalcode_changes"],
};

Config file resolution order:

1. --config CLI flag
2. POSTI_CONFIG environment variable
3. $XDG_CONFIG_HOME/posti/config.js (defaults to ~/.config/posti/config.js)
4. ./posti.config.js in current directory

Environment Variables

All settings can be set via environment variables (overrides config file):

POSTI_DIALECT=postgres
POSTI_HOST=localhost
POSTI_PORT=5432
POSTI_USER=posti
POSTI_PASSWORD=posti
POSTI_DATABASE=posti
POSTI_TABLE_PREFIX=posti_
POSTI_CONFIG=/path/to/config.js

Priority: CLI flags > environment variables > config file > defaults

Progress

The CLI displays a multi-bar progress indicator during processing:

files   ████████░░░░░░░░░░░░ 1/3 files
BAF_20250315.zip ████████████░░░░░░░░ Loading (4500/12000)

Steps shown per file: Downloading, Extracting, Converting encoding, Parsing, Loading (with row count), Upserting.

Programmatic Usage

import { run, loadConfig } from "posti";

const config = await loadConfig({
  dialect: "postgres",
  host: "localhost",
  database: "posti",
  user: "posti",
  password: "posti",
});

// With progress bar (default)
await run(config, { force: true });

// Silent mode (no progress output, useful for scripts/programmatic use)
await run(config, { silent: true });

Database Migrations

Posti uses a lightweight custom migration system. Migration files are TypeScript modules in src/db/migrations/ that use dialect-aware helpers to generate correct SQL for all supported databases.

Automatic migrations

By default, posti run applies any pending migrations before loading data. To disable this:

# Skip migrations on startup
posti run --no-migrate

# Or in config file
export default {
  migrate: false,
  // ...
};

Manual migrations

Run migrations explicitly without loading data:

posti migrate --dialect postgres --host localhost --database posti --user posti --password posti

How migrations work

• Migrations are tracked in a {tablePrefix}schema_migrations table (created automatically)
• Each migration runs once — already-applied migrations are skipped
• Migrations must be idempotent (use IF NOT EXISTS / IF EXISTS) since MySQL/MariaDB auto-commit DDL
• No down migrations — write a new forward migration to undo changes

Adding a new migration (development)

1. Create src/db/migrations/NNN_description.ts:

   import type { Migration } from "../migrate.js";
   import { execSQL, quoteIdentifier } from "../operations.js";
   
   export const myMigration: Migration = {
     name: "002_add_index",
     up: async (db, config) => {
       const table = quoteIdentifier(db.dialect, `${config.tablePrefix}addresses`);
       const col1 = quoteIdentifier(db.dialect, "postal_code");
       const col2 = quoteIdentifier(db.dialect, "municipality_id_code");
       await execSQL(db, `CREATE INDEX IF NOT EXISTS idx_example ON ${table} (${col1}, ${col2})`);
     },
   };

2. Register it in src/db/migrations/index.ts:

   import { myMigration } from "./002_add_index.js";
   export const migrations = [initialSchema, myMigration];

3. Add tests in test/unit/migrate.test.ts

Posti Postal Code Services

Documentation

Homepage:

• Finnish
• English
• Swedish

Service Description and Terms of Use:

• Finnish
• English
• Swedish

Data Files

The files can be downloaded from: https://www.posti.fi/webpcode/

Posti provides three data files:

| File | Table | Description | Update Frequency | |------|-------|-------------|------------------| | BAF | addresses | Basic Address File — street addresses with postal codes | Weekly (Saturdays after 15:00 EET/EEST) | | PCF | postalcodes | Postal Code File — all postal codes with metadata | Daily (except Sundays) | | POM | postalcode_changes | Postal Code Changes — history of postal code changes | Monthly (3rd business day) |

File Paths (XDG)

The tool follows the XDG Base Directory Specification:

| Purpose | Default Path | |---------|-------------| | Config | ~/.config/posti/config.js | | Cache (downloaded files) | ~/.cache/posti/ | | State (processing history) | ~/.local/state/posti/latest.json |

Override with XDG_CONFIG_HOME, XDG_CACHE_HOME, and XDG_STATE_HOME environment variables.

Development

Setup

The toolchain comes from mise: mise.toml pins every tool to an exact version (Node.js 26.13.0, the current LTS; pnpm 12; git-cliff; jq; shellcheck) and the committed mise.lock records each download's checksum.

git clone https://github.com/Kirbo/posti.git
cd posti
mise install
pnpm install

Bump a tool: change its pin in mise.toml, run mise lock and commit both files (for pnpm, also package.json packageManager; mise run check-versions says what disagrees). npm packages and tools younger than a day are never installed (pnpm-workspace.yaml minimumReleaseAge, mise.toml minimum_release_age).

Running Tests

# All tests
pnpm test

# Unit tests only
pnpm test:unit

# Integration tests (all four databases; see Local Databases below)
pnpm test:integration

# SQLite integration tests only (no server needed)
pnpm test:sqlite

# Watch mode
pnpm test:watch

# With coverage
pnpm test:coverage

Local Databases (Docker)

To run integration tests against MySQL, MariaDB, and PostgreSQL:

# Start database containers
docker compose up -d

# Services:
#   MySQL     — localhost:3306 (user: posti, password: posti, db: posti)
#   MariaDB   — localhost:3307 (user: posti, password: posti, db: posti)
#   PostgreSQL — localhost:5432 (user: posti, password: posti, db: posti)

# Run integration tests
pnpm test:integration

# Stop containers
docker compose down

Other Commands

# Type check
pnpm typecheck

# Lint (Biome; shellcheck for scripts/*.sh)
pnpm lint
pnpm lint:shell

# Build (compile TypeScript)
pnpm build

# Check the package as npm consumers get it (tarball contents, publint, attw, smoke test)
pnpm check:package

# Everything CI checks except the MySQL/MariaDB/PostgreSQL integration tests
mise run check

# Run CLI in dev mode
pnpm start -- --dialect sqlite --database ./test.db --force

# Run CLI in watch mode
pnpm dev -- --dialect sqlite --database ./test.db

Project Structure

src/
  cli.ts                  # CLI entry point
  cli/args.ts             # Argument parser
  index.ts                # Library exports
  config/                 # Configuration loading, Zod schema, XDG paths
  db/                     # Database connection, operations, dialect-specific SQL, migrations
  fetcher/                # URL parsing from Posti index page
  parser/                 # Fixed-width file parsing, encoding conversion, ZIP extraction
  loader/                 # Main orchestration pipeline, state tracking
  utils/                  # Logger, array chunking, time formatting

test/
  unit/                   # Unit tests (no external dependencies)
  integration/            # Database integration tests

scripts/                  # CI + release helpers (versions check, package check, next version, release)
docs/SECRETS.md           # npm trusted publishing (no secrets in this repo)

Commits and releases

Commits follow Conventional Commits: they decide the next version. On master, CI computes it with git-cliff (cliff.toml, scripts/next-version.sh): a breaking change (feat!: or a BREAKING CHANGE: footer) bumps the major, feat the minor, fix and perf the patch; other types alone release nothing. mise run next-version previews it.

The release job then stamps the version into package.json (in CI only: the repository keeps 0.0.0-development), stages it on npm and creates the GitHub release and tag with the notes (scripts/release.ts). A staged version is not installable yet: the maintainer approves it on npmjs.com ("Staged Packages" tab, with 2FA). Staging uses npm trusted publishing from GitHub Actions: no npm token exists. Setup and the npmjs.com values: docs/SECRETS.md. VERSION=x.y.z RELEASE_DRY_RUN=1 node scripts/release.ts (after pnpm build) runs the release checks and npm stage publish --dry-run locally.

CI

.github/workflows/ci.yml, on GitHub-hosted runners: check (the toolchain from mise.lock in locked mode, a frozen install, lint, typecheck, build, package check), audit, test on Node 22, 24 and 26 (unit + SQLite) and integration against MySQL, MariaDB and PostgreSQL service containers; on master also version and release. Pull requests get the same checks; a weekly run re-checks everything, including pnpm audit. Docs-only changes run nothing.

License

MIT