tokkit

Encode JavaScript objects into compact, URL-safe tokens using schema-based bit packing and Base62 encoding. Zero dependencies.

import { createCodec, t } from "tokkit";

const codec = createCodec({
  salary: t.int(0, 100_000, { step: 1000 }),
  remote: t.bool(),
  role: t.enum(["dev", "design", "pm"]),
});

const token = codec.encode({ salary: 45000, remote: true, role: "dev" });
// → "4xK9f"

const data = codec.decode(token);
// → { salary: 45000, remote: true, role: "dev" }

Share complex state via URL without a backend:

https://myapp.com/results?t=4xK9f

Install

npm install tokkit

Why tokkit?

The problem: You want to share application state via URL (filters, configs, calculator results), but JSON.stringify + encodeURIComponent produces bloated query strings that break on social media, messaging apps, and QR codes.

The solution: tokkit uses schema-based bit packing to encode your data into the minimum number of bits, then encodes those bits as a compact Base62 string (0-9, A-Z, a-z). No special characters, no percent-encoding, no backend needed.

| Approach | Output for {salary: 45000, remote: true, role: "dev"} | |---|---| | JSON + encodeURIComponent | %7B%22salary%22%3A45000%2C%22remote%22%3Atrue%2C%22role%22%3A%22dev%22%7D (74 chars) | | tokkit | 4xK9f (5 chars) |

Field Types

t.int(min, max, options?)

Integer in a range. Optionally quantized with step.

t.int(0, 100)                    // 0–100, step 1 → 7 bits
t.int(0, 100_000, { step: 1000 }) // 0–100000, step 1000 → 7 bits
t.int(18, 65)                    // 18–65 → 6 bits

Bits: ⌈log₂((max - min) / step + 1)⌉

t.bool(options?)

Boolean value. 1 bit.

t.bool()  // true or false → 1 bit

t.enum(variants, options?)

One value from a fixed set of strings.

t.enum(["low", "medium", "high"])  // 3 variants → 2 bits
t.enum(["a", "b", "c", "d"])      // 4 variants → 2 bits

Bits: ⌈log₂(variants.length)⌉

t.float(min, max, precision, options?)

Fixed-point decimal. Internally scaled to integer.

t.float(0, 1, 2)    // 0.00–1.00 → 7 bits (101 values)
t.float(0, 100, 1)  // 0.0–100.0 → 10 bits (1001 values)

Bits: ⌈log₂((max - min) × 10^precision + 1)⌉

t.constant(value)

A fixed value known from the schema. Uses 0 bits — not encoded in the token.

t.constant("v2")   // always "v2", 0 bits
t.constant(42)     // always 42, 0 bits

t.string(maxLength, charset?)

Variable-length string with optional custom charset.

t.string(10)                        // up to 10 chars, alphanumeric
t.string(6, "0123456789abcdef")     // hex string, up to 6 chars

t.array(element, maxLength)

Variable-length array of a single element type.

t.array(t.int(0, 255), 5)          // up to 5 bytes
t.array(t.bool(), 10)              // up to 10 flags
t.array(t.enum(["a", "b"]), 3)     // up to 3 enum values

t.object(schema)

Nested object with its own field layout.

t.object({
  x: t.int(0, 1000),
  y: t.int(0, 1000),
  visible: t.bool(),
})

t.optional(inner)

Nullable wrapper. Uses 1 presence bit + inner bits when present.

t.optional(t.int(0, 100))          // null or 0–100
t.optional(t.object({              // null or nested object
  vr: t.int(0, 5000),
  va: t.int(0, 5000),
}))

Default Value Compression

Any field that supports default uses 1 bit when the value matches the default, instead of the full field width.

const codec = createCodec({
  salary: t.int(0, 100_000, { step: 1000, default: 5000 }),
  remote: t.bool({ default: false }),
  role: t.enum(["dev", "design", "pm"], { default: "dev" }),
  rating: t.float(0, 5, 1, { default: 3.0 }),
});

// When all values match defaults → 4 bits (1 per field)
// When values differ → 1 + field bits per field

Supported by: t.int, t.bool, t.enum, t.float.

Codec Options

const codec = createCodec(schema, {
  checksum: "crc8",       // "crc8" (default) | "crc16" | "none"
  fingerprint: true,      // schema fingerprint in token (default: true)
  fingerprintBits: 16,    // 8 | 16 | 32 (default: 16)
  schema: {               // optional metadata (does not affect encoding)
    id: "my-schema",
    version: 1,
  },
});

Checksum

Detects token corruption (typos, truncation). CRC-8 adds 8 bits of overhead.

try {
  codec.decode("corrupted_token");
} catch (e) {
  // InvalidChecksumError
}

Fingerprint

Detects schema mismatches — tokens encoded with a different schema are rejected immediately instead of returning garbage data.

try {
  codec.decode(tokenFromDifferentSchema);
} catch (e) {
  // SchemaMismatchError
}

Token Layout

Tokens are packed as a single bigint, then encoded as Base62:

Low bits ←————————————————————————————→ High bits
[checksum (8/16/0)][version (4)][fingerprint (0/8/16/32)][payload (variable)]

API

codec.encode(payload)

Encode a payload object into a Base62 token string.

const token = codec.encode({ salary: 8000, remote: true, role: "dev" });

codec.decode(token)

Decode a token back into the original payload.

const data = codec.decode(token);

codec.validate(token)

Non-throwing validation. Returns diagnostic info without decoding the payload.

const result = codec.validate(token);
// {
//   valid: true,
//   checksumValid: true,
//   versionValid: true,
//   fingerprintValid: true,
// }

codec.inspect(token)

Full token debugging — decode with detailed bit-level information.

const info = codec.inspect(token);
// {
//   valid: true,
//   version: 1,
//   checksumValid: true,
//   fingerprintValid: true,
//   payload: { salary: 8000, remote: true, role: "dev" },
//   tokenLength: 5,
//   bits: { payload: 10, fingerprint: 16, checksum: 8, version: 4, total: 38 },
// }

codec.info()

Schema metadata and bit layout without a token.

const info = codec.info();
// {
//   version: 1,
//   totalBits: 10,
//   checksumBits: 8,
//   versionBits: 4,
//   fingerprintBits: 16,
//   overheadBits: 28,
//   totalBitsWithOverhead: 38,
//   estimatedBase62Chars: 7,
//   fields: [
//     { name: "salary", kind: "int", bits: 7, offset: 0 },
//     { name: "remote", kind: "bool", bits: 1, offset: 7 },
//     { name: "role", kind: "enum", bits: 2, offset: 8 },
//   ],
// }

codec.exportManifest()

Export the codec as a portable JSON manifest. The manifest fully describes the schema and can be used to recreate the codec in any language.

const manifest = codec.exportManifest();
// {
//   manifestVersion: 1,
//   protocolVersion: 1,
//   schema: { id: "my-schema", version: 1 },
//   options: { checksum: "crc8", fingerprint: true, fingerprintBits: 16 },
//   fields: {
//     salary: { type: "int", min: 0, max: 100000, step: 1000, default: 5000 },
//     remote: { type: "bool", default: false },
//     ...
//   },
// }

createCodecFromManifest(manifest)

Recreate a codec from a manifest JSON. Tokens produced are bit-for-bit identical.

import { createCodecFromManifest } from "tokkit";

const json = JSON.parse(fs.readFileSync("manifest.json", "utf-8"));
const codec = createCodecFromManifest(json);

const data = codec.decode(token); // works identically

Error Handling

All errors extend TokkitError:

| Error | When | |---|---| | ValidationError | Payload value is out of range or invalid type | | DecodeError | Token is empty or not a string | | InvalidChecksumError | Token bytes are corrupted | | SchemaMismatchError | Token was encoded with a different schema | | UnsupportedVersionError | Token protocol version is not supported | | SchemaError | Schema definition is invalid | | ManifestError | Manifest JSON is invalid or malformed |

import { TokkitError, InvalidChecksumError } from "tokkit";

try {
  codec.decode(token);
} catch (e) {
  if (e instanceof InvalidChecksumError) {
    // token is corrupted
  }
}

TypeScript

Full type inference from schema definition:

const codec = createCodec({
  name: t.string(20),
  age: t.int(0, 150),
  active: t.bool(),
});

type Payload = InferPayload<typeof codec>;
// { name: string; age: number; active: boolean }

Use Cases

• Shareable calculator results — salary simulators, loan calculators, tax comparisons
• Filter/search state — e-commerce filters, job board criteria, map views
• Config sharing — editor settings, theme configs, A/B test variants
• QR codes — compact tokens fit better in QR codes than JSON URLs
• Deep links — mobile app state encoded in the URL

License

MIT