canfile

v3.0.1

Published

13 days ago

The Can Archive file format and toolkit.

0High
0Medium
0Low

thesummit

canfile file-format parser serializer extractor reader writer filesystem data-format nodejs npm

CanFile

CanFile is a standalone archival format and command-line toolkit for creating, inspecting, extracting, compressing, and encrypting portable archives.

Current release: 3.0.0

CanFile is implemented as an ESM Node.js package and is designed around a binary container format with explicit metadata, deterministic parsing rules, checksums, randomized reversible CanByte encoding, optional DEFLATE compression, and optional AES-256-GCM authenticated encryption.

Features

Binary CanFile container format
Recursive directory archiving
File and directory entries
SHA-256 integrity checks
CanByte randomized reversible encoding
DEFLATE compression
AES-256-GCM authenticated encryption
Password-derived encryption keys using scrypt
Combined compression and encryption
Archive inspection with caninfo
Gzip outer-layer support through cgz
Safe extraction path handling
Legacy format reading
ESM-native implementation
Zero npm runtime dependencies

Security model

CanByte is an encoding layer, not cryptography. It provides a randomized reversible representation and must not be treated as encryption.

When encryption is requested, CanFile uses AES-256-GCM. The encryption key is derived from the supplied password using scrypt. The password itself is never stored in the archive.

Authenticated encryption means that an incorrect password or modified ciphertext causes authentication failure rather than silently producing corrupted plaintext.

Installation

Install the published package globally:

npm install -g canfile

The package provides:

can
uncan
caninfo
cgz
can-help

The package can also be installed locally in another Node.js project.

Requirements

CanFile requires a Node.js runtime supporting modern ECMAScript modules and the Node.js standard-library APIs used by the implementation.

No third-party runtime npm dependencies are required.

Quick start

Create an archive:

can archive.can file.txt

Create an archive recursively:

can -r archive.can directory/

Extract an archive:

uncan archive.can

Extract into a specific directory:

uncan archive.can -d output/

Inspect an archive:

caninfo archive.can

Create a compressed archive:

can -c archive.can file.txt

Create an encrypted archive:

can -e archive.can file.txt

Create a compressed and encrypted archive:

can -c -e archive.can file.txt

Command reference

can

The can command creates CanFile archives.

Usage:

can [options] <output.can> <input>...

Options:

-c, --compress
    Compress CanByte payloads using DEFLATE.

-e, --encrypt
    Encrypt CanByte payloads using AES-256-GCM.

-r, --recursive
    Recursively include directory contents.

-v, --version
    Display the CanFile package version.

-h, --help
    Display help.

Directories are represented as directory entries. Their contents are included when recursive mode is enabled.

uncan

The uncan command extracts CanFile archives.

Usage:

uncan <archive.can>
uncan <archive.can> -d <directory>
uncan -d <directory> <archive.can>

Encrypted archives request their password interactively.

Extraction validates the archive structure, decrypts encrypted payloads, decompresses compressed payloads, reverses CanByte encoding, verifies original size, verifies SHA-256 checksums, and writes the resulting files.

caninfo

caninfo provides read-only archive inspection.

It reports:

archive size
format
format version
flags
entry count
original data size
stored data size
entry paths
entry types
compression/encryption mode
original size
stored size
payload offset
SHA-256 checksum

caninfo does not decrypt archive payloads.

cgz

cgz provides an outer gzip layer around CanFile archives.

This is separate from CanFile's internal DEFLATE payload compression.

can-help

can-help displays the command overview and examples.

CanFile format

CanFile is a binary container.

The format contains a header followed by an entry table and payload data. The entry table describes every stored file or directory.

The format version currently used by the writer is version 5.

Readers retain compatibility with earlier supported versions.

Format versions

Version 3 is retained as a legacy readable format.

Version 4 introduced per-file SHA-256 checksums.

Version 5 adds the current CanByte and authenticated-encryption modes while retaining compatibility with the preceding structure.

The reader accepts versions 3, 4, and 5.

Entry types

An entry represents either:

a folder
a regular file

Folders do not contain payload data.

Files contain payload metadata and payload bytes.

Checksums

Version 4 and later file entries contain SHA-256 checksums representing the original file contents.

The checksum is calculated before CanByte transformation, compression, or encryption.

During extraction the reconstructed original data is hashed and compared with the stored checksum.

This allows corruption to be detected independently of encryption.

Payload offsets

Each file entry records the location of its payload inside the archive.

Offsets allow the reader to locate file data without having to scan the entire archive sequentially for every entry.

CanByte

CanByte is CanFile's randomized reversible binary representation.

It is intentionally separate from encryption.

The purpose of CanByte is to transform arbitrary source bytes into a different binary representation while retaining exact reversibility.

Encoding the same input more than once may produce different output.

Decoding the generated representation reconstructs the exact original byte sequence.

Important security distinction

CanByte is not cryptography.

It does not provide confidentiality against an attacker who understands the format.

Encrypted CanFiles therefore perform:

source
    -> CanByte
    -> AES-256-GCM
    -> archive

rather than attempting to use CanByte as an encryption primitive.

CanByte pipeline

Normal CanByte storage:

source -> CanByte -> CanFile

Compressed storage:

source -> CanByte -> DEFLATE -> CanFile

Encrypted storage:

source -> CanByte -> AES-256-GCM -> CanFile

Compressed encrypted storage:

source -> CanByte -> DEFLATE -> AES-256-GCM -> CanFile

Decoding reverses the corresponding pipeline.

Compression

CanFile supports raw, DEFLATE, CanByte, CanByte plus DEFLATE, encrypted CanByte, and compressed encrypted CanByte payload modes.

Compression identifiers:

0  raw
1  deflate
2  CanByte
3  CanByte + deflate
4  CanByte + AES-256-GCM
5  CanByte + deflate + AES-256-GCM

The archive-level compressed flag identifies compressed CanFile containers, while the per-entry compression identifier describes the actual payload pipeline.

DEFLATE is provided by Node.js compression primitives in the JavaScript implementation.

The cgz tool is a separate gzip wrapper and should not be confused with CanFile's internal DEFLATE modes.

Encryption

CanFile encryption uses AES-256-GCM.

The password is converted into a 256-bit encryption key with scrypt.

Parameters used by the current implementation:

scrypt N = 16384
scrypt r = 8
scrypt p = 1
key length = 32 bytes
salt length = 16 bytes
IV length = 12 bytes
authentication tag length = 16 bytes

Every encrypted payload receives a fresh salt and IV.

The password is never stored in the archive.

Authentication

The authenticated data for each encrypted file incorporates:

entry path
zero separator byte
original SHA-256 checksum

The authenticated-data construction binds the encrypted payload to its specific archive entry.

Changing the password, ciphertext, authenticated metadata, or authentication tag causes authentication to fail.

The implementation reports authentication failures as:

CanFile: authentication failed (wrong password or corrupted payload)

Password handling

The CLI reads passwords interactively without displaying entered characters.

Encrypted archive creation requires confirmation of the password.

Extraction requests the password only when encrypted entries are present.

Empty passwords are rejected by the CLI password reader.

Integrity

CanFile performs multiple integrity checks.

Structural validation

The reader validates:

magic/header information
supported format version
archive flags
entry metadata
compression identifiers
payload boundaries

Size validation

After decoding and decompression, the reconstructed payload size is compared with the original size recorded in the entry.

SHA-256 validation

For version 4 and later file entries, the extracted plaintext is hashed with SHA-256 and compared with the stored checksum.

Authentication validation

Encrypted entries additionally receive AES-GCM authentication validation.

These checks operate at different layers and therefore provide complementary failure detection.

Safe extraction

Archive extraction must not allow an archive entry to escape the requested destination directory.

CanFile validates extraction paths before writing files.

This protects against malicious archive paths involving parent-directory components or absolute paths.

Applications embedding the library should retain this behavior rather than blindly concatenating archive paths with filesystem destinations.

JavaScript API

CanFile's implementation is organized into focused ESM modules.

can-format

Defines the binary-format constants, header information, entry structures, compression identifiers, and format versions.

canbyte

Provides the reversible randomized CanByte transformation.

can-crypto

Provides encrypted payload creation and authenticated decryption.

password

Provides interactive terminal password input for the CLI.

reader

Parses CanFile headers, metadata, entry tables, and payload boundaries.

extractor

Performs the reverse payload pipeline and filesystem extraction.

writer

Provides the archive-writing layer.

cgz

Provides the gzip wrapper functionality used by cgz.

The command-line programs are thin entry points around these implementation modules.

Development

The repository uses ECMAScript modules.

Syntax-check the command-line programs:

node --check bin/can
node --check bin/uncan
node --check bin/can-help
node --check bin/cgz
node --check bin/caninfo

Check all source modules:

for f in src/*.js; do node --check "$f" || exit 1; done

Create a package preview:

npm pack --dry-run

Create the package tarball:

npm pack

Functional testing

A complete functional release test should verify:

normal archive creation
normal extraction
compressed archive creation
compressed extraction
encrypted archive creation
encrypted extraction
compressed encrypted archive creation
compressed encrypted extraction
byte-for-byte comparison against the original files
archive inspection with caninfo

Encrypted tests should use a known test password and should never store production passwords in source control.

Compatibility

The JavaScript implementation currently reads CanFile format versions 3, 4, and 5.

Version 4 and version 5 archives carry SHA-256 file checksums.

Version 5 introduces the current encrypted payload representation.

Future native implementations should preserve binary compatibility with the format rather than merely reproducing command-line behavior.

A compatible implementation must preserve:

header semantics
entry semantics
compression identifiers
CanByte decoding rules
checksum interpretation
encrypted payload structure
authentication-data construction
path safety behavior

Troubleshooting

Authentication failed

If extraction reports:

CanFile: authentication failed (wrong password or corrupted payload)

verify that the password exactly matches the password used during creation.

If the password is correct, the encrypted payload or authenticated metadata may have been modified.

There is intentionally no plaintext password recovery mechanism.

Empty extraction directory

When archiving a directory, use recursive mode:

can -r archive.can directory/

Without -r, the directory entry itself may be stored without recursively including its contents.

Files appear under a source directory

This is expected when the input directory itself was supplied as the archive source.

For example:

can -r archive.can source/

may extract as:

output/source/...

This preserves the archive entry paths.

Inspecting encrypted archives

caninfo can display encrypted entry metadata without decrypting the payload.

Use uncan with the correct password to recover file contents.

Package contents

Before publishing a release, always run:

npm pack --dry-run

Check for accidental test archives, generated output directories, temporary files, credentials, or unrelated development artifacts.

Release checklist

Before publishing a release:

update package version
update README
run syntax checks
run functional tests
test normal archives
test compressed archives
test encrypted archives
test compressed encrypted archives
verify checksums
verify caninfo output
verify CLI help
inspect npm pack contents
remove release-test artifacts
verify no passwords or secrets are included
verify package size
run npm publish
verify the published version with npm view
install the published package separately
repeat smoke tests against the published package

Never publish a package simply because the working tree passes tests. The published tarball itself is the artifact users receive.

Native implementation roadmap

CanFile is designed so that a future native implementation can preserve the same archive format.

A C++ implementation can provide standalone binaries for environments where Node.js is undesirable or unavailable.

A potential native tree is:

native/
  CMakeLists.txt
  include/
    canfile/
  src/
    format.cpp
    canbyte.cpp
    crypto.cpp
    compression.cpp
    reader.cpp
    writer.cpp
    extractor.cpp
  tools/
    can.cpp
    uncan.cpp
    caninfo.cpp
    cgz.cpp

A native implementation should be independently tested against archives produced by the JavaScript implementation.

The compatibility target is the binary format, not JavaScript source-code equivalence.

Potential native dependencies include OpenSSL for AES-256-GCM and scrypt and zlib for DEFLATE/gzip compatibility.

Design principles

CanFile separates representation, compression, encryption, integrity, and container structure.

CanByte provides reversible randomized representation.

DEFLATE provides optional compression.

AES-256-GCM provides authenticated confidentiality.

SHA-256 provides content integrity verification.

The binary container provides metadata and random-access payload locations.

This separation makes the format easier to inspect, test, implement in another language, and evolve without confusing encoding with cryptography.

Technical reference: archive construction

CanFile treats archive construction as part of a layered archive implementation rather than as an unrelated utility feature.