@openai-hce/cli

Command-line utilities for Hierarchical Columnar Encoding (HCE). Encode JSON into HCE and decode HCE back to JSON directly from your terminal.

Installation

Install globally to expose the hce command everywhere:

npm install -g @openai-hce/cli

For project-local usage:

npm install @openai-hce/cli
# or
pnpm add @openai-hce/cli
# or
yarn add @openai-hce/cli

Then run with npx hce … from your project.

Commands at a glance

| Command | Purpose | |---------|---------| | hce encode | Convert JSON into HCE format | | hce decode | Convert HCE back to JSON | | hce convert | Explicit JSON ↔ HCE conversion helper | | hce stats | Display basic file statistics | | hce validate | Check whether a file is valid JSON or HCE |

Use hce <command> --help for the full option list.

Encode JSON

hce encode input.json -o output.hce

Useful options:

• -r, --root-key <name> – override the root label in the output.
• --field-delimiter <char> – set the field separator (default ,).
• --record-delimiter <char> – set the record separator (default |).
• --nested-delimiter <char> – set nested array separator (default ;).
• --missing-value <char> – placeholder for undefined and null (default space).
• --no-auto-grouping – disable secondary grouping.
• --stats – print compression statistics after encoding.

Example

curl -s https://api.example.com/users \
  | hce encode -r users --stats \
  | tee users.hce

Sample statistics output:

Input size: 45.1 kB
HCE size: 19.6 kB
Compression: 56.6 %
Time: 11 ms

Decode HCE

hce decode users.hce --pretty -o users.json

Helpful options:

• --pretty – pretty-print JSON ( 2 spaces).
• --compact – emit minified JSON (default).
• --extract <root> – write only the specified root group.
• --field-delimiter, --record-delimiter, --nested-delimiter, --missing-value – must match the encoder if custom delimiters were used.

Example

cat users.hce | hce decode --pretty | jq '.users[0]'

Output:

{
  "type": "user",
  "id": 1,
  "name": "Alice",
  "role": "admin"
}

Convert helper

The convert command wraps encode and decode with explicit format parameters.

hce convert data.json --from json --to hce -o data.hce
hce convert data.hce --from hce --to json --pretty

Validate files

hce validate payload.hce
hce validate payload.json --format json

Returns exit code 0 when the file is valid. Use --verbose to show the location of any parsing issues.

File statistics

hce stats payload.hce

Displays size, record counts and delimiter configuration. When run against JSON the command automatically encodes a sample block to estimate achievable compression.

Pipelines and scripting

# Encode logs as part of a nightly job
jq -c '.' logs.json | hce encode -r logs -o logs.hce

# Decode within a Node.js script
hce decode logs.hce | node scripts/process-logs.mjs

Combine the CLI with tools such as jq, shell pipes, or cron to build lightweight data pipelines.

Related packages

• @openai-hce/encode – programmatic encoder.
• @openai-hce/decode – programmatic decoder.

License

MIT © OpenAI HCE Team