deukpack

v1.9.0

Published

2 months ago

DeukPack — Universal schema multi-hub for Protobuf, Thrift, OpenAPI, and .deuk. Protobuf-aligned wire; fast C#/C++/JS codegen.

DeukPack: AI-Native Universal Schema Multi-hub

Universal Schema Multi-hub for Protobuf, Thrift, OpenAPI, and .deuk. (High-Performance)

Languages / 언어: English · 한국어

Turn any IDL (Protobuf, OpenAPI, JSON Schema, .deuk) into type-safe, deterministic code across C#, C++, TypeScript, JavaScript, Java, and Elixir — with a single, unified serialization API.

📢 [v1.9.0 Update] Official Python (Pure/Rust) Support and Industry Standard BMT Benchmark Revamp

We are pleased to announce the addition of the official Python (3.6+) binary engine (Pure/Rust Accelerated) to the DeukPack ecosystem. Furthermore, to ensure objective reliability in enterprise environments, we have discarded legacy measurement methods and elevated the benchmark environments across all languages to industry-standard frameworks (such as BenchmarkDotNet, mitata, and pytest-benchmark).

You can review the newly validated, highly transparent and strict performance metrics—measured under real-world complex object conditions—in our updated Performance Matrix White Paper.

Unified API: You no longer need to call verbose factory methods. All languages now intuitively share the same 2-Method struct syntax: Hero.Pack() and Hero.Unpack(). (Legacy APIs are temporarily maintained for backward compatibility).

⚠️ CAUTION (C# / Unity Users): If you manually copy generated .cs runtime files, you MUST completely empty that legacy folder prior to importing the new files to prevent duplicate declaration compile errors. (npm / UPM users are automatically updated).

Why DeukPack: The AI-Ready Advantage

1. Universal IDL Gateway (OpenAPI, JSON Schema, Protobuf, Thrift, CSV)

Modern systems often struggle with a fragmented mix of Specs—legacy (Thrift), modern (Protobuf/gRPC), and web-native (OpenAPI/JSON Schema). DeukPack acts as a Single Source of Truth that aggregates diverse IDL sources into one unified, semantic model — wire-compatible with your legacy protocols.

2. IDL-to-AI Semantic Mapping

DeukPack extracts metadata from IDL comments and field structures, transforming them into a Semantic Context that AI can instantly grasp. Designers evolve into architects defining data lineage via a machine-readable semantic layer.

3. AI-Native Execution Bridge (MCP Plugin Support)

The Model Context Protocol (MCP) server auto-generation feature (DeukPackMcp) lets AI agents (Cursor, Claude, etc.) browse live documentation and execute backend methods directly.

4. Core Memory Optimization (In-place Reuse)

Engineered for high efficiency. 60–100% reduction in memory allocation and 250% increase in JS parsing speed vs. classic industry flows. See the Performance section for raw numbers.

⚡ Two Words. Every Language.

DeukPack v1.7.6 introduces a universal 2-method serialization API: Pack and Unpack.
Regardless of language or wire format — binary, JSON, In-place — you only need to remember two verbs.

Pack    → Serialize (data out)
Unpack  → Deserialize (data in)

Pass a format parameter to switch protocols. Call Unpack on an existing instance for in-place overwrite.
That's the entire API surface.

Unity / C# Notice (GC Spike Prevention): NEVER use var h = Hero.Unpack(bin); (Factory method) for high-frequency network packets (Hotpath). It implicitly triggers new allocations, causing GC spikes and severe frame drops. You MUST use Hero.Unpack(cachedHero, bin); to overwrite pre-allocated (pooled) objects if you want to achieve true in-place architecture without GC stutters.

// C# / Unity: 1.Create  2.Pack  3.Unpack (In-place)
var hero = new Dto.Hero { id = 1, name = "Deuk" };
byte[] bin = Dto.Hero.Pack(hero);          // Serialize (Static)
Dto.Hero.Unpack(hero, bin);                // In-place Update

// TypeScript / JavaScript: 1.Create  2.Pack  3.Unpack (No Class Wrappers)
const hero = Dto.Hero.create({ id: 1, name: "Deuk" });
const bin = Dto.Hero.pack(hero);           // Serialize
Dto.Hero.unpack(hero, bin);                // In-place Update

// C++ (Native): 1.Create  2.Pack  3.Unpack (Memory Safe)
Dto::Hero hero; hero.id = 1; hero.name = "Deuk";
auto bin = Dto::Hero::Pack(hero);          // Serialize
Dto::Hero::Unpack(hero, bin);              // In-place Deserialize

// Java: 1.Create  2.Pack  3.Unpack (High-Performance)
Dto.Hero hero = new Dto.Hero(1, "Deuk");
byte[] bin = Dto.Hero.pack(hero);          // Serialize (Static)
Dto.Hero.unpack(hero, bin);                // In-place Overwrite (Static)

# Elixir: 1.Create  2.Pack  3.Unpack (BEAM Native)
hero = %Dto.Hero{id: 1, name: "Deuk"}      # Immutable Struct
bin = Dto.Hero.pack(hero)                  # Serialize
hero_parsed = Dto.Hero.unpack(bin)         # BEAM Pattern Match

🚀 Quick Start

npx deukpack init

1. Define your schema (or import OpenAPI / Protobuf)

namespace Dto

struct Hero {
    1> int32 id
    2> string name
    3> float hp
}

🔄 Backward Compatibility — Existing Code is Safe

All legacy method names are preserved as deprecated aliases. No breaking changes.

Your existing project will compile without errors. IDE will show soft deprecation warnings to guide migration at your own pace.

🎮 Real-World Pattern: Unity Game Client (In-place Reuse)

Dto.Hero cachedHero = new Dto.Hero(); // Allocated exactly ONCE at startup

void OnNetworkMessage(byte[] inputData) {
    // Zero-Garbage Deserialization — NO new class allocations, NO GC spikes!
    cachedHero.Unpack(inputData);
    Debug.Log($"Hero: {cachedHero.name}, HP: {cachedHero.hp}");

    // Mutate and re-serialize (Note: byte[] reuse requires Stream API)
    cachedHero.hp -= 10f;
    byte[] outputData = Dto.Hero.Pack(cachedHero);
    network.Send(outputData);
}

🚀 Release Roadmap

⚡ Performance: The Zero-Bottleneck Foundation

| Environment | Metric | 3rd-Party Tag-based | 3rd-Party RPC-based | DeukPack | | :--- | :--- | :---: | :---: | :---: | | C# / Unity | Speed | ~ 45 ms | ~ 85 ms | ~ 28 ms | | | Memory | +4.5 MB | +12.0 MB | 0 MB (Zero) | | C++ (Native) | Speed | ~ 14 ms | ~ 22 ms | ~ 12 ms | | | Memory | Heap Alloc | Heap Alloc | Manual Pool | | Java (Backend) | Speed | ~ 25 ms | ~ 38 ms | ~ 35 ms | | | Memory | Continuous | Large Objects | +2.1 MB (Min) | | JavaScript (V8) | Speed | ~ 54 ms | ~ 190 ms | ~ 158 ms | | | Memory | +4.2 MB | -1.9 MB | Immediate Reclaim | | Elixir (BEAM) | Speed | ~ 62 ms | ~ 98 ms | ~ 31 ms | | | Memory | +12.8 MB | +14.5 MB | 0 MB (Native Match) |

Figures based on decoding a 10,000-row payload. Results vary by environment.
Detailed cross-language matrix · Benchmarking Guide

Feature Matrix

| Category | Feature | TS / JS | C# / Unity | C++ | Java | Elixir | Python | | :--- | :--- | :---: | :---: | :---: | :---: | :---: | :---: | | IDL Core | Basic Types / Aliases | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | | Inheritance | extends support | ✅ | ✅ | ✅ | ✅ (v1.5) | ✅ | ✅ | | Unified API | Pack / Unpack (2-method) | ✅ v1.8.0 | ✅ v1.8.0 | ✅ v1.8.0 | ✅ v1.8.0 | ✅ v1.8.0 | ✅ v1.9.0 | | Protocols | Native Pack (.dpk) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ (Pure/Rust)| | | Protobuf Compatible | ✅ | ✅ | ✅ | ✅ | - | 🚧 | | | Thrift Compatible (T-Series) | ✅ | ✅ | ✅ (v1.5) | ✅ (v1.5) | - | - | | | JSON (Tagged / POJO) | ✅ | ✅ | ✅ (v1.5) | ✅ | - | ✅ | | | YAML / CSV | ✅ | ✅ (v1.2.7) | 🚧 | 🚧 | - | - | | Optimizations| In-place / JIT | ✅ (v1.6) | ✅ | ✅ (v1.4.2) | 🚧 | ✅ (BEAM) | ✅ (v1.9) | | | Write Logic Overrides | ✅ | ✅ | ✅ (v1.5) | ✅ (v1.5) | - | - | | Data/Meta | tablelink / MetaTable | ✅ | ✅ | ✅ (v1.5) | ✅ | - | - | | AI Integration | MCP Tool Auto-Generation | ✅ (v1.5) | 🚧 | - | - | - | - | | | Intelligent Context | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | | | IDE IntelliSense | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |

Installation

npm install deukpack
npx deukpack init
npx deukpack run         # default: ./deukpack.pipeline.json

Tutorials and OS-specific notes: deukpack.app/tutorial.

🛡️ Security & Reliability (OOM Defense / Anti-DDoS)

DeukPack implements strict defense-in-depth against network-layer parsing vulnerabilities (OOM, Buffer Flooding, Infinite Recursion):

Universal OOM Defense (v1.7.0+): Enforced MAX_SAFE_LENGTH (10MB) and MAX_ELEMENT_COUNT (1,000,000) limits across all engines. Malicious packets are discarded (Fail-Fast) before any memory allocation.
Progressive Chunk Validation: Replaces legacy ReadToEnd() with length pre-evaluation, neutralizing JSON stack bombing in Node.js and Java backends.
Continuous DDoS Fuzzer Suite: CI-integrated test-fuzz-oom.js injects parsers with 2GB+ abnormal buffers.

Documentation & Links

Contact: [email protected]

Development

npm ci
npm run build
npm test
npm run benchmark                     # Node serialize smoke
npm run bench:format-parity           # parser comparison
npm run bench:cross-lang              # Node vs .NET pack

☕ Support & Contact

DeukPack is completely open-source (Apache 2.0). Built to solve fundamental GC bottleneck and model synchronization issues from 30 years of server architecture experience.

📩 Contact / Technical Inquiries: [email protected]
☕ Support the Project: Sponsor via Ko-fi

Starring the repo or sharing it with teams juggling Protobuf/Thrift helps immensely.

Works well with (Deuk Family)

Want agents to do more with your specs? Use DeukPack — IDL in, deterministic types and serializers out.
Want agents to behave predictably in your repo? Use DeukAgentRules — versioned AGENTS.md and rule templates.

npm install -D deuk-agent-rule
npx deuk-agent-rule init --non-interactive

⚡ Performance & Benchmarks

To ensure objective performance testing, we have adopted industry-standard benchmarks such as BenchmarkDotNet, mitata, and pytest-benchmark. Achieving 60-100% reduction in memory allocation and 2.5x to 5x faster parsing speeds compared to legacy industry formats. If you find any errors or have suggestions regarding the benchmark scenarios, please let us know.

Performance test technical document

Ecosystem (Deuk Family)

Contributing

Fork → feature branch → PR.
See RELEASING.md for release layout.

License

Apache License 2.0 — LICENSE · NOTICE.

Acknowledgments

The broader IDL / OpenAPI / schema communities; DeukPack is a standalone pipeline (not an Apache Thrift subproject).