@corellium/checksum

A library and cli for working with corellium checksums. Supports NodeJS and the browser.

Corellium Hashes are in the format <algo>$<chunkSizeInBytes>$<hash>.

Why?

We want to be able to generate file hashes in a performant way in a distributed system. To achieve this, we're going to hash the individual chunks of a file across the nodes consuming them during an upload. Then, when the upload is finished, we'll take the individual chunk hashes and hash those. This will give us a "composite hash" of the entire file for verification.

We chose the format <algo>$<chunkSizeInBytes>$<hash> so that we have enough information to rebuild the hash preserved in the hash itself. Given a file X and it's hash sha1$256000$abcde you can regenerate the hash for X by hashing it in chunks of 256kB with sha1 and then hashing all of those hashes with sha1. If the file is identical, then you should end up with abcde hash.

CLI

corellium-checksum ./path/to/file

If testing locally, npm link @corellium/checksum and then run.

Example

Here's a short gif showing the CLI in action. We generate a 100kB file of 0s and then generate a corellium checksum with 33kB chunks. After we generate the corellium checksum, we verify the hash by splitting the file apart in 33kB chunks, hashing each chunk with the same algorithm (sha1), and then hashing all the chunks.

For a 10gB file of random data it takes about 10 seconds when configured to hash in chunks of 50mB.

Browser Testing

In order to facilitate easier testing of the this packages browser compatibility, a simple https server has been created in this repo under the browser-testing directory. See the README.md in that directory for more information.