Uniquer

Save equal files only once, avoid duplicates.

Install

NPM

npm install uniquer

Yarn

yarn add uniquer

Usage

Synchronously

import Uniquer from 'uniquer';

const fileName = Uniquer.writeSync('./output/', '.txt', 'Hello, world!');

// ./output/315f5bdb...5894edd3.txt
console.log(fileName);

Asynchronously

import Uniquer from 'uniquer';

const fileName = await Uniquer.write('./output/', '.txt', 'Hello, world!');

// ./output/315f5bdb...5894edd3.txt
console.log(fileName);

This may be all you need to use Uniquer. For advanced usage, see API

Advantages

📦 Save storage

No duplicates, more storage.

💻 Save performance

Only write a file if it doesn't exist.

📁 Get unique file names

No need to think of a creative file name.

🧹 Keep it clean

No duplicates, less files, cleaner.

How it works

The concept is that simple:

1. Create the SHA256 hash of the passed data.
2. Get the file name by appending the file extension.
3. Check if a file with this name already exists. If...
   • ...not, create it.
   • ...yes, do nothing.

This means: Same data, same file name. So, if you try to write the same data multiple times, it will:

• write the data only once.
• only create a single file.

There is no need to worry that there could be a duplicate file name, as there wasn't any collision with SHA256 ever. More here.

API

Uniquer.writeSync(directory, extension, data[, options])

Write data to a unique file synchronously.
Calling multiple times with the exact same data, will result in the same file name.

| | | | | ----------- | ---------------------------- | ---------------------------- | | directory | String | Path to the output directory | | extension | String | File extension | | data | String \| ArrayBufferView | The file's data | | [options] | String \| WriteFileOptions | Encoding or WriteFileOptions | | returns | String | File name of created file |

Example

import Uniquer from 'uniquer';
// 315f5bdb...5894edd3.txt
const fileName = Uniquer.writeSync('./output/', '.txt', 'Hello, world!');

Uniquer.write(directory, extension, data[, options])

Write data to a unique file asynchronously.
Calling multiple times with the exact same data, will result in the same file name.

| | | | | ----------- | ---------------------------- | ---------------------------- | | directory | String | Path to the output directory | | extension | String | File extension | | data | String \| ArrayBufferView | The file's data | | [options] | String \| WriteFileOptions | Encoding or WriteFileOptions | | returns | Promise<String> | File name of created file |

Example

import Uniquer from 'uniquer';
// 315f5bdb...5894edd3.txt
const fileName = await Uniquer.write('./output/', '.txt', 'Hello, world!');

Uniquer.getFileName(extension, data)

Resolve the unique file path, based on the file hash + extension.

| | | | | ----------- | --------------------------- | ------------------ | | extension | String | File extension | | data | String \| ArrayBufferView | The file's data | | returns | String | Resolved file name |

Example

import Uniquer from 'uniquer';
// 315f5bdb...5894edd3.txt
const fileName = Uniquer.getFileName('.txt', 'Hello, world!');

Uniquer.getFilePath(directory, extension, data)

Resolve the unique file path, based on the file hash + extension.

| | | | | ----------- | --------------------------- | ---------------------------- | | directory | String | Path to the output directory | | extension | String | File extension | | data | String \| ArrayBufferView | The file's data | | returns | String | Resolved file path |

Example

import Uniquer from 'uniquer';

// /home/user/output/315f5bdb...5894edd3.txt
const filePath = Uniquer.getFilePath('/home/user/output/', '.txt', 'Hello, world!');

Uniquer.getFileHash(data)

Get the SHA256 hash of the data, converted to hex format.

| | | | | --------- | --------------------------- | ---------------- | | data | String \| ArrayBufferView | The file's data | | returns | String | SHA256 file hash |

Example

import Uniquer from 'uniquer';

const hash = Uniquer.getFileHash('Hello, world!');

// 315f5bdb...5894edd3
console.log(hash);