opacity-streamable

v2.2.1

Published

2 years ago

Arbitrary size interface for the Opacity network

Downloads

0High
0Medium
0Low

edmundmai

opacity-streamable

A proof of concept, streamable implementation of the Opacity Protocol (not fully compatible) with a few significant changes:

CryptoJS replaced with node-forge
Encryption/decryption key is the SHA-256 hash of the handle, per-chunk IV (see #109)
Communication with the nodes via IXI module to save bandwidth
Pipelined processing:
- File > Encryption > Upload
- Download > Decryption > Blob

API Reference

opacity-streamable

Importing opacity-streamable

Example (As an ES Module)

import Opacity from 'opacity-streamable'

Example (As a Node Module)

const Opacity = require(`opacity-streamable`)

Example (As a UMD Module)

window.Opacity = Opacity.default

opacity-streamable
- .Download
  - .EVENTS
  - .toBuffer(handle, options) ⇒ Download
  - .toBlob(handle, options) ⇒ Download
- .Upload

Opacity.Download

Kind: static class of opacity-streamable
Emits: METADATA, DOWNLOAD_PROGRESS, FINISH

.Download
- .EVENTS
- .toBuffer(handle, options) ⇒ Download
- .toBlob(handle, options) ⇒ Download

Download.EVENTS

Events fired during the download lifecycle

Kind: static constant of Download

.EVENTS

"DOWNLOAD_PROGRESS"

Fired when a successful poll is performed while retrieving a file

Kind: event emitted by EVENTS
Properties

| Name | Type | Description | | --- | --- | --- | | progress | Object | a progress object | | progress.progress | Number | the percentage of progress for the download |

"FINISH"

Fired when the file has been reconstructed and is ready for use

Kind: event emitted by EVENTS
Properties

| Name | Type | Description | | --- | --- | --- | | file | File | Buffer | the file as an object as the target type of the download instance | | metadata | Object | the metadata object associated with the file |

"METADATA"

Fired when the file metadata has been reconstructed and is ready for use

Kind: event emitted by EVENTS
Properties

| Name | Type | Description | | --- | --- | --- | | fileName | String | the name of the file being downloaded | | ext | String | the file extension of the file being downloaded | | numberOfChunks | Number | the number of chunks that the file is stored in |

Download.toBuffer(handle, options) ⇒ Download

Kind: static method of Download

| Param | Type | Default | Description | | --- | --- | --- | --- | | handle | String | | the handle of the file to download | | options | Object | | the options for the download | | options.iotaProviders | Array.<Object> | Array.<IOTA> | | an array of IOTA initialization Objects or IOTA instances | | [options.autoStart] | Boolean | true | immediately start the download |

Example (To Buffer object (node))

const download = Opacity.Download.toBuffer(handle, {
  iotaProviders: [
    { provider: 'https://poll.oysternodes.com:14265/' },
    { provider: 'https://download.oysternodes.com:14265/' }
  ]
})

download.on('meta', metadata => {
  console.log(metadata)
  // {fileName: "opacity.txt", ext: "txt", numberOfChunks: 2}
})
download.on('finish', filedata => {
  console.log(filedata)
  // {file: Buffer(), metadata: {…}, target: Download}
})

Download.toBlob(handle, options) ⇒ Download

Kind: static method of Download

Example (To Blob object (browser))

const download = Opacity.Download.toBlob(handle, {
  iotaProviders: [
    { provider: 'https://poll.oysternodes.com:14265/' },
    { provider: 'https://download.oysternodes.com:14265/' }
  ]
})

download.on('meta', metadata => {
  console.log(metadata)
  // {fileName: "opacity.txt", ext: "txt", numberOfChunks: 2}
})
download.on('finish', filedata => {
  console.log(filedata)
  // {file: Blob(), metadata: {…}, target: Download}
})

Opacity.Upload

Kind: static class of opacity-streamable
Emits: INVOICE, CHUNKS_PROGRESS, UPLOAD_PROGRESS, FINISH

.Upload

Upload.EVENTS

Events fired during the upload lifecycle

Kind: static constant of Upload

.EVENTS

"INVOICE"

Fired when an invoice is recieved from the broker node

Kind: event emitted by EVENTS
Properties

| Name | Type | Description | | --- | --- | --- | | handle | String | the handle of the file uploaded | | address | String | an ethereum address to send the pearl to | | cost | Number | the cost of the file upload |

"CHUNKS_PROGRESS"

Fired when a chunk is uploaded to the broker

Kind: event emitted by EVENTS
Properties

| Name | Type | Description | | --- | --- | --- | | progress | Object | a progress object | | progress.progress | Number | the percentage of progress for the chunk upload |

"UPLOADED"

Fired all chunks have been uploaded to the brokers.

Kind: event emitted by EVENTS

"META_ATTACHED"

Fired when the meta chunk has been attached. This is needed in order to resume polling for upload progress.

Kind: event emitted by EVENTS
Properties

| Name | Type | Description | | --- | --- | --- | | target | Object | the upload object | | handle | String | the handle of the uploaded file | | numberOfChunks | Number | the number of chunks for the file | | metadata | Object | the metadata object |

"UPLOAD_PROGRESS"

Fired when a chunk is attached to the tangle

Kind: event emitted by EVENTS
Properties

| Name | Type | Description | | --- | --- | --- | | progress | Object | a progress object | | progress.progress | Number | the percentage of progress for the chunk attachment |

"FINISH"

Fired when the file has been completely attached to the tangle

Kind: event emitted by EVENTS
Properties

| Name | Type | Description | | --- | --- | --- | | handle | String | the handle of the file uploaded | | metadata | Object | the metadata object associated with the file |

Upload.EVENTS

Events fired during the upload lifecycle

Kind: static constant of Upload

.EVENTS

"INVOICE"

Fired when an invoice is recieved from the broker node

Kind: event emitted by EVENTS
Properties

"CHUNKS_PROGRESS"

Fired when a chunk is uploaded to the broker

Kind: event emitted by EVENTS
Properties

| Name | Type | Description | | --- | --- | --- | | progress | Object | a progress object | | progress.progress | Number | the percentage of progress for the chunk upload |

"UPLOADED"

Fired all chunks have been uploaded to the brokers.

Kind: event emitted by EVENTS

"META_ATTACHED"

Fired when the meta chunk has been attached. This is needed in order to resume polling for upload progress.

Kind: event emitted by EVENTS
Properties

"UPLOAD_PROGRESS"

Fired when a chunk is attached to the tangle

Kind: event emitted by EVENTS
Properties

| Name | Type | Description | | --- | --- | --- | | progress | Object | a progress object | | progress.progress | Number | the percentage of progress for the chunk attachment |

"FINISH"

Fired when the file has been completely attached to the tangle

Kind: event emitted by EVENTS
Properties

| Name | Type | Description | | --- | --- | --- | | handle | String | the handle of the file uploaded | | metadata | Object | the metadata object associated with the file |

Upload.fromFile(file, options) ⇒ Upload

Kind: static method of Upload

| Param | Type | Default | Description | | --- | --- | --- | --- | | file | File | | the file to upload | | options | Object | | the options for the upload | | options.iotaProvider | Object | IOTA | | an IOTA initialization Object or IOTA instance | | options.alpha | String | | the endpoint for the alpha broker | | options.beta | String | | the endpoint for the beta broker | | options.epochs | Number | | the number of years to store the file | | [options.autoStart] | Boolean | true | immediately start the upload |

Example (From File object (browser))

const file = fileInput.files[0];
const upload = Opacity.Upload.fromFile(file, {
  iotaProvider: { provider: 'https://poll.oysternodes.com:14265/' },
  alpha: 'https://broker-1.oysternodes.com/',
  beta: 'https://broker-2.oysternodes.com/',
  epochs: 1
});

upload.on('invoice', invoice => {
  console.log(invoice)
  // {address: "<ETH_ADDRESS>", cost: 20}
});
upload.on('finish', filedata => {
  console.log(filedata)
  // {handle: "<OPACITY_HANDLE>", metadata: {…}, target: Upload}
});

Upload.fromData(buffer, filename, options) ⇒ Upload

Kind: static method of Upload

| Param | Type | Default | Description | | --- | --- | --- | --- | | buffer | Buffer | | the data Buffer to upload | | filename | String | | the name of the file | | options | Object | | the options for the upload | | options.iotaProvider | Object | IOTA | | an IOTA initialization Object or IOTA instance | | options.alpha | String | | the endpoint for the alpha broker | | options.beta | String | | the endpoint for the beta broker | | options.epochs | Number | | the number of years to store the file | | [options.autoStart] | Boolean | true | immediately start the upload |

Example (From Buffer object (node))

const fs = require('fs');
const path = './path/to/file';
const filename = 'somefile.txt';

fs.readFile(`${path}/${filename}`, (err, data) => {
  if (err) throw err;

  const upload = Opacity.Upload.fromData(data, filename, {
    iotaProvider: { provider: 'https://poll.oysternodes.com:14265/' },
    alpha: 'https://broker-1.oysternodes.com/',
    beta: 'https://broker-2.oysternodes.com/',
    epochs: 1
  });

  upload.on('invoice', invoice => {
    console.log(invoice)
    // {address: "<ETH_ADDRESS>", cost: 20}
  });
  upload.on('finish', filedata => {
    console.log(filedata)
    // {handle: "<OPACITY_HANDLE>", metadata: {…}, target: Upload}
  });
});

What's next?

Possible things to look into:

Full compatibility
Pluggable source streams for uploads and target streams for downloads, allowing:
Node compatability
Video streams via MediaSource
Arbitrary size downloads via StreamSaver

Credits

This repo is based on the official opacity/webinterface and shares a fair amount of code with the project where it makes sense.

Big thanks to everyone working on Opacity!

Published

Vulnerabilities

Links

Maintainers

Keywords

Readme

opacity-streamable

API Reference

opacity-streamable

Opacity.Download

Download.EVENTS

"DOWNLOAD_PROGRESS"

"FINISH"

"METADATA"

Download.toBuffer(handle, options) ⇒ Download

Download.toBlob(handle, options) ⇒ Download

Opacity.Upload

Upload.EVENTS

"INVOICE"

"CHUNKS_PROGRESS"

"UPLOADED"

"META_ATTACHED"

"UPLOAD_PROGRESS"

"FINISH"

Upload.EVENTS

"INVOICE"

"CHUNKS_PROGRESS"

"UPLOADED"

"META_ATTACHED"

"UPLOAD_PROGRESS"

"FINISH"

Upload.fromFile(file, options) ⇒ Upload

Upload.fromData(buffer, filename, options) ⇒ Upload

What's next?

Credits