files-sync-stream

Sync files (or any blobs of data) between two peers, in both directions, over any transport. Originally made for avion.

• files-sync-stream is transport-agnostic. You need to pass two channels: one for sending signaling data and one for sending chunks of data.
• You decide where and how to store the files. files-sync-stream accepts a generic file read function and emits data events when receiving data.

Tings still to be implemented:

• A basic algorithm to determine the size of the next chunk to be transferred. Should be based on the time to transfer the last chunk.
• Handling of connection loss, including continuing the sync. This needs a basic have/want logic.

Installing

npm install files-sync-stream

Usage

This assumes the dataTransport and signalingTransport streams are connected the other peer somehow. You would do this for both peers.

const createEndpoint = require('files-sync-stream')

// 1st peer

const leader = createEndpoint(dataTransport, signalingTransport, true)

leader.on('file', (file) => { // handle incoming file
	file.on('start', () => {
		console.log('leader started receiving', file.id)
	})
	file.on('data', (chunk) => {
		console.log('leader received', file.bytesTransferred, chunk.toString('hex'))
	})
	file.on('end', () => {
		console.log('leader finished receiving', file.id)
	})
})
leader.on('done', () => console.log('leader is done'))

// 2nd peer

const follower = endpoint(dataTransport, signalingTransport)
follower.on('file', (file) => {
	// handle incoming file…
})

Use endpoint.add(read, metadata) to transfer a file:

const fromBuffer = require('files-sync-stream/from-buffer')

const data = Buffer.from('aef18a02dd912638', 'hex')
follower.add(fromBuffer(data), {name: 'file.bin'})

Contributing

If you have a question or have difficulties using files-sync-stream, please double-check your code and setup first. If you think you have found a bug or want to propose a feature, refer to the issues page.