PigeonFS

Peer-to-peer file sharing and distributed storage using PeerPigeon

PigeonFS enables direct file transfer between browsers with no central server. Send any file type (videos, images, documents, etc.) directly to peers, plus distributed key-value storage and searchable datasets.

Features

Direct P2P File Transfer

• 📁 Any File Type - Send videos (MP4), images, PDFs, documents, archives - anything!
• 🚀 Direct Transfer - Files sent peer-to-peer via WebRTC data channels (no server middleman)
• ⚡ Chunked Streaming - 64KB chunks with progress tracking for large files
• 🔄 Automatic Reconnect - Handles connection drops and peer discovery
• 💾 Browser Download - Received files can be saved directly to disk
• 📊 Transfer Stats - Real-time progress, speed, and file info
• 🔒 Optional Encryption - End-to-end encryption with AES-GCM (toggle in UI)

Distributed Storage (PagingStorage)

• 💾 Key-Value Store - Store any data type (strings, numbers, objects, binary)
• 🔄 DHT Routing - Consistent hashing distributes data across peers automatically
• 📑 Page-based Architecture - Efficient 4KB pages with automatic splitting/merging
• 🔐 Replication - Configurable replication factor (default: 3 copies)
• 💽 Persistence - IndexedDB backing with LRU cache for remote pages
• ⚖️ Load Balancing - Automatic redistribution when peers join/leave
• 🌐 Peer Sync - Real-time synchronization across connected peers

Dataset Search

• 📁 JSON Dataset Upload - Upload structured datasets to make searchable locally
• 🔍 P2P Search - Search datasets without loading them - queries route through peers
• 📚 Multiple Datasets - Generic system supports any dataset (Bible, dictionaries, wikis, etc.)
• ⚡ Book.js Integration - Fast radix tree indexing for efficient full-text search
• 💾 Smart Caching - Auto-caches search results to IndexedDB to contribute to network robustness
• 🌐 Gossip Protocol - Search requests broadcast to all connected peers
• 📊 Real-time Stats - Track local/peer results, network latency, and storage usage

Architecture

• PeerPigeon - P2P networking, gossip protocol, peer discovery, WebRTC data channels
• WebRTC Data Channels - Direct browser-to-browser file streaming
• PagingStorage - Distributed hash table for key-value storage and sync
• Book.js - Radix tree structure for fast prefix/substring search on datasets
• Vue 3 - Reactive UI with real-time updates
• IndexedDB - Persistent storage for DHT data, datasets, and cached results
• Vite - Fast dev server and build system

Installation

From NPM

npm install pigeonfs

From Source

git clone https://github.com/PeerPigeon/PigeonFS.git
cd PigeonFS
npm install

Usage

Desktop App (Recommended)

Download and run the desktop application with integrated server management:

# Run in Electron (development)
npm run electron

# Build desktop app for distribution
npm run electron:build

Features:

• 🖥️ Native desktop app for Windows, macOS, and Linux
• 🎛️ Built-in server control panel with GUI
• 📋 Live server logs and status monitoring
• ⚙️ Easy configuration without terminal commands

See ELECTRON.md for building and distribution.

Browser (Development)

npm run dev

Production Build

npm run build
npm run preview

CLI Server

Run a persistent Node.js peer to host files and datasets. Includes web upload interface at http://localhost:3000.

# Run with Node.js
npm run server

# Or build standalone executable (no Node.js required)
npm run package
./dist/pigeonfs-server-macos

Upload files via:

• Browser UI - Built-in upload interface in the "Server File Management" section
• Web interface - Drag & drop at http://localhost:3000
• HTTP API - curl -F "[email protected]" http://localhost:3000/upload
• Manual placement - Copy to data/files/ directory

Browser features:

• 📤 Upload files directly to server from browser
• 🔍 Search server files using Book.js index
• ⬇️ Download files from server
• 📋 View complete file list

See NODE_SERVER.md for full details.

Open in multiple browser windows/tabs to test P2P file transfer and storage across peers.

File Transfer Example

1. Connect to network - Enter a network namespace and click Connect
2. Select a file - Choose ANY file (video, image, PDF, etc.) from your computer
3. Get peer ID - Copy your peer ID and share with recipient
4. Send file - Paste recipient's peer ID and click "Send File"
5. Receive file - File appears in recipient's "Received Files" section
6. Download - Click download button to save file to disk

Works with:

• 🎥 Videos (MP4, WebM, AVI, etc.)
• 🖼️ Images (JPG, PNG, GIF, etc.)
• 📄 Documents (PDF, DOCX, TXT, etc.)
• 📦 Archives (ZIP, RAR, TAR, etc.)
• �� Audio (MP3, WAV, OGG, etc.)
• Any other file type!

Distributed Storage Example

1. Connect to network - Enter a network namespace and click Connect
2. Store key-value data - Use the PagingStorage section:

// Automatically routes to responsible peers via DHT
await storage.set('user:123', { name: 'Alice', age: 30 })
await storage.set('file:readme', 'Hello World!')

3. Retrieve from any peer - Data automatically fetched from DHT:

const user = await storage.get('user:123') // Routes to correct peer

4. View stats - See pages, keys, replication status across network
5. Export/Import - Download or restore full storage state

Dataset Search Example

1. Load a dataset - Click "Load King James Bible" (or add your own dataset)
2. Search locally - Type query like "faith" and press Enter
3. Search via peers - Open another tab/window, DON'T load the dataset, just search
4. See peer results - First tab serves results to second tab via P2P network

Optional Encryption

Enable end-to-end encryption for all P2P communications:

Browser:

1. Check the "🔒 Enable End-to-End Encryption" checkbox before connecting
2. All messages, file transfers, and storage will be encrypted

Node.js Server:

ENABLE_CRYPTO=true npm run server

How it works:

• Uses UnSEA crypto library (AES-GCM + ECDH key exchange)
• Each peer generates ECDSA key pair on initialization
• Messages encrypted with recipient's public key
• ~10-20% performance overhead
• ⚠️ All peers must use the same encryption setting (all on or all off)

When to use encryption:

• Sensitive file transfers (medical records, legal documents, etc.)
• Private conversations or data
• Untrusted networks
• Compliance requirements (HIPAA, GDPR, etc.)

5. Auto-caching - Second tab caches received results to help other peers

How It Works

Direct File Transfer Flow (WebRTC Data Channels)

Sender                    PeerPigeon Network           Receiver
  |                              |                         |
  |--select file---------------->|                         |
  |--get peer ID---------------->|                         |
  |                              |<--share peer ID---------|
  |                              |                         |
  |--establish WebRTC----------->|<--establish WebRTC------|
  |                              |                         |
  |==file chunks (64KB)=========>|==file chunks==========>|
  |                              |                         |--save to disk

Distributed Storage Flow (DHT-based)

Client                  DHT Hash Ring              Replica Peers (3x)
  |                          |                           |
  |--set('key', value)------>|                           |
  |                          |--hash(key)--------------->|
  |                          |--route to peers---------->|
  |                          |                           |--store locally
  |                          |<--acknowledge-------------|
  |<--success----------------|                           |

Search Flow (Gossip-based)

Searcher                    Network                     Data Holder
    |                          |                              |
    |---(search query)-------->|                              |
    |                          |---(gossip broadcast)-------->|
    |                          |                              |---(index search)
    |                          |                              |
    |                          |<---(results)-----------------|
    |<---(results + cache)-----|                              |

Dataset Support

Currently includes:

• King James Bible - 31,102 verses, full-text searchable

Easily add more datasets by adding to availableDatasets object with:

• name - Display name
• icon - Emoji icon
• url - Dataset JSON URL
• loader - Transform function to convert to Book.js format

Inspiration & Attribution

PigeonFS uses Book.js for its searchable dataset feature, which was inspired by Gun DB and Mark Nadal's vision for decentralized data structures. The Book.js radix tree implementation enables fast full-text search across distributed datasets.

Special thanks to:

• Mark Nadal - For pioneering P2P database concepts and Gun DB, which inspired the Book.js search architecture
• Gun DB community - For inspiration on distributed data structures
• Book.js - Fast radix tree implementation for full-text search
• PeerPigeon - WebRTC mesh networking and gossip protocol

License

MIT