fs2 — Minimal Two-Machine Directory Sync (MVP)

fs2 is a small Rust CLI that syncs one directory from a host machine to a client machine over TCP:

• Host runs: fs2 sync
• Client runs: fs2 connect fs2://...
• Client applies an initial snapshot, then both sides stay in sync

This is intentionally MVP-quality and optimized for clarity and demo velocity.

Status and Safety Notes

This project is early WIP. It works, but you should assume rough edges.

Important limitations:

• No encryption (traffic is plaintext TCP)
• Auth is a bearer token in the URL
• No NAT traversal or relay support yet
• Full-file transfers (not block-level deltas)
• Conflict handling is minimal (last write wins in practice)
• Non-UTF-8 paths are not handled losslessly

Some safety guards do exist, but they are intentionally minimal:

• Incoming paths are validated to stay under the sync root
• Hash mismatches fail closed (the file is not applied)
• Frame payloads are capped at 1 MiB
• Handshake and snapshot/control reads have timeouts
• PipeNet mode binds the local listener to 127.0.0.1

Do not use this for sensitive data.

Requirements

• Node.js 18+ if installing via npm
• Rust 1.85+ (edition = "2024") for local development or installing from source via Cargo
• Internet access to pipenet.dev (default mode)
• For direct TCP mode: two machines that can reach each other and an open port (default: 4855)

Install

Install globally via npm (downloads a prebuilt binary; the published npm package does not include Rust sources):

npm i -g fs2-cli

Supported npm platforms right now:

• darwin-arm64
• darwin-x64
• linux-x64-gnu
• linux-x64-musl
• win32-x64-msvc

Other platforms should install from source via Cargo.

Run via npx without a global install:

npx fs2-cli --help

Install the binary locally from this repo:

cargo install --path . --force

Or via the provided Makefile:

make install

Quickstart (Two Terminals / Two Machines)

1) Host: start a sync session

On the host machine:

cd /path/to/project
fs2 sync

You will see output like:

• A single shareable client command: fs2 connect 'fs2://...'
• The URL is embedded in that command (there is no extra banner text)
• If you started the host with npx fs2-cli sync, the printed command will match: npx fs2-cli connect 'fs2://...'

Tip: set NO_COLOR=1 if you want plain, non-colored output.

By default, fs2 sync uses a PipeNet tunnel (no public port needed). For local/offline development or direct TCP hosting, use --direct:

fs2 sync --direct

Direct mode is often paired with an explicit public address:

fs2 sync --direct --public-addr 203.0.113.10:4855

2) Client: connect and choose a destination root

On the client machine:

fs2 connect 'fs2://HOST/SESSION.TOKEN'

If you want an explicit destination root:

fs2 connect 'fs2://HOST/SESSION.TOKEN' --root /path/to/destination

Notes:

• connect --root will create the directory if it does not exist
• When you omit --root, fs2 compares the current directory with the remote snapshot and warns if there is no overlap
• With RUST_LOG=info, the client logs when it has connected and when it starts pulling the initial snapshot
• Files may be overwritten

URL Format

Connection URLs are intentionally short and copy-friendly:

Direct (default port 4855):
  fs2://HOST/SESSION.TOKEN

Direct (non-default port):
  fs2://HOST:4900/SESSION.TOKEN

PipeNet:
  fs2://pipenet/SESSION.TOKEN?t=odd-robin-62

CLI Reference

Top-level help:

fs2 --help

fs2 sync

Hosts a sync session from a root directory.

fs2 sync --direct --root /path/to/project --port 4855 --public-addr 203.0.113.10:4855

Options:

• --root <PATH>: root directory to sync (default: current directory)
• --port <PORT>: port to listen on (default: 4855)
• --public-addr <ADDR:PORT>: advertised reachable address
• --direct: use direct TCP instead of PipeNet
• --remote <direct|pipenet>: connection strategy (default: pipenet)
• --pipenet-host <URL>: PipeNet host (default: https://pipenet.dev)
• --pipenet-subdomain <NAME>: request a specific PipeNet subdomain

fs2 connect

Connects to the host and syncs into a destination root. By default it syncs into the current directory.

fs2 connect 'fs2://HOST/SESSION.TOKEN'
fs2 connect 'fs2://HOST/SESSION.TOKEN' --root /path/to/destination

Arguments:

• <TARGET>: connection URL printed by fs2 sync

Options:

• --root <PATH>: destination root (default: current directory)

Makefile Shortcuts

The Makefile provides convenient wrappers around cargo run:

make sync ROOT=. PORT=4855 PUBLIC_ADDR=203.0.113.10:4855
make connect URL='fs2://HOST/SESSION.TOKEN' ROOT=./mirror

Note: make sync defaults to REMOTE=direct to keep local dev flows offline-friendly. Set REMOTE=pipenet to use the tunnel:

make sync REMOTE=pipenet ROOT=. PORT=4855

Developer Onboarding (Fast Iteration + Hot Reload)

The goal is to run the current repo code from anywhere, and optionally restart it automatically when you change Rust files.

1) One-time setup: link dev shims into your PATH

This makes fs2-dev and fs2-dev-watch available from any directory.

make dev-link-all
export PATH="$HOME/.local/bin:$PATH"

Notes:

• Default bin dir is ~/.local/bin (override with DEV_BIN_DIR=...)
• Default shim names are fs2-dev and fs2-dev-watch

2) Run the dev binary from any directory

fs2-dev always runs this repo's current code via cargo run.

fs2-dev sync --direct --root /path/to/host --port 4855
fs2-dev connect 'fs2://HOST/SESSION.TOKEN' --root /path/to/client

3) Enable hot reload (auto-restart on code changes)

Install cargo-watch once:

cargo install cargo-watch

Then use fs2-dev-watch for the command you want to auto-restart:

fs2-dev-watch sync --root /path/to/host --port 4855

Important behavior:

• Hot reload restarts the process, which creates a new session URL
• Re-run the client after a host restart

4) Quick end-to-end dev flow

Run a full local host+client flow (with temp dirs and faster scan settings):

make dev-flow

Keep the temp dirs and logs if you want to inspect them:

FS2_KEEP_DEV_FLOW=1 make dev-flow

5) Faster feedback during development

You can speed up scan timing during development via env vars:

FS2_PERIODIC_RESCAN_SECS=2 FS2_RESCAN_DEBOUNCE_MS=100 fs2-dev sync --root /tmp/a

Supported overrides:

• FS2_PERIODIC_RESCAN_SECS (default: 15)
• FS2_RESCAN_DEBOUNCE_MS (default: 350)
• FS2_RECENT_APPLY_TTL_SECS (default: 10)

How It Works (Current Design)

High-level flow:

1. Host listens on TCP
2. Client connects and performs a small handshake
3. Client requests a snapshot
4. Host streams all files
5. Client acks the snapshot
6. Both sides watch + rescan and exchange updates

Key design choices:

• File scanning uses the ignore crate and respects .gitignore
• .git/ is always excluded
• Temporary files containing .fs2-tmp- are excluded
• Updates are whole-file transfers in chunks (256 KiB)
• Incoming filesystem paths are validated before apply/delete
• Integrity mismatches are treated as errors rather than warnings
• Protocol frames larger than 1 MiB are rejected

Logging and Debugging

Logging is powered by tracing. The default log level is warn to keep the URL output clean. Use RUST_LOG to increase verbosity when needed:

RUST_LOG=info fs2 sync
RUST_LOG=debug fs2 connect 'fs2://HOST/SESSION.TOKEN'

Repository Map

If you are modifying code, these are the most important files:

• src/cli.rs: CLI surface area
• src/session.rs: orchestration (handshake, snapshot, live sync)
• src/sync.rs: scan/diff/apply and watcher integration
• src/transport/tcp.rs: framing and TCP transport
• src/protocol.rs: protocol message definitions

There is also a design sketch in rough-idea.md.

Known Gaps / Next Improvements

Some natural next steps:

• Add transport security and stronger authentication
• Improve conflict handling and safety checks
• Add integration tests that run two local processes
• Consider resumable transfers and/or block-level deltas

Publishing to npm

The root fs2-cli package is now a thin wrapper that depends on platform-specific packages containing the prebuilt binary. The published npm package intentionally excludes Rust sources, so npm installs must resolve to a matching platform package. The Cargo fallback only works in a repo checkout where Cargo.toml is present. Platform packages are scoped under @t3dotgg (for example @t3dotgg/fs2-cli-darwin-arm64) to avoid npm spam detection on new unscoped platform package names.

Preferred: CI release on tag

There is now a GitHub Actions release workflow that publishes platform packages first and the root wrapper last.

1. Update the version in the root package.json.
2. Sync versions: npm run sync:platform-versions
3. Ensure the repo is clean: git diff -- package.json npm/platforms
4. Commit, tag, and push: git tag v<version> && git push && git push --tags

Before the first CI publish, configure npm trusted publishing for fs2-cli and each @t3dotgg/fs2-cli-* platform package, and point it at the release.yml workflow file. Once trusted publishers are configured, this workflow does not require an NPM_TOKEN secret.

Note: npm provenance currently requires a public GitHub repository. This repo is private, so the release workflow publishes without provenance.

Manual: publish a single platform + wrapper

For the current host platform:

1. Sync versions: npm run sync:platform-versions
2. Build the platform package binary: npm run prepare:platform
3. Publish the platform package: cd npm/platforms/<suffix> && npm publish --access public
4. Publish the wrapper from the repo root: npm publish --access public

For linux musl builds, also set a target triple:

FS2_PLATFORM_SUFFIX=linux-x64-musl \
CARGO_BUILD_TARGET=x86_64-unknown-linux-musl \
npm run prepare:platform