@desklink/host
v0.5.15
Published
Per-user desktop capture, encode and input engine for a remote desktop inside your own app, with a versioned local control protocol
Maintainers
Readme
It is a process with a documented local protocol, not a service. A consumer starts it, owns its stdin/stdout, carries the SDP/ICE over its own authenticated channel, and stops it. Nothing in the engine's public surface refers to any particular application.
What it does
The engine supports Linux desktop sessions, and macOS (preview) desktop sessions.
- Capture through the XDG Desktop Portal (
ScreenCast) and PipeWire, with compositor consent, from an explicitly selected X display's root window, or from a selected macOS display through ScreenCaptureKit. The macOS backend currently requests images at the session frame cap; the source may supply fewer frames. The portal path asks for shared-memory buffers rather than DMA-BUFs. - Encode VP9 with libvpx in real time, single pass, no lookahead: one frame in, one packet out. Software by default, because a hardware encoder would put a vendor driver dependency on every machine class. Tuned for a desktop: the desktop's own pixels up to 4K (a phone zooms in, and text has to survive it), screen-content mode, key frames on request and when the coded size changes, and a refinement pass that re-codes a still desktop once at a fine quantizer so it reads sharp. Motion the CPU cannot code at the frame rate is coded at half size until it stops or shrinks enough to keep up; the refinement pass is always full size.
- Encode H.264 as well, for a receiver that asks for it. The engine offers
VP9 first and H.264 beside it, and codes whichever the receiver's answer puts
first, so VP9 stays the default; the iOS receiver asks for H.264 because iOS
decodes it in hardware and VP9 in software. H.264 comes from the platform's
encoder, loaded at runtime — VideoToolbox on macOS, NVENC or VA-API on Linux —
and otherwise from Cisco's prebuilt openh264, which the engine downloads from
Cisco on first need and checks against Cisco's published hash. openh264 is
never built from source or shipped in this package (OpenH264 Video Codec
provided by Cisco Systems, Inc.). See
docs/PROTOCOL.md(Capabilities, Codecs). - Transport with WebRTC — ICE, DTLS, SRTP, RTP — and one data channel for the session's pointer/keyboard/clipboard.
- Carry video the consumer encoded itself, on an
encodedsource: Annex-B H.264 access units arrive as they are produced, and the session's pointer, keyboard and wheel messages come back to the consumer as events instead of being applied. This is for a stream whose encoder lives elsewhere — a phone mirror from a device helper — so the helper keeps its own hardware encoder and the engine keeps the WebRTC path. Seedocs/PROTOCOL.md. - Input through inputtino
(MIT) over
uinput/libevdevfor portal capture, XTest for an X display, or Quartz events on macOS. Held input is released when the session ends. - Clipboard explicitly, in both directions, only when asked. It is never
polled and never used as a hidden way to type. On Wayland, writes require
wl-copyfromwl-clipboard; macOS uses the plain-text general pasteboard. X11 uses the session's selected display; see the clipboard protocol for supported targets, transfer limits and selection lifetime.
What it deliberately does not do
- It does not decide who the user is. The consumer pairs; a paired consumer gets all available view, input and clipboard access, without per-session grants.
- It does not open a public listener, run as root, install udev rules, join groups or raise capabilities.
- For local system audio capture and remote playback limitations, see the audio protocol.
- It does not arbitrate between two controllers, or remember a session across a restart.
Install and run
The npm badge tracks the registry version. npm installs the matching optional
engine package: @desklink/host-linux-x64-gnu or @desklink/host-darwin-arm64.
Supported platforms: Linux x64 with glibc 2.36 or newer and macOS arm64 (preview).
See the changelog for release changes.
npm install @desklink/host
npx desklink-host path # the engine this install will run
npx desklink-host capabilities # what this machine can do right now
npx desklink-host doctor # check every Linux requirement before a sessiondoctor prints one PASS/FAIL line per requirement with the exact free package
command for each failure, and exits nonzero when one is missing. It starts no session.
On Linux x64 with glibc 2.36 or newer — Debian 12, Ubuntu 24.04, Fedora 37
or later, Arch — that is the whole install. npm also installs the optional
platform package @desklink/host-linux-x64-gnu, which carries the prebuilt
engine, and resolveEngine() finds it there. No Rust toolchain, compiler or
install script is involved, so --ignore-scripts installs work. The engine
requires the system's libpipewire-0.3.so.0, libxkbcommon.so.0,
libevdev.so.2, libstdc++.so.6, libXcursor.so.1, libX11.so.6,
libXext.so.6, libwayland-client.so.0, libfontconfig.so.1, and libfreetype.so.6
(normally supplied by a modern Linux desktop). If one is missing, the host reports missing system library: <name>
with code missing-system-library rather than failing silently; it does not
bundle these system libraries. libvpx and inputtino are linked into the engine.
Wayland point labels use Fontconfig (MIT) and FreeType (FTL) from the system;
this software is based in part on the work of the FreeType Team.
The one step no install does for you is kernel input access, and only the portal backend needs it.
macOS arm64 (preview) also has a prebuilt engine. There is no prebuilt engine for other platforms. Another Linux (arm64, musl) can build it from source.
macOS (preview)
The macOS engine shares protocol v3's WebRTC session path: selected-display
ScreenCaptureKit capture, VP9 encoding (H.264 through VideoToolbox for a receiver
that asks), Quartz pointer/keyboard events, and
plain-text NSPasteboard operations. capabilities is non-prompting; session.open
requests Screen Recording and, for input, Accessibility consent when needed.
For persistent grants, install the signed DesklinkHost.app at one fixed path.
Build the engine on the Mac with cargo build --release (the packed release CLI, built with
DESKLINK_MACOS_CLI, has no bundle identity and the installer refuses it), then run
release/install-mac-app.sh <engine> <absolute path to DesklinkHost.app> [existing signing PEM]
from this package. It prefers a usable Developer ID Application or Apple
Development identity in the login keychain; otherwise it imports the supplied
existing certificate/key (to preserve existing grants), or creates one local
certificate/key in ~/Library/Application Support/Desklink/signing.pem, and
trusts it in the login keychain. Unlock the login keychain in the Mac GUI if
import or trust asks; never supply a person's password to automation. Do not
regenerate that identity, move the app, or replace and re-sign its executable
per run. Rebuild, install once at the same path with the same identity, and
verify codesign -dv --verbose=4 <app> and codesign -dr - <app>; the designated
requirement must be certificate-based, not a cdhash requirement. Signatures
made from the same certificate have a stable designated requirement even when
the binary hash changes. The app does not pre-seed or bypass TCC. Launch it
from the logged-in Mac session, grant Screen & System Audio Recording and
Accessibility to DesklinkHost once in System Settings, then launch twice
and check log show --last 5m --predicate 'process == "tccd"' for new denials.
TCC belongs to the responsible app: launching the unsigned CLI or a Node
bridge from SSH/Terminal can instead request grants for Node or Terminal.
To verify a cue visually, capture through the granted identity itself:
session.frame writes the latest pre-encode frame to a path. A
screencapture taken from an SSH shell is attributed to that shell's
responsible app and comes back without app windows, so a cue that is on
screen will not appear in it.
Opting in to observeInput (pointer buttons, cursor and keys on the desktop,
for recording what the person does) also needs Input Monitoring for
DesklinkHost: System Settings › Privacy & Security › Input Monitoring. Without
it the open reply says inputObservation.code: "missing-input-monitoring" and no
events are sent. See the protocol's "Observing the desktop's own input".
macOS currently uses a US ANSI virtual-key map for character chords (the
current input layout is diagnostic); free-form text uses Unicode key events.
The published engine is an unsigned macOS arm64 CLI in the optional
@desklink/host-darwin-arm64 platform package; no Developer ID identity or
notarization is required. The macOS (preview) engine is enabled by default; DESKLINK_MACOS=0 explicitly disables it.
To build and pack it, build on an Apple Silicon Mac with Rust 1.97.1+ and static
Homebrew libvpx, then follow
Building and packing a release.
For a source build, DESKLINK_VPX_STATIC_DIR is the libvpx install prefix
(containing include/ and lib/libvpx.a), not its lib/ directory.
The app-bundled signed harness is for local TCC qualification, not npm
installation. See the macOS protocol.
Point DESKLINK_ENGINE at a binary built elsewhere if you have one. (MUXR_DESKLINK_ENGINE still works as a deprecated fallback and will be removed in a later release.) The
package never searches PATH for a same-named program: "a binary called
desklink-host" is not evidence of which program is about to be given control of
someone's desktop.
Building from source
cd packages/desktop-host # from the repository root
cargo build --release --manifest-path engine/Cargo.toml
./bin/desklink-host.mjs path # prints the binary this package will run
./bin/desklink-host.mjs capabilitiesOn macOS, the launcher resolves the local candidate by default. The app-bundled dev harness is not the npm package; a source build is an unsigned CLI and its TCC permissions are attributed to the responsible launching app.
On Linux, a source build links system libraries through pkg-config (and
libxcb directly), compiles native/vpx_shim.c against libvpx's headers, and
builds the vendored inputtino project with CMake. On a clean Linux machine,
install:
| Need | Debian/Ubuntu | Fedora | Arch |
|---|---|---|---|
| Rust toolchain (cargo, cc) | rustup + build-essential | rustup + gcc gcc-c++ | rustup + base-devel |
| CMake | cmake | cmake | cmake |
| pkg-config | pkg-config | pkgconf-pkg-config | pkgconf |
| libvpx (vpx.pc) | libvpx-dev | libvpx-devel | libvpx |
| PipeWire 0.3.65+ (libpipewire-0.3) | libpipewire-0.3-dev | pipewire-devel | pipewire |
| xkbcommon | libxkbcommon-dev | libxkbcommon-devel | libxkbcommon |
| FreeType (FTL; dynamically linked) | libfreetype6-dev | freetype-devel | freetype2 |
| Fontconfig (MIT; dynamically linked) | libfontconfig1-dev | fontconfig-devel | fontconfig |
| Wayland (wayland-client) | libwayland-dev | wayland-devel | wayland |
| libevdev | libevdev-dev | libevdev-devel | libevdev |
| libxcb (source/container builds only) | libxcb1-dev | libxcb-devel | libxcb |
| Xcursor | libxcursor-dev | libXcursor-devel | libxcursor |
| Xext | libxext-dev | libXext-devel | libxext |
These are build prerequisites, not additional runtime requirements for the
prebuilt package; in particular, libxcb is not a prebuilt runtime requirement.
Nothing is downloaded by the build itself. cargo test --manifest-path
engine/Cargo.toml needs all of these installed; only a Linux machine with them
compiles and tests the crate.
On macOS the source build does not use these Linux native libraries or shims;
run cargo test --bin desklink-host --manifest-path engine/Cargo.toml for its
binary tests (the full test command also builds a Linux-only example).
Kernel input access
Ordinary tests do not create real desktop devices. To opt into the hardware
probe explicitly, run DESKLINK_TEST_UINPUT=1 cargo test this_host_can_create_and_destroy_the_virtual_devices
inside engine/ on a desktop you are authorized to control.
Creating virtual input devices needs write access to /dev/uinput. That is
whole-desktop control of the logged-in session, so it is a decision the user
makes, not something an installation does for them:
npx desklink-host setup-input # where @desklink/host is installed
npx -p @desklink/host desklink-host setup-input # anywhere else, e.g. under an application that depends on itThat prints the exact, narrowly scoped rule — and changes nothing. Without
uinput access a portal session still opens and captures; input actions are
refused with input-unavailable until access is set up. An X
session instead uses XTest and does not need /dev/uinput.
Licence and provenance
Apache-2.0. The Linux engine links two permissive native libraries — libvpx
(BSD-3-Clause) and inputtino (MIT, vendored under engine/vendor/) — and calls
the portal over D-Bus rather than linking it. See NOTICE. The prebuilt engine
links libvpx statically and carries every licence text it owes, including each
linked crate's, in its package's THIRD_PARTY_LICENSES.txt, with the Rust
standard library's own notices in COPYRIGHT-rust-library.html. Its build
refuses any crate whose licence is not permissive.
No GPL/AGPL remote-desktop code is copied, linked or derived; other remote-desktop projects were read as architecture references only.
Connecting an application to it
The engine needs one thing from a consumer: somewhere for the SDP and ICE
candidates to go, and somewhere for the answer and the client's candidates to
come back. See docs/PROTOCOL.md for the authoritative wire contract.
import { EngineClient, resolveEngine } from '@desklink/host';
const engine = resolveEngine(); // prebuilt, or a source build
const client = await EngineClient.start(engine.command, engine.args);
const session = await client.openSession({});
// Everything the engine wants to tell the client arrives here, in order.
client.drainEvents().forEach((event) => myChannel.send(event));
// Bring the client's answer and candidates back.
await client.acceptAnswer(session.sessionId, session.generation, answerSdp);
await client.addCandidate(session.sessionId, session.generation, candidate, sdpMid, sdpMLineIndex);If the engine's control pipe fails, outstanding EngineClient requests reject
with EngineRefused, not an unhandled stream error. Its code preserves the
stream error code (for example EPIPE or ERR_STREAM_WRITE_AFTER_END), or
engine-control-stream when absent; cause holds the original error. The client
kills the unusable engine and refuses later requests as not running. Always
await client.stop() in cleanup: it waits for the child to exit even after a
pipe failure. The regression check is src/engineProcess.pipe.spec.ts.
For cursor-free recordings, open with cursor: 'hidden' and inspect
session.cursor. On Wayland the portal leaves the cursor out of the pixels and
the compositor supplies its position through ext-image-copy-capture-v1;
positions is true once it has. Subscribe to session.cursor events for
source-coordinate x, y, visible, the cursor image's hotspot where
known, and monotonic timestamp_us (the same clock as local frame metadata).
A compositor without that protocol still captures, cursor hidden, and says so
in cursor.limitation. X11 reports positions even on still
frames and already captures root pixels without a cursor. macOS and consumer-encoded
sources return cursor-unavailable for hidden requests. Omitting the option
preserves existing behavior. See the cursor protocol.
myChannel is the application's own authenticated connection, whatever that is.
There is no second identity system, no pairing ceremony and no account: the
engine trusts the consumer's pairing. A paired consumer gets all available
view, input and clipboard access; missing input or clipboard backends fail per
action.
For the application's pairing, identity and signaling transport, see the
BYOKit integration guidance.
If you have no channel of your own
desklink-host bridge is the minimal runnable example for third-party consumers:
it re-serves the same protocol over a WebSocket and serves a reference client
at / so the first run is one command:
desklink-host bridge --listen 127.0.0.1:19400
# engine /path/to/desklink-host
# bridge ws://127.0.0.1:19400/desktop
# token <generated>
# open http://127.0.0.1:19400/?token=<generated>Open that URL and you have a desktop in a browser. examples/reference-client.html
is that page's source: one file, no build step, and the shortest complete
description of the protocol that exists. The bridge operator chooses the capture
source with its flag or environment; clients cannot choose or override it.
The bridge keeps a session in memory for 30 seconds after its last socket
leaves. Bridge.start({ reattachMs }) or CLI --reattach-ms MS changes that
window; 0 restores immediate close. Any authenticated attacher cancels the
deadline and can send session.restart_ice with the held id. This is not an
automatic reconnect or durable session store. A replacing open, explicit close,
engine-side end or bridge shutdown cancels retention; late abandoned opens
still close immediately. A revocation received while detached is replayed once
to the next attacher, not treated as permission to revive a spent lease.
Two things the bridge is honest about: ws:// is plaintext, so keep it on a
private network or put it behind TLS; and the token in the URL is the
pairing credential for full view, input and clipboard, so treat it as a secret.
Stopping the bridge refuses new requests and stops the listener. It shuts down
the engine while client transports remain open so an active control channel can
receive {"kind":"revoked","code":"closed"} (with a reason), then closes the
sockets. Teardown remains bounded if the engine or a client does not respond;
notification failure does not preserve access. Restarting with a new token
also prevents the old pairing from opening future sessions. Applications that
revoke a pairing must explicitly close its live sessions and refuse future
opens; merely dropping its signaling socket now starts the retention window.
Packaging
The engine is a native executable, so the package ships a small JavaScript
launcher and depends on one platform package per supported target, which npm
installs only where its os, cpu and, where specified, libc match.
The platform inventory is defined in release/pack.mjs. It writes both optional
dependencies into the published manifest, pinned to the host version. The pins
are deliberately absent from the source package.json: before publication,
the registry cannot supply the entries needed for a complete frozen workspace
install.
resolveEngine looks for the platform package first and falls back to a source
build, and it never searches PATH: a program that happens to be called
desklink-host is not evidence of which program is about to be given control of
a desktop.
The platform packages must be usable without lifecycle scripts, because
a global install with npm install --global --ignore-scripts must work. Nothing
here downloads or chmods anything at install time; the executable arrives with
the executable bit already set in the tarball.
platformTag() includes the libc (-gnu or -musl) because a glibc binary on a
musl system fails in ways that read as "broken install" rather than "unsupported
platform". Linux variants are published only after their own qualification; macOS arm64
is published as a default-enabled preview, not as qualified desktop support.
See Install and run for current support. Another variant,
such as Linux arm64, needs a real machine to run a real journey first. macOS
is a separate backend, not a separate build.
Building and packing a release
The Linux engine build needs Docker and Node, with no local Rust toolchain or system libraries. The macOS engine build needs the Apple Silicon prerequisites in macOS (preview). Run the following from the repository root after installing workspace dependencies:
packages/desktop-host/release/build-engine.sh # -> dist-desklink/engine-linux-x64-gnu/
npm run build
node packages/desktop-host/release/pack.mjs --engine dist-desklink/engine-linux-x64-gnu
packages/desktop-host/release/check-install.sh # fresh install in a clean containerbuild-engine.sh builds the engine from this checkout in the container that
release/linux-x64-gnu.Dockerfile pins: base images by digest, Debian packages
by snapshot date, the Rust toolchain by version, libvpx by commit, crates by
Cargo.lock. The same source gives the same executable, byte for byte. The
build fails if libvpx ends up dynamically linked, if the executable needs a
glibc newer than 2.36, or if any crate's licence is not permissive
(release/notices.mjs, which also writes THIRD_PARTY_LICENSES.txt).
provenance.json records the source commit, the pinned inputs and the
executable's SHA-256.
npm run build compiles both the host and AXI CLI; packing rejects missing
compiled output before producing tarballs. Run pack.mjs once for each platform,
building and packing the Mac output from the repository root on Apple Silicon:
packages/desktop-host/release/build-engine-macos-arm64.sh
npm run build
node packages/desktop-host/release/pack.mjs --engine dist-desklink/engine-darwin-arm64 --platform darwin-arm64
packages/desktop-host/release/check-install.shThe Mac release build copies libvpx headers and its archive into an archive-only prefix and rejects an executable that still links a libvpx dylib.
The engine-macos-seam CI job also runs this build, packing and clean-install
check on a hosted ARM64 runner. Its darwin-arm64-release artifact contains the
engine output, native tarball, install log and darwin-build-manifest.txt.
The manifest records the candidate source commit/tree and archive hash,
architecture, lockfile and builder hashes, actual Homebrew/static-libvpx inputs,
engine version/capabilities, linked libraries and output hashes. On pull requests
the job checks out the candidate head rather than the synthetic merge commit.
Verify the downloaded artifact's source, version, architecture and hashes before
including it in the release. This job does not qualify interactive capture,
input, clipboard or OS-consent journeys.
Each call keeps the other platform tarball and also packs the host,
React Native receiver and AXI CLI. All package versions must match before
publication. Packing publishes nothing.
The package smoke instead uses npm pack on this checkout's host package as a
registry stand-in, without Docker or a native build. check-install.sh installs
the current platform's host and engine tarballs into an empty project and checks both installed versions. On Linux it uses a clean container with no Rust toolchain, display or /dev/uinput, and checks the typed missing-library error before installing runtime libraries. On macOS it uses a temporary project. Both checks resolve the prebuilt engine, perform the protocol handshake and probe capabilities without opening a capture session.
Publishing is automatic. Merging a version bump to main runs
.github/workflows/release.yml: the test, build, build-pack and per-OS engine
jobs gate a publish job (release/publish-set.sh) that releases every package
whose exact version is not yet on npm, in dependency order — the engine
platform packages first, then @desklink/host, then @desklink/axi, then
@desklink/react-native — all with --provenance. A version lockstep check
fails the workflow when @desklink/host, @desklink/react-native and
@desklink/axi disagree, so a half-bump never reaches publish. The publish job
publishes nothing unless every tarball of the set is present and the registry
answers for each package.
Each package then waits until the registry serves the one before it, within a
30-minute budget that covers npm's processing and CDN lag (shared through
release/readback-budget.json with the manual release/publish.mjs route).
The first failure stops the job with an error naming what is served and what
is not, so a failed run leaves only a prefix of that order published.
The workflow uses trusted publishing (OIDC) and no npm token. Re-runs skip
versions already published.
release/publish.mjs is an older manual upload of the pack.mjs tarballs. It
is not the release route. --dry-run still checks that tarball set:
node packages/desktop-host/release/publish.mjs --dry-run