@pdsjs/git
v0.1.2
Published
Host git repositories natively in an atproto account. The repository's refs and history live in the account's PDS as a `dev.pdsjs.git.repo` record plus chunked blob storage, so a repo rides along in CAR backups, migrates with the account, and is served by
Readme
@pdsjs/git
Host git repositories natively in an atproto account. The repository's refs
and history live in the account's PDS as a dev.pdsjs.git.repo record plus
chunked blob storage, so a repo rides along in CAR backups, migrates with the
account, and is served by any stock PDS with zero server-side changes.
How it works
- One record per repository (collection
dev.pdsjs.git.repo, rkey = repo name). The record holds the ref list and an ordered chain of git bundles. - Each push creates an incremental bundle (
git bundle), splits it into chunks under the PDS blob upload limit, uploads them withuploadBlob, and advances the record with aswapRecordcompare-and-swap. Concurrent pushes lose the swap and are told to fetch and retry. - Fetch and clone read the record, download only the bundles whose heads are missing locally, and unbundle them into the object database.
- Each push also compacts the chain's tail: the smallest run of trailing bundles merges into one, keeping every bundle at least twice the combined size of the bundles after it. The chain length stays logarithmic in the repository size, and a push rewrites only the tail. Once the chain grows past a threshold anyway, the helper repacks everything into a single full bundle, which also drops history no ref reaches. Blobs from dropped bundles lose their record reference and are reaped by the PDS's orphan cleanup.
Usage
Install so git-remote-atproto is on your PATH, then use atproto:// remote
URLs:
npm install -g @pdsjs/git
git clone atproto://alice.example.com/my-project
git remote add origin atproto://alice.example.com/my-project
git push origin mainCloning and fetching are anonymous. Pushing authenticates with an OAuth session or a password.
OAuth login
git-remote-atproto login alice.example.comThe browser opens the PDS's own consent page; approving stores a session
under ~/.config/atproto-git/, and pushes use it from then on. Tokens are
DPoP-bound and refresh silently for two weeks before the login must be
repeated. logout <identifier> deletes the session.
The default grant is the full atproto scope. --scope narrows it to
exactly what git needs, so the stored session can push bundles and nothing
else:
git-remote-atproto login alice.example.com \
--scope 'repo:dev.pdsjs.git.repo blob:*/*'Password fallback
Without a stored session, pushing reads an app password from either:
- the
ATPROTO_GIT_PASSWORDenvironment variable, or - the git credential store, under
protocol=atprotoandhost=<authority>:
printf 'protocol=atproto\nhost=alice.example.com\nusername=alice.example.com\npassword=app-password-here\n\n' \
| git credential approveEnvironment variables
| Variable | Purpose | Default |
| --- | --- | --- |
| ATPROTO_GIT_PASSWORD | Password for pushes and private reads | credential store |
| ATPROTO_GIT_IDENTIFIER | Session identifier | remote URL authority |
| ATPROTO_GIT_SERVICE | PDS base URL, skips handle/DID resolution | resolved from identity |
| ATPROTO_GIT_USER_SERVICE | The caller's own PDS base URL, for private reads by a member | resolved from identity |
| ATPROTO_GIT_PLC_URL | PLC directory for DID resolution | https://plc.directory |
| ATPROTO_GIT_CONFIG_DIR | Where OAuth sessions are stored | ~/.config/atproto-git |
| ATPROTO_GIT_CHUNK_SIZE | Blob chunk size in bytes | 1048576 |
| ATPROTO_GIT_REPACK_THRESHOLD | Bundle count that triggers a full repack | 16 |
| ATPROTO_GIT_COMPACT_FACTOR | Geometric factor for tail compaction, 0 disables it | 2 |
Keep ATPROTO_GIT_CHUNK_SIZE under the PDS blob upload limit (5 MB by
default in pds.js).
Benchmark
bench/push-costs.mjs measures the chain policies against a local PDS:
uploaded bytes per push, worst-case push, chain length, and smart HTTP
read latency with and without ETag revalidation. From the workspace root:
node packages/git/bench/push-costs.mjs --commits 60 --bytes 8192Private repositories
A repository can live in a permissioned space
(proposal 0016,
implemented by @pdsjs/spaces) instead of the public repo. The remote URL
names the space between the authority and the repository:
git clone atproto://alice.example.com/space/com.example.devteam/team/secret-repoThe record format is unchanged; only where it is stored and who can read it differ. Requirements and behavior:
The PDS hosting the repository needs spaces enabled (
spaces: truein the Node adapter,PDS_ENABLE_SPACESon Cloudflare), and the space must exist before the first push (com.atproto.simplespace.createSpace).Every member writes their own copy. The space model permits no other write: a member's push goes to a
dev.pdsjs.git.reporecord in their own repo on their own PDS, under the same space and repository name. The space authority's record is the canonical one. Itsmainis the repository'smainand its default branch is what a fresh clone checks out; every other member's branches appear namespaced by their handle, asorigin/<handle>/<branch>. Merging a member's work means the authority mergesorigin/<handle>/mainlocally and pushes.Space writes need either an OAuth session whose
space:scope covers the repo, or the account password. An app password session deliberately has no space access, matching the scope rules in@pdsjs/core. The narrowest push grant:git-remote-atproto login alice.example.com --scope \ 'space:com.example.devteam?authority=did:plc:...&skey=team&collection=dev.pdsjs.git.repo blob:*/*'The owner reads with the same session. A space member reads through the credential chain: the helper authenticates to the member's own PDS, obtains a delegation token, exchanges it at the space authority for a credential bound to an ephemeral key, and presents that credential with a DPoP proof per request. A member's OAuth login needs
action=readin itsspace:scope, or their account password. SetATPROTO_GIT_IDENTIFIERto the member's handle or DID; with a password,ATPROTO_GIT_USER_SERVICEskips resolving their PDS (an OAuth session already knows it).Pushes use a compare-and-swap the same way public pushes do. The swap field is a pds.js extension to
com.atproto.space.putRecord; on a server without it, concurrent pushes fall back to last-write-wins.The read-only smart HTTP endpoint and the public repository browser serve public records only, so over the network a space repository is reachable through the helper alone. The account page is the owner's surface for it: the repositories section lists private repos with the space they live in, browses their files, manages the space's members, and its New repository dialog creates the space behind a private repo and shows the push commands.
Each member's bundle chain is self-contained: a member's first push uploads their full history rather than an increment over the objects other chains hold. That costs some duplicate storage at personal scale, and buys chains that survive any other member's repack or force push. A fetch replays every member's chain, skipping bundles whose heads are already present, and a member whose host cannot be reached costs their branches, not the fetch.
Branch protection
The account page's repository settings can protect branches, the enforceable half of a forge's rules. A protected branch refuses force pushes and deletion, and a repository holding one refuses deletion outright; the server checks ancestry against the bundle chain before it accepts the record, so a compliant helper is not what the guarantee rests on.
Rules live in a dev.pdsjs.git.config record beside the repo record, under
the same rkey. A separate record because every push rebuilds the repo record
whole; the config survives them all, rides in backups, and migrates with the
account. Enforcement binds the copy on the enforcing PDS: your canonical
branches on your server. A member's copy on their own server answers to that
server, the same way a fork does.
Rules that need a central merge authority, such as required reviews, have no enforcement point in this model and are deliberately absent.
The config record also carries a runner DID, naming the account whose check
records describe this repository. runnerOf reads it. Checks themselves come
from @pdsjs/git-ci, which a PDS needs no part of.
Read-only smart HTTP
A PDS can also serve these repositories to stock git over plain HTTP, so nobody needs the helper installed to read:
git clone https://pds.example.com/git/alice.example.com/my-projectEnable it with experimental: { git: { http: true } } in the Node adapter's
createServer options, or PDS_EXPERIMENTAL_GIT_HTTP = "true" in the
Cloudflare Worker's vars.
The handler
advertises refs straight from the record and answers fetches by merging the
bundle chain's packfiles byte-for-byte (no delta recomputation, no git binary
server-side), which keeps it comfortably inside Cloudflare Worker CPU and
memory budgets for personal-scale repositories.
The same endpoint serves raw files, for a reader that wants the source rather than a page around it:
GET /git/<handle-or-did>/<repo>/raw/<ref>/<path>A path answers with the file's bytes, and a directory answers with its
entries one per line, each directory carrying a trailing slash. That is
enough for a crawler to walk a whole tree through one route. Text is served
as text/plain whatever the extension, so a page held in a repository cannot
run as script on the PDS's own origin.
The endpoint is read-only. Pushes over HTTP are refused with a pointer to
the atproto:// remote. Shallow clones (--depth) are not supported, and
every fetch response is built from whole bundles, so a client several pushes
behind downloads a bit more than a negotiating server would send.
Stability
This whole feature is an experiment, and each layer says so in its own way:
- The package versions at 0.x, so its exports may change in any release.
- The server adapters take the feature under the
experimentaloption group, and the Worker underPDS_EXPERIMENTAL_*vars; both are exempt from semver. - The
dev.pdsjs.git.repolexicon is deliberately unpublished (no_lexiconDNS record), so consumers seevalidationStatus: 'unknown'and know there is no contract yet. Publishing the lexicon is the record format's 1.0 moment.
The record format still evolves conservatively even while experimental, because records outlive software: new fields are optional, existing fields never change meaning, and readers ignore what they do not recognize. The record is fully derivable from the git data itself, so if a breaking change is ever unavoidable, old repositories heal on their next push and a truly incompatible design would ship as a sibling collection rather than a mutation of this one.
Record validation
The lexicons ship as gitRepoLexicon and gitIdentityLexicon. A pds.js instance validates these
records strictly if you seed the resolver:
import { defineLexicon } from '@bigmoves/lexicon';
import { gitIdentityLexicon, gitRepoLexicon } from '@pdsjs/git';
import { LexiconResolver } from '@pdsjs/lexicon-resolver';
const lexiconResolver = new LexiconResolver({
schemas: [defineLexicon(gitRepoLexicon), defineLexicon(gitIdentityLexicon)],
});Unseeded servers accept the records with validationStatus: 'unknown',
matching how the reference PDS treats unresolvable lexicons.
Limits
- A repository under the two-segment URL form is public, like all PDS records and blobs. Privacy comes from the space form above.
- Writing requires the helper; reading works with stock git wherever the PDS enables the smart HTTP endpoint, and with the helper everywhere else.
- A fresh clone downloads the whole bundle chain. Fetches skip bundles whose heads are already present, which approximates negotiation at personal scale.
- SHA-1 repositories only over HTTP; the helper itself is happy with either object format.
