offshoot-fanout
v0.7.0
Published
Propagate a template change down to all its descendant repos and branches via `stem` git remotes, with dry-run and conflict reporting. The maintainer-side companion to offshoot.
Maintainers
Readme
offshoot-fanout
Propagate a template change down to every descendant repo and report merge conflicts — the maintainer-side companion to offshoot.
| | offshoot | offshoot-fanout |
|---|---|---|
| who runs it | the descendant's developer | the template maintainer |
| mechanism | orphan template branch + transform, for descendants that don't share history | stem remote + git merge against shared history |
| direction | pull one update into one project | push one change to every descendant |
offshoot-fanout walks a family of repos that derive from each other via a stem git remote (the parent template), and merges the parent's current branch into each descendant, top to bottom, so a change made anywhere in the hierarchy flows to the leaves in one pass. No push required: each child fetches its parent's local ref, so a freshly-merged intermediate template cascades its new state to its own children immediately.
The unit of work is a node: a (repo, branch) pair, not a repo. That is what makes the destination branch something the tool controls and reports, instead of "whatever the child happened to have checked out".
The parent remote is named
stemby default (an offshoot grows from a stem; fanout flows down the stems).--remote <name>overrides it everywhere. This is deliberately distinct fromoffshoot'stemplatebranch, which uses an orphan-branch transform model — the two mechanisms serve disjoint repo types.
Requires the git binary and Node 20+.
Subcommands
offshoot-fanout status one-command triage: what needs doing (wiring, downstream, upstream)
offshoot-fanout fanout propagate a change down the hierarchy (default)
offshoot-fanout clone rebuild a whole tree on a bare machine, from one repo name
offshoot-fanout drift list descendant commits not yet in their parent (candidate backports)
offshoot-fanout backport cherry-pick a descendant commit up onto an ancestor (its home), optionally cascade
offshoot-fanout discover find repos that share common ancestry; optionally wire remotes; optionally save a registry
offshoot-fanout link set/create the parent remote on repo(s)
offshoot-fanout rename-remote bulk-rename the parent remote (e.g. original -> stem)
offshoot-fanout config show or write a repo's config (which lives on an orphan branch)
offshoot-fanout skills install the agent skill into ~/.agents/skillsRun offshoot-fanout <subcommand> --help for full options. A saved hierarchy lives at
~/.offshoot-stems/<root>.json (write one with discover --save; use it elsewhere with --registry).
Agent skill
This package ships a reconcile-template-tree skill that drives the whole flow (map → triage → find each change's home → land + cascade + backport → resolve conflicts → verify). Install it for your agent:
offshoot-fanout skills # list the skill(s) and whether they are installed
offshoot-fanout skills install # copy into ~/.agents/skills
offshoot-fanout skills install --project # or into ./.agents/skills, beside the codeSkills are copied (not symlinked), so re-run install after upgrading the package.
fanout (default)
From inside the repo you changed (the "spot"):
offshoot-fanout --dry-run # report only; no merge commits, no branch changes
offshoot-fanout # merge down the tree◆ template-svelte@main source
└─ ✗ template-svelte-tailwind@main CONFLICT — conflict in 2 file(s) — aborted
web/src/lib/core/notifications/Notifications.svelte
├─ ⊘ template-svelte-shadcn@main skipped — parent not updated (conflict)
│ └─ ⊘ jolly-roger@main skipped — parent not updated (skipped)
└─ …
– jolly-roger-work is a linked worktree of jolly-roger (same repository) — not a nodeEvery line names the node as repo@branch, because the destination branch is the thing that used to be invisible.
Options: --source, --base-dir, --repos, --registry, --branch (global override), --remote (default stem), --config-branch (default offshoot), --no-config, --verify, --ignore (repeatable), --dry-run, --leave-conflicts, --no-color. Exit code is non-zero if any node conflicts/errors/has a dirty tree/fails --verify.
Nodes: (repo, branch)
Two kinds of edge feed a node:
- cross-repo: the
stemremote match (a child'sstemURL normalizes to a parent'soriginURL), from the parent's primary branch into each of the child's root branches. Needs a fetch. - in-repo: a branch whose stem is another branch in the same repo.
variant/fullderives frommainexactly the way a child repo derives from its parent, and no fetch is involved. A branch can name several stems, which makes it an integration node (see below).
A repo with no config is a single node: main if that branch exists, otherwise the repo's current HEAD branch (which the report says out loud). So a tree where every repo only uses main behaves exactly as before. --branch <name> is a global override: every repo becomes one node at that branch, and branches config is ignored.
A topological sweep over the nodes gives the correct order for free: shadcn@main → jolly-roger@main → jolly-roger@variant/full. A failed node still marks its descendants skipped.
Integration nodes (several stems)
A branch that combines independent extensions names them all:
"extended/complete": {"stem": ["extended/hosted-account", "extended/local-signer"]}The alternative, chaining them (hosted-account stemming from local-signer), says something different and usually wrong: that one extension is built on the other, so each inherits the previous one's work. Several stems keeps them independent and combines them in one place.
The node graph is then a DAG rather than a tree, which changes three things:
- Order. An integration node is processed only once every stem is done, so it never merges one side's stale state. Ordering is a topological sweep, not a walk.
- Merging. Its stems are merged one at a time, in the order the config lists them, so each gets its own conflict (and its own chance at
--leave-conflicts). - Reporting. It is rendered in full under its first stem and cross-linked under the others, and counted once:
├─ ✓ jolly-roger@extended/hosted-account merged — merged 3 file(s) into extended/hosted-account
│ └─ ✓ jolly-roger@extended/complete merged — merged 6 file(s) into extended/complete from extended/hosted-account and extended/local-signer
└─ ✓ jolly-roger@extended/local-signer merged — merged 3 file(s) into extended/local-signer
↳ jolly-roger@extended/complete (also merges from here; shown under jolly-roger@extended/hosted-account)If any stem fails, the integration node is skipped rather than merged from the stems that did work. If a later stem conflicts, the earlier merges are already committed and are kept: the state is honest, re-running continues from there, and the message says both what landed and what blocked (conflict in 2 file(s) merging extended/local-signer — aborted; extended/hosted-account already merged).
One limit worth knowing: --dry-run predicts each stem against the branch as it stands, because git merge-tree needs a commit and there is no commit for "the branch after stem 1 merged". For a single-stem node that is exact; for an integration node the stems after the first are an approximation, and the message says so.
Merging into a branch that is not checked out
git checkout is never used: it mutates your working state and dies on a dirty tree. Instead:
- if the target branch is already checked out (in the repo's main worktree, or in one of its linked worktrees), the merge happens in place, there, exactly as before, so you find the conflict where you expect it and
--leave-conflictsbehaves as documented; - otherwise the merge happens in a temporary linked worktree (
git worktree add), removed afterwards; - on conflict inside a temporary worktree,
--leave-conflictskeeps it and prints its path (otherwise the flag would be meaningless for that node). A failed--verifykeeps it for the same reason; - a dirty tree only blocks the branch that is actually checked out, not the whole repo. A dirty branch is reported as
dirtyby--dry-runandstatustoo, so a dry-run never promises a merge that the real run then refuses.
✗ jolly-roger@main CONFLICT — conflict in 1 file(s) — left in a temporary worktree: /tmp/offshoot-fanout-worktrees/jolly-roger-main-6f1a2b3c
left in place for inspection: /tmp/offshoot-fanout-worktrees/jolly-roger-main-6f1a2b3c
when done: git -C <repo> worktree remove --force /tmp/offshoot-fanout-worktrees/jolly-roger-main-6f1a2b3cResolve it there, commit, then re-run fanout; the node reports up to date and the cascade continues past it. Kept worktrees are never cleaned up behind your back, so remove them yourself when done (the command is printed, and git worktree list in the repo shows any you have forgotten).
Config on an orphan branch
A template should not have to carry an offshoot-specific file in its working tree, so config lives on an orphan branch (default offshoot) in fanout.config.json, and is read without ever checking it out:
git show offshoot:fanout.config.json
git show origin/offshoot:fanout.config.json # fallback on a fresh clone{
"branches": {
"main": {},
"variant/full": {"stem": "main"}
},
"verify": "pnpm install && pnpm --filter ./web check"
}branchesis opt-in: when present, only the listed branches participate. That is what keepswork,variant/offlineandwebsiteout of the cascade without naming them. A branch with nostemis a root node fed by the cross-repostemremote.verifyis a command run in a merged node. It only ever runs behind--verify, never automatically, because it means executing a command string read from a git ref. Pass/fail is reported per node in the same tree output.
Absent config means today's defaults, so a repo that matches the defaults stays completely free of offshoot references. offshoot is an ordinary word, so a branch of that name holding no fanout.config.json is treated as a name collision (defaults apply, and the report says so) rather than an error. A config file that is there and does not parse is a real error: the node fails and its descendants are skipped, without any merge being attempted.
offshoot-fanout config show --repo ./jolly-roger # resolved config + where it came from
offshoot-fanout config set --repo ./jolly-roger --file ./fanout.config.json
offshoot-fanout fanout --config-branch offshoot/fanout # a different branch
offshoot-fanout fanout --no-config # ignore config branches entirelyconfig set writes with plumbing (hash-object -w, mktree, commit-tree, update-ref), so your working tree, index and current branch are never touched, and the orphan branch is created if absent.
Why a branch and not a file at the root. An in-tree
.offshoot-fanout.jsonin the root template would cascade into every descendant and conflict at every level on every change, because the file's entire content is per-repo (each repo's branch list, each repo's verify command). A file that must differ at every level, living on a branch that merges to every level, is a permanent conflict generator. An orphan branch has no merge base with anything, so it never propagates and never conflicts.
Naming. The default branch name is flat:
offshoot. Git cannot hold both a branch namedoffshootand anyoffshoot/*branch (a ref file cannot also be a directory), so pick one convention: keep the flat name, or use a nested config branch (--config-branch offshoot/fanout) and never create the flat one.
Ignoring things
Linked worktrees are always skipped, not as an option. A linked worktree (git worktree add) is the same repository: it has a .git, and it inherits its repo's stem remote, so remotes alone can never tell them apart. It is detected by comparing git rev-parse --git-dir with --git-common-dir, never becomes a node, and is mentioned once as a worktree of its repo:
– jolly-roger-work is a linked worktree of jolly-roger (same repository) — not a nodeExplicit ignores are for repos that exist on disk but must stay out of the tree (a deprecated template whose folder has not been deleted yet). This is maintainer-local state, so it lives in the CLI and the registry, never in a repo's config branch:
offshoot-fanout status --ignore template-onchain-app --ignore ./old-thing
offshoot-fanout discover . --save --root ./template-svelte --ignore template-onchain-app--ignore takes a repo name or a path and is repeatable; discover --save merges it into the registry's ignore array, preserving any existing entries rather than clobbering them. Ignored and skipped nodes stay visible in the report (– deprecated@main ignored), never silently absent.
verify
None of a real template tree's repos necessarily has a root check script, and rediscovering the right command per repo by hand is exactly the toil this removes. Declare it once per repo on its config branch, then opt in:
offshoot-fanout --verify✓ jolly-roger@main merged — merged 12 file(s) into main in a temporary worktree
✓ verify passed
└─ ✓ jolly-roger@variant/full merged — merged 12 file(s) into variant/full in a temporary worktree
✗ verify FAILED (exit 1): 5 errors, 0 warningsA failing verify does not un-merge anything and does not block the cascade (the merge already happened); it is reported per node and makes the exit code non-zero. When the node was merged in a temporary worktree, that worktree is kept and its path printed, so there is somewhere to go and reproduce the failure.
drift
Find descendant commits that aren't yet in their parent, the candidate backports. Reported per node, against that node's stem: the parent repo's primary branch for a root branch, or the sibling branch named by its stem for an in-repo branch. An integration node is compared against all of its stems at once (git log <branch> --not <stem1> <stem2>), so it is never reported as drifting from what it just merged.
offshoot-fanout drift ./dev-folder --remote stem▲ conquest-website-2@main (95 ahead of template-svelte-tailwind-blog@main)
e76298ef notifications: adopt the generic service + service-worker DI from jolly-roger
a434315d core: sync framework-agnostic drift from jolly-roger
▲ template-svelte-tailwind@main (22 ahead of template-svelte@main)
…Review the list, then backport the commits that genuinely belong upstream. Read-only (fetches objects).
backport
Move a descendant commit up to its home ancestor: cherry-pick it onto the ancestor, then (with
--cascade) fan it back out to every descendant.
offshoot-fanout backport e76298ef --from ./conquest-website-2 --to ./template-svelte --cascade--to defaults to the --from repo's immediate stem parent; pass a higher ancestor when the change's
home is further up the tree. --dry-run shows the commit without applying; --leave-conflicts keeps
a conflicted cherry-pick in progress for manual resolution (same pattern as fanout).
The full loop: drift spots relevant descendant changes → backport --to <home> lands one upstream →
fanout (or backport --cascade) spreads it back down. The judgement of which commit belongs where
stays with you; the tool does the mechanics.
discover
Find repos that share common git ancestry — scans a folder, groups repos by shared commit history, and proposes a parent→child tree per family (root = fewest commits, tie-broken by oldest HEAD). Direction can't be proven once both sides diverged past the fork, so the tree is a proposal — review before applying.
offshoot-fanout discover ./some-folder --remote stem
offshoot-fanout discover ./some-folder --add-remotes --root ./template-svelte # wire the unwired descendants of that root
offshoot-fanout discover ./some-folder --add-remotes --yes --dry-run # preview what would be added
offshoot-fanout discover ./some-folder --save --root ./template-svelte # persist that hierarchy to a registry--add-remotes adds the stem remote to unwired repos (pointing at their detected parent's origin URL). It needs --root <repo> (anchor direction; only that root's subtree is wired) or --yes (accept the proposed tree). --dry-run shows what would be added without adding. --save writes a registry (see below); scoped to one hierarchy with --root.
clone: rebuilding a tree from its host
Given only the root repo, clone every repo of its tree and wire each one's stem remote:
offshoot-fanout clone wighawag/template-svelte --dir ~/dev --dry-run
offshoot-fanout clone wighawag/template-svelte --dir ~/devNo registry, no manifest, no local state. It works in two steps, and they answer two different questions:
- Who is in the family? Every descendant contains the family's root commit, so one commit search on the host lists them all, including repos under owners you had forgotten. (Commit search only indexes default branches, which is exactly why the root commit is the right probe: it reaches
maineverywhere, while the branch an edge was cut from may not.) - Who is whose parent? Each repo's own
stemfield, read straight off its config branch without cloning.
The second step cannot be replaced by the first. Direction is not recoverable from shared history: run the inference against the real ten-repo tree and three of ten edges come out wrong, including jolly-roger placed under its own grandchild reveal-or-die, because that descendant happens to have fewer commits than its ancestor. A tree assembled that way would fan out backwards. So clone uses only stated edges, and reports the ones it cannot read.
A repo that shares the root commit but has no stem field is discarded, and listed with the reason. That is the second thing the field buys: it marks which repos are actually maintained as part of the tree, so an old experiment, a bug reproduction and a stranger's copy stay out on purpose rather than by luck.
Re-running is safe: an existing clone with the right origin is kept and only re-wired, and a stem remote already pointing elsewhere is never clobbered.
The clone is runnable, not just present
git clone leaves one local branch. Everything else is a remote-tracking ref, which the cascade can neither check out nor merge into. Measured on this tree, a fresh clone followed immediately by the tool's own primary verb reported
✗ template-commit-reveal@with/pixi-js CONFLICT — conflict in 0 file(s) merging main
⊘ template-commit-reveal@with/all skipped — parent not updated (conflict)
⊘ reveal-or-die@main skipped — parent not updated (skipped)with nothing whatsoever wrong with the tree. (That message was itself a second bug, since fixed: git merge-tree exits 1 both for a real conflict and for "not something we can merge", so an absent branch was rendered as a zero-file conflict. It now names the branch and how to create it.) So clone also creates a local tracking branch for every branch a repo's own fanout.config.json names, using the branches object it has already parsed to find the tree's edges. It creates the config branch too, because without a local offshoot the next config stem starts a second, parentless config history and prints a push command origin rejects.
A named branch that origin does not have is reported, and the command exits non-zero. The tree comes back visibly thin rather than invisibly thin. An existing clone is fetched before this comparison, so "not on origin" is a fact and not a claim about a stale ref cache; --dry-run writes nothing and labels its answer provisional instead. The config branch is the one branch this will move, and only by fast-forward: it holds no work, it is never checked out, and a stale copy silently changes which nodes exist, because a local offshoot outranks origin/offshoot when the config is read. A diverged one is reported, never resolved. --no-branches skips all of it.
The exit code follows one rule: non-zero when what came back is not what was asked for. A repo that failed to clone, a repo skipped because its directory was taken, a config that could not be read, a branch a config names that is not here, or a search the host itself truncated. A missing credential is the exception and stays zero, because --require-auth already exists to make that call.
What is not created: any branch no config names. branches exists precisely to keep scratch branches out of the cascade without naming them, so work, old and v1 stay remote-tracking refs. tooling is the same case and is worth stating because it looks like an exception: it is a maintainer's local cache of the stem's orphan tooling branch, deliberately in no config's branches and unmergeable by construction, so materialising it would mean inventing an edge no config states. Fetch it when you want it:
git fetch stem tooling:toolingThat is also the reason the stem remote is wired but not fetched: the cascade fetches a cross-repo edge from the parent's sibling clone, not from that remote, so a fetch per repo would buy nothing.
You can ask for any node, not just the true root, and get the subtree beneath it. The rest of the family shares the same root commit and so is discovered too, but it is listed rather than cloned:
offshoot-fanout clone wighawag/template-commit-reveal # 2 repos, not the whole tree of 11Clones use ssh by default (or gh's git_protocol when that says https, or --prefer-https). That is not a style preference: discovery is authenticated and therefore sees private members, and an https clone of a private repo fails asking for a username a script cannot answer. ssh uses the key you already push with, so whatever the discovery step can see, the clone step can fetch.
Authentication is not optional for a tree with private members
An unauthenticated commit search cannot see private repos at all, and does not say so. It returns a smaller answer that looks complete. On the tree this was built against, that is 12 repos found instead of 17.
So a credential is taken from GITHUB_TOKEN, then GH_TOKEN, then gh auth token — if you are logged into the gh CLI, private members are included with no setup at all. The report always names the source it used, and says loudly when it had none:
offshoot-fanout clone wighawag/template-svelte --dry-run
# authenticated via gh
offshoot-fanout clone wighawag/template-svelte --require-auth # missing credential is fatalUse --require-auth in provisioning scripts. A machine that silently restores two thirds of a tree is worse than one that refuses to start.
Publishing the parent: the stem field
{
"stem": "github:wighawag/template-svelte-tailwind",
"branches": {"main": {}, "with/all": {"stem": "main"}}
}Top-level stem is the parent repo; branches.<name>.stem is a branch in this repo. null declares this repo the root of its tree, which is not the same as saying nothing.
It is provider:owner/name, not a URL, so the same tree cloned over ssh on one machine and https on another still describes one relationship. (A URL is still accepted, and required for a self-hosted host.) This is the spelling .offshoot.json already uses for the non-fork flow, so both halves of the tool name a parent the same way.
The stem remote still wins for merging. It is what git actually fetches, and pointing it at a sibling checkout on disk is the normal way to work on a tree locally, so a published id must never silently retarget a merge. The config is the portable truth and the fallback when no remote exists. Absence stays valid indefinitely.
When the two disagree it is reported, never silently resolved — a local-path remote is compared through its own origin first, so a local checkout of the right parent is not mistaken for drift:
offshoot-fanout config show --repo ./jolly-roger
# parent repo:
# github:wighawag/template-svelte-shadcn (config; `stem` remote agrees)Migrate a tree whose edges exist only on one machine by publishing what each stem remote already knows:
for repo in */; do offshoot-fanout config stem --from-remote --repo "$repo"; done
offshoot-fanout config stem --root --repo ./template-svelte # the top of the tree
# then push each config branch:
git -C ./jolly-roger push origin offshootRegistry
The registry is local state and stays that way: filesystem paths and your ignore list, neither of which belongs in a config other people read. Topology now comes from the stem field.
discover --save persists each wired hierarchy (defined by the real stem remotes, not shared commits) to ~/.offshoot-stems/<root>.json, one file per hierarchy, so several independent trees coexist. Each entry records the repo's originUrl, its current stemUrl (null = unwired), and its real stem parent; an optional ignore: [] array carries the maintainer-local exclusions. The other commands then take --registry <file> to operate off the saved tree instead of scanning:
offshoot-fanout fanout --registry ~/.offshoot-stems/template-svelte.json --dry-run
offshoot-fanout drift --registry ~/.offshoot-stems/template-svelte.json
offshoot-fanout backport <sha> --from ./jolly-roger --registry ~/.offshoot-stems/template-svelte.json --cascadelink
Manually set/create the parent remote on repo(s) — for the one-off cases discovery can't handle:
offshoot-fanout link [email protected]:wighawag/template-svelte.git --to ./my-app --remote stemAdds (or repoints) stem on each --to repo (default: cwd).
rename-remote
Migrate the parent-remote name across a folder, e.g. original → stem:
offshoot-fanout rename-remote original stem ./dev-folder --dry-run # preview
offshoot-fanout rename-remote original stem ./dev-folder # do itRenames original→stem in every repo that has original; repos without it are skipped; a stem that already exists with a different URL is left untouched (not clobbered). --dry-run is read-only (it reports "would rename" and performs nothing).
Library
import {propagate, formatReport, discoverAncestry} from 'offshoot-fanout';
const result = await propagate({sourcePath: '/path/to/template-svelte', dryRun: true});
console.log(formatReport(result));
const families = discoverAncestry('/path/to/dev-folder'); // FamilyTree[]How fanout works
- Discover: scan the source's parent dir for git repos, read each one's
originandstemremotes. Linked worktrees are excluded here. - Plan each repo: read its config branch with
git show(never checking it out) to get its participating branches and their in-repo stems; with no config, one node atmain(or the checked-out branch). - Build the graph: match each repo's
stemURL (normalized across SSH/HTTPS/.git/case) to another repo'soriginURL to find its parent, then wire parent-primary → child-root edges and in-repo branch edges. The result is a DAG, since a branch may name several stems. - Cascade (topological sweep): each node waits for every stem, then fetches its parent's local clone (freshest post-merge state) and merges the pinned commit with
git merge --no-ff, in place or in a temporary worktree. On conflict, abort (or leave with--leave-conflicts) and report the files. Anything the sweep cannot reach is a stem cycle, and is reported as such rather than dropped. - A failed node's descendants are marked
skipped(still visited/rendered) rather than merged against stale state.
--dry-run fetches objects, then computes the merge in memory with git merge-tree --write-tree: no branch, index, working tree or worktree is touched. (On git older than 2.38 it falls back to git merge --no-commit --no-ff + git merge --abort.) One consequence worth knowing: a dry-run evaluates each node against its parent's current state, so a node below one that would merge can report up to date where a real run would merge.
Adding a repo to the hierarchy
git remote add stem <parent-template-url>The repo must share at least one common ancestor commit with its parent (created by cloning/forking). If it was scaffolded with placeholders and shares no history, that's offshoot's job (orphan branch + transform), not this one.
