Document git-refs Checkpoint Backend Architecture

I've reviewed the ref-backend code and all the documentation in both locations. Here's my assessment. (Note: ../doc doesn't exist — I read ../docs, the Mintlify public site, which is what I assume you meant. Say if you meant something else.)

What exists today

The git-refs backend is fully implemented in this checkout (checkpoint/refs_store.go, persistent_refs.go, pushqueue.go, refs_naming.go, registry.go, routing_store.go), but the documentation lags the code badly. Coverage is split across two audiences:

Internal architecture docs (cli_experiments/docs/architecture/)

• sessions-and-checkpoints.md — the only place with real ref-backend content. It covers ULID vs 12-hex IDs, kind-based read routing (kindRoutingStore), and generation well. But its two structural sections are still git-branch-only:
  • The Storage table (lines 143–147) lists shadow branch + entire/checkpoints/v1 and has no row for refs/entire/checkpoints/<shard>/<id>.
  • The Package Structure block (lines 466–483) predates the refs files entirely — no refs_store.go, pushqueue.go, registry.go, routing_store.go.
• checkpoint-scenarios.md — zero ref-backend mentions. No condensation/push scenario is documented for git-refs.
• The best prose on the backend model actually lives in code comments (registry.go has excellent doc comments on gitBacked, primary/mirror taxonomy) and in the design docs at devenv root (checkpoint-approaches.md, checkpoint-base-design.md), not in the arch docs.

Public docs (../docs/, Mintlify)

• guides/checkpoints/overview.mdx — says every checkpoint has a "12-character ID" (now wrong — ULIDs are 26 chars under git-refs) and points only at the entire/checkpoints/v1 branch.
• glossary.mdx — has "Checkpoints branch" and "Shadow branch" entries but nothing for git-refs, checkpoint backend, primary/mirror, or ULID. Its shadow-branch naming (entire/<sessionID>-<worktreeID>) also contradicts the arch doc (entire/<commit[:7]>-<worktreeHash[:6]>).

The core gap

There is no document that explains the ref-based backend as a system — the pluggable primary/mirror taxonomy, the refs/entire/checkpoints/<shard>/<id> layout, asymmetric sharding (hex first-2, ULID last-2), the per-ref non-force push with fetch-replay recovery, the flock push-discovery queue, config topology (checkpoints.{primary,mirrors} + ENTIRE_CHECKPOINTS_PRIMARY), and the branch→refs migration/coexistence story. A reader has to reconstruct it from four scattered sources.

Proposed documents

New — internal (highest priority):

1. docs/architecture/ref-checkpoint-backend.md — the missing explanation doc. Backend taxonomy & gitBacked capability → ref layout & sharding → ID formats (why ULID under refs) → push/fetch model (queue, fast-forward-only, fetch+replay recovery) → config topology & rollout states → read routing & coexistence → migration. This is the one I'd write first; most of it can be lifted from registry.go comments and the settled design.

Edits — internal: 2. sessions-and-checkpoints.md — add a refs/entire/checkpoints/ row to the Storage table, refresh Package Structure with the refs files, and add a one-line pointer to the new doc. 3. checkpoint-scenarios.md — add a git-refs condensation + push/recovery scenario alongside the existing branch ones.

Edits — public: 4. overview.mdx — fix "12-character ID" → "short ID" (or explain both formats). 5. glossary.mdx — add entries for checkpoint backend / git-refs / primary+mirror; reconcile the shadow-branch naming with the arch doc.

I'd recommend starting with #1 (the new architecture doc) since it's the linchpin the others reference, then the two quick arch-doc edits.

Want me to write ref-checkpoint-backend.md now? I can draft it grounded in the current code and the settled design, then do the small edits to the surrounding docs.

Doc committed. Now a second focused commit to make it discoverable — cross-links from the sibling architecture doc and the CLAUDE.md deep-dive reference list.

Done. On branch soph/ref-backend-docs, two focused commits:

1. 5f8eca85b — the new docs/architecture/ref-checkpoint-backend.md (now soft-wrapped, one line per paragraph). It covers primary/mirror taxonomy + gitBacked, the refs/entire/checkpoints/<shard>/<id> layout and last-two-char sharding, the per-checkpoint write path, the push-discovery queue and fast-forward-only push with fetch+replay recovery, on-demand ref fetch, kind-based read routing/coexistence, config + env override + rollout states, checkpoint-version formats, key files, and known limitations.
2. add234df7 — cross-links so it's discoverable: from sessions-and-checkpoints.md's overview and the checkpoint Deep-Dive Reference list in CLAUDE.md (AGENTS.md is a symlink, so agents get it too).

Everything is grounded in the current code, not the older design notes — I verified against refs_naming.go, refs_store.go, pushqueue.go, routing_store.go, registry.go, checkpointpolicy/format.go, and settings/checkpoints.go. Notably, sharding is last-two-chars for both ID formats, which differs from the earlier asymmetric plan in the design notes.

These are docs-only changes (no mise run check needed for Go), but note two things:

• The sessions-and-checkpoints.md file is otherwise hard-wrapped; my inserted paragraph is a single soft-wrapped line, so it's stylistically different from the rest of that file. Say the word if you'd rather I match the file's wrapping there.
• Next up from the original plan, when you want them: the git-branch-centric Storage table + Package Structure in sessions-and-checkpoints.md (edit #2), a git-refs scenario in checkpoint-scenarios.md (#3), and the public-docs fixes (#4/#5 — the "12-character ID" line and the missing glossary entries).

Want me to proceed with any of those?

Confirmed by the code — here's the explanation.

What the bullet means

The scenario it's protecting against: "downgrade safety." Suppose you roll out git-refs as the primary backend. Now imagine one of these:

• A teammate (or CI, or a server-side reader) is still on an older CLI that only understands the entire/checkpoints/v1 branch and has never heard of refs/entire/checkpoints/*.
• You hit a bug in the refs backend and need to roll back to git-branch.

In either case you'd want the checkpoints to also exist in the old v1 format, so nothing that only reads v1 goes blind. That redundancy is what "downgrade safety" buys.

How you'd normally get it: run git-branch as a mirror. That's the Parallel rollout state:

Mirrors receive best-effort write fan-out, so every condensation writes the checkpoint into both the per-checkpoint ref (primary) and the v1 branch (mirror). So far so good.

The actual gap

The catch is local vs. remote. The mirror fan-out writes v1 into the local object store, but the pre-push path never pushes it. Look at PrePush (strategy/manual_commit_push.go:49-54):

When the primary is git-refs, PrePush pushes the per-checkpoint refs from the push queue and returns. It never reaches the code below (line 121) that pushes the v1 branch. So with git-branch as a mirror today, you get: refs pushed to the remote, v1 written locally but not pushed.

The result: a remote reader on an old CLI would find the v1 branch stale (or empty), because its updates never left the machine that made them. The mirror gives you local downgrade safety but not remote/shared downgrade safety.

Why it's phrased as "a rollout concern, not automatic behavior"

The bullet is making two points:

1. The refs store never touches v1 itself. By design it only manages refs/entire/checkpoints/*. Keeping v1 populated at all is a configuration choice (add the git-branch mirror), not something the refs store does on its own.
2. Even with that config, the remote-push half is deferred — "a later step." Closing it means teaching the git-refs pre-push path to also push the mirror's v1 branch (not just the refs), so a downgrade/old-reader story works across machines, not only on the box that wrote the checkpoints.

So the limitation is really: full downgrade safety needs both (a) the git-branch mirror configured and (b) that mirror's v1 ref actually pushed at pre-push time — and only (a) exists right now.

If that reading matches your intent, I can tighten the bullet in the doc to spell out the local-written-but-not-pushed distinction, which is the part most likely to be misread. Want me to?

Your model is correct, and I mostly agree. Let me lay out why, and then the one thing I'd nail down before committing to it.

Your reasoning holds up against the code

The read routing already delivers exactly "write once, read from both":

• A modern CLI on git-refs primary reads hex via refs → branch fallback and ULID via refs (kindRoutingStore.readOrder), and List unions both. So it sees everything regardless of which backend a checkpoint lives in.
• An old CLI keeps writing hex to v1 (it doesn't know the checkpoints config exists), and modern CLIs still read those via the branch fallback. So "someone on an old CLI can write and everyone modern can still read" is accurate.
• An old CLI can't read ULID/refs checkpoints — and you're right that this is a feature, not a regression. It fails closed, and the checkpoint_min_version policy already turns that into an explicit "upgrade" nudge (warns, never blocks git). That's a much better contract than silently half-working.

Mirroring is the wrong tool here — and it under-delivers today

The mirror's only real job in this migration was downgrade safety, and as currently built it doesn't even provide it: under a git-refs primary the mirror writes v1 locally but never pushes it (the early return in PrePush). So the remote never gets v1 anyway. Advertising a "run git-branch as a mirror" knob that promises redundancy it doesn't deliver is worse than not having it.

Dropping mirror-of-branch from the branch→refs story also removes two liabilities, not just one:

1. The deferred "push the mirror's v1 ref" work disappears entirely.
2. Known-limitation #4 (the ULID⇒branch write guard) collapses into a trivial invariant. That guard only exists because git-branch could legitimately receive ULIDs as a mirror. If git-branch is never a mirror of refs, then a ULID reaching the branch store is unambiguously a bug you can just reject — no topology-role awareness needed.

So the migration becomes clean: flip the primary, let read routing be the compatibility layer, let the policy be the upgrade forcing-function. No dual-write.

The one thing I'd verify first: who else reads v1?

Your "everyone can still read" holds for readers that implement the routing — the CLI does. The question I can't answer from this repo is whether the server-side reader (entire backend / entire.io web UI / EntireDB) reads the pushed entire/checkpoints/v1 branch, and whether it understands refs/entire/checkpoints/* yet.

That's where the actual remaining value of keeping v1 on the remote lives — not old CLIs, but the product surface. Two cases:

• If the server already reads (or will read, before rollout) the per-checkpoint refs → your plan is straightforwardly right. Don't advertise mirroring; branch→refs is a primary switch plus read routing.
• If the server only reads the v1 branch → going refs-only on the remote means checkpoints stop appearing in the web product during migration. In that case you still need v1 reaching the remote somehow — but the honest fix is "upgrade the server reader," not "quietly dual-write via a mirror that doesn't push."

Either way your instinct stands; the server reader just decides whether refs-only can be the remote story now or after a server change.

Want me to check the ../entire (and ../entire.io) side to see how checkpoints are read server-side? That would settle it. And regardless of the answer, I'd rewrite the two doc bullets to drop the mirror-as-downgrade-safety framing and state the read-routing-plus-policy compatibility model instead — shall I do both?

Done. The doc now reflects your decision. Branch soph/ref-backend-docs has three commits:

1. 5f8eca85b — the new architecture doc
2. add234df7 — cross-links for discoverability
3. 8a9196c72 — reframe the migration as read-routing + policy, not mirroring

What changed in that last commit:

• Taxonomy note — the one-of-each-type rule still exists, but it no longer sells "git-branch as a mirror of git-refs" as the rollout; compatibility is read routing + version policy across all readers (CLI, entire.io, entire-api). Mirroring stays documented only as a general mechanism (e.g. non-git-backed targets).
• Config example — refs-only, no mirrors block.
• Rollout table — dropped the "Parallel" (dual-write) state; now just Default → Refs-only, explicitly "a primary flip, not a dual-write phase."
• Migration section — spells out the mixed-fleet story you described: everyone routes refs-first with branch fallback (server included), old clients keep writing hex to v1 that everyone modern still reads, and an old client failing to read the new format is the intended upgrade forcing-function, not something to paper over.
• Known limitations — removed the "v1 mirror push for downgrade safety" bullet entirely; the ULID-on-branch bullet is now reframed as a simple write-path reject (since git-branch is never a mirror of refs), not a topology-role-aware guard.

Remaining from the original plan, whenever you want them: the git-branch-centric Storage table + Package Structure in sessions-and-checkpoints.md, a git-refs scenario in checkpoint-scenarios.md, and the public-docs fixes (the "12-character ID" line and missing glossary entries). Want me to pick those up, or push this branch / open a PR first?

Everything checks out — no stale file references, and both intra-doc anchor links resolve to real headings.

Done — all internal-repo (cli_experiments) doc work

Branch soph/ref-backend-docs, five commits off main:

All grounded in the current code, and the doc set now tells one consistent story: read routing (on CLI and server) plus the version policy is the compatibility mechanism, not dual-writing.

What's left (separate repo)

The only remaining items from the original plan are in ../docs (the Mintlify public site, a different git repo), so they're outside "this repo":

• guides/checkpoints/overview.mdx — the stale "12-character ID" line
• glossary.mdx — missing git-refs / backend / primary-mirror entries and the shadow-branch naming mismatch

Want me to push soph/ref-backend-docs and/or open a PR now? And separately, should I pick up the ../docs public-site fixes in that repo?

PR opened: https://github.com/entireio/cli/pull/1693

• Branch soph/ref-backend-docs pushed and tracking origin.
• 5 commits, docs-only, targeting main.
• The PR body captures the new doc's scope, the migration-framing decision from our discussion (read routing + policy, not mirroring), the supporting edits, and explicitly notes the public-site (entire/docs) fixes are a separate-repo follow-up.

Want me to pick up those ../docs public-site fixes next (the "12-character ID" line and the missing glossary entries)?

Fixed and pushed to PR #1693.

The paragraph now reads:

Both build the exact same checkpoint subtree; they differ only in the base path they write it at and in where the result is committed. The git-branch store writes each checkpoint under its shard prefix <id[:2]>/<id[2:]>/ inside the single v1 tree, so many checkpoints share one tree. The git-refs store writes with no prefix (an empty base path), so the checkpoint subtree is the root of that checkpoint's own commit tree...

This also corrected an inaccuracy: basePath is a per-write argument to applySessionWrite/applyTranscriptBackfill/etc., not something the treeWriter is "anchored at" — git-branch passes CheckpointID.Path()+"/", git-refs passes "". And nice detail confirmed by the push output — the CLI itself is running the git-refs backend here, so it pushed the checkpoint ref for this very commit.