Record turn-end work in session state instead of a shadow branch

Commit

Soph4h ago

Every agent turn used to commit a full tree snapshot (code plus a redacted copy of the metadata directory) to a per-base, per-worktree shadow branch entire/<commit>-<worktree>. That is where the storage growth came from, and the branch was the source of truth for everything downstream: commit linking, carry-forward, condensation, lifecycle gates, pending rows and explain. Deleting it at the wrong moment (pre-push and post-condense cleanup) made uncondensed work unreachable.

Turn end now writes no git objects. SaveStep records the step in session state: StepCount, FilesTouched, and TouchedFileHashes, the blob hash of each touched regular file as the agent left it, computed with git hash-object so clean filters apply as git add would ("" marks a recorded deletion; an absent entry falls back to name matching). Hashing runs outside the session lock, and a step path that is neither present nor a recorded deletion is dropped as a transcript-parsing phantom. The sanitized transcript copy under .entire/metadata/<session>/ stays as before and is what condensation reads when the live transcript is gone.

Consumers move onto that state:

  • Commit linking compares the committed or staged blob of a new file with its recorded hash; the hunk-level comparison is gone.
  • Carry-forward keeps a path while its recorded content is not what was committed and the worktree still differs; remaining files stay in state (StepCount=1, transcript offsets reset) instead of being rewritten onto a new shadow commit.
  • Lifecycle gates (condense-by-ID, eager and session-end condensation, orphan deletion, the zombie sweep, doctor) ask State.HasPendingWork instead of whether a shadow ref exists. HasPendingWork ignores FullyCondensed; the sweep, IsCondensableEndedSession and doctor's ENDED classification still exclude fully condensed sessions, so a dead session is not re-condensed into IDLE on every commit.
  • BaseCommit simply follows HEAD (syncBaseCommitToHead, rewrite remapping); there is no branch to migrate.
  • Task-record completion, per-tool hooks and Codex child-file merges add paths through MergeUnhashedFilesTouched, which drops any older recorded hash so a rewritten file is not judged against it.

Removed with the writer: the ephemeral store and shadow ref locking, shadow migration, post-condense and pre-push shadow cleanup, SaveTaskStep incremental task checkpoints (the Claude Code post-todo handler now only reads its input), pending step rows, explain of temporary checkpoints, and the task checkpoint getters. The tree-building helpers the metadata store still needs move to checkpoint/tree_build.go.

Branches older CLIs left behind are deleted once per repository at session start: only the strict entire/<7+hex>-<6hex> shape, each with a compare-and-delete update-ref, recorded by a marker in the git common dir. entire clean removes the strict shape; entire clean --all and uninstall also remove the bare entire/<hex> form behind confirmation.

These changes ship together because every one of them read the branch this commit stops writing: splitting them leaves a tree where a consumer reads a store that no longer exists, or where cleanup.go still calls the deleted ephemeral store.

The hookperf-tagged commit hook benchmark is re-targeted onto the new state and compiles again (it referenced agent.AgentType and FirstPrompt, which no longer existed).

Addresses #2663, #2378, #2350, #1730, #1807.

Co-Authored-By: Claude Fable 5.1 noreply@anthropic.com Entire-Checkpoint: 01M49BR4WBZHW5J8EPH5Y6VRH3

Checkpoints

can you review if we could get rid of shadow branches and what functionality we would need to cut to do it. please check e2e test scenarios too

Claude CodeOpus 5.5
View session
Checkpoint 1