Free commit graph store immediately after extracting first-parent chain

main

Commit

Soph5mo ago

The commit graph fetch for linux stores 1.4M commit objects in memory.NewStorage() (~4.6 GB). We only need the 75k first-parent chain hashes (~3 MB) for checkpoint placement. Previously the store stayed alive for the entire bootstrap run because the function scope held it.

Now: extract the chain, nil the store, force GC. The 4.6 GB becomes garbage-collectible immediately after chain extraction (~22s into the run) instead of persisting for the full ~17 minutes.

The transient spike during the fetch is unavoidable with current tools:

  • git protocol v2 has no "first-parent only" fetch command
  • tree:0 filter gives all 1.4M commits, not just the 75k first-parent
  • go-git's pack parser needs the full store for delta resolution between commits during pack decoding

A streaming parent-map extractor could eliminate the spike entirely but requires go-git packfile internals work (custom storer or scanner integration). The GC approach is the pragmatic fix.

Also adds FirstParentChainFromMap to planner/checkpoint.go for future use when a streaming extractor is implemented.

Co-Authored-By: Claude Opus 4.6 (1M context) noreply@anthropic.com Entire-Checkpoint: 56d371ce68d4

Checkpoints

can you review the changes in this branch

Claude CodeOpus 4.6[1m]
View session
Checkpoint 1