Pre-check PACK header object count before pushing oversized batches

main

Commit

Soph5mo ago

When a batch's FetchPack returns, peek at the first 12 bytes of the pack stream to read the PACK header's object count. Multiply by 750 bytes/object (a conservative compressed-object average) to estimate total pack size. If the estimate exceeds BatchMaxPack, subdivide immediately and re-fetch smaller ranges — avoiding a multi-GiB transfer that the target would reject anyway.

This is a zero-cost optimization: 12 bytes read + one multiplication. The PACK header is prepended back to the reader via MultiReader so the push receives a valid packfile if the check passes.

Combined with the post-push subdivide from the previous commit, the batch loop now has two layers of protection:

  1. Pre-push: PACK header estimate catches obviously-oversized batches before any significant transfer (this commit).
  2. Post-push: target body-limit rejection triggers subdivide for batches that passed the estimate but still exceeded the target's actual limit (previous commit).

For the linux kernel against a 2 GiB target:

  • Estimate plans 3 batches (even by commit count)
  • Batch 3 fetches, PACK header shows 6.5M objects × 750 = ~4.9 GiB
  • 4.9 GiB > 2 GiB BatchMaxPack → subdivide before pushing
  • Re-fetch two halves (~3.2M objects each, ~2.4 GiB)
  • If still over, subdivide again — converges in O(log n) splits

Co-Authored-By: Claude Opus 4.6 (1M context) noreply@anthropic.com Entire-Checkpoint: dbedffba0bc5

Checkpoints

can you review the changes in this branch

Claude CodeOpus 4.6[1m]
View session
Checkpoint 1