Stream-parse the pack as it uploads to expose objects-sent counter

main

Commit

Soph4mo ago

Replaces packReadCounter with packStreamObserver, a wrapping ReadCloser that adds two new instruments to the byte counter:

ObjectsSent() — objects whose zlib stream has been fully walked by us (and therefore fully handed off to the HTTP layer) TotalObjects() — pack-header object count, populated as soon as the first 12 bytes flow through

The implementation tees the pack via io.Pipe to a goroutine running go-git's packfile.Scanner, which sequentially emits HeaderSection, one ObjectSection per object, and FooterSection. Each object increments the atomic; each header populates TotalObjects and closes HeaderReady so callers can wait without polling.

The current bootstrap loop only logs the new counters (objects_sent, total_objects_in_pack) on push failures so verbose runs can sanity-check the parse — no behaviour change yet. The next commit uses these counters to abort uploads early once we project we'll exceed the budget.

Cost note: Scanner does full zlib decompression to find object boundaries (compressed size isn't recorded in the pack format), plus a per-object SHA-1 over the inflated bytes. On modern hardware this runs at 200–500 MB/s per core, well above typical upload speeds, so the observer adds CPU but should not bottleneck the upload. If profiling later shows otherwise, the escape hatch is a custom format walker that drops Scanner's hashing — but the zlib walk itself is unavoidable.

Entire-Checkpoint: a66652708dc7

Checkpoints

can you rebase soph/progress-indicators onto soph/smart-subdivision

Claude CodeOpus 4.7[1m]
View session
Checkpoint 1