Abort uploads early once projected to exceed the target body limit

main

Commit

Soph4mo ago

Layered on top of the streaming pack observer from the previous commit. The observer now consults a caller-supplied aborter on every Read; when it returns true, subsequent Reads surface ErrPackUploadAborted and the upload is short-circuited. The bootstrap loop wires this in via a new selfImposedBudget local:

  • initialised from p.TargetMaxPack
  • ratchets down each time we observe a smaller server-side cutoff (parsed limit from a 413, or — more commonly with reverse proxies that don't announce the limit in their 4xx body — the bytes that flowed before the connection was cut)
  • persists across attempts so each retry starts with the tightest known ceiling

shouldAbortPush makes the per-Read decision in two regimes:

  • header parsed AND at least one full object observed: project final size as bytesSent × totalObjects ÷ objectsSent; abort if that projection exceeds 95% of budget. Catches front-loaded packs (cli-checkpoints-shape) at ~10% of the upload instead of the full ~limit.

  • header still in flight or zero objects observed: fall back to a flat bytesSent ≥ 95% × budget threshold. Covers the "no parser signal yet but we know we must cut by N bytes" case from a prior 413.

A minBytesBeforeAbort floor (8 MiB) keeps the projection from firing on the header alone; an aborted-early flag lets the push failure path distinguish self-cut from server-cut so the user- facing notice and slog line read sensibly in both cases.

Combined with the calibration heuristic from soph/smart-subdivision, each subsequent attempt costs ~5% of the previous attempt's wasted upload instead of ~100% — for the 500 MiB Cloudflare cap that's ~25 MiB per failed retry instead of 500 MiB.

Entire-Checkpoint: 4d50f20d0358

Checkpoints

can you rebase soph/progress-indicators onto soph/smart-subdivision

Claude CodeOpus 4.7[1m]
View session
Checkpoint 1