❯ GITSYNC_MAX_REF_UPDATES_PER_PUSH=5000 go run ./cmd/git-sync replicate --all-refs --stats --verbose \

Claude Code·Opus 4.8[1m]·

Soph·2mo ago·2hr 42min·5 Checkpoints·10 file changes·+831/-76·678.9K tokens

The entire:// scheme isn't a thing git-sync understands. It only knows https://, http://, ssh://, git+ssh://, scp-style git@host:path, and file://. So transport.ParseURL hands the entire:// URL straight to Go's HTTP transport, which rejects it — that's the unsupported protocol scheme "entire" you're seeing.

The Entire git server speaks plain smart-HTTP over HTTPS. Just use https://:

Auth still works the way you expect: the Entire DB credential lookup keys off the host (aws-us-east-2.entire.io), reading ~/.config/entire/hosts.json + the keyring for an active user and refreshing the OAuth token as needed (internal/auth/entiredb.go). That path only activates for http/https endpoints (auth.go:38,167), so switching to https:// is also what enables the automatic token auth — entire:// would have been treated as anonymous even if it had reached the transport.

If you've logged in through the entire CLI for that host, this should authenticate transparently. If not, log in first (or pass --target-token / --target-bearer-token).

One note: entire:// is the shorthand the entire CLI uses, not git-sync. git-sync is the lower-level standard-git-protocol tool, so it wants the real https:// URL.

That confirms it. Here's why git works with entire:// but git-sync doesn't:

entire:// is backed by a git remote helper, not a real network protocol.

When git sees a URL like entire://host/path, it doesn't know that scheme natively either — it follows its remote-helper convention and execs a binary named git-remote-entire from your PATH (git's git-remote-<scheme> protocol). The Entire CLI installs that helper. The source lives at github.com/entireio/cli/cmd/git-remote-entire, and the entiredb codebase references it all over (e.g. bk_pipeline.go:122, the CI pipeline literally downloads and chmod +x's git-remote-entire before any clone). That helper does the OAuth/JWT token exchange and proxies the smart-HTTP protocol to the real https://host/... endpoint.

So the layering is:

git clone entire://… → git shells out to git-remote-entire → helper authenticates and talks HTTPS to the server. The entire:// scheme is just a trigger for the helper.
git-sync … entire://… → git-sync does not use git's remote-helper mechanism. It implements the git wire protocol itself via its own go-git transport layer (newConn in internal/syncer/syncer.go:352), which only dispatches on the schemes it implements (http, https, ssh, git+ssh, file). entire:// falls through to Go's HTTP round-tripper, which has never heard of it → unsupported protocol scheme "entire".

Notably, the entiredb team is already moving away from the entire:// + git-remote-entire combo toward plain HTTPS with a JWT auth env — see cmd/entire-backup/git.go:110: "Replaces the legacy entire:// URL + git-remote-entire…". That's exactly the path git-sync is built for.

So for git-sync, point it at the real https://aws-us-east-2.entire.io/et/... URL (which is what the helper would've dialed anyway), and let git-sync's built-in Entire DB token lookup handle auth by host.

stateless-connect — that's the answer to your question. Yes, git-sync absolutely could use the helper, and that capability is exactly the hook for it.

How it would work

stateless-connect is the gitremote-helpers(7) capability designed to bridge the protocol-v2 smart stream over a helper's stdin/stdout. It's the same mechanism git itself uses to run protocol v2 through a remote helper. The flow git-sync would implement:

In newConn (internal/syncer/syncer.go:357), for an unrecognized scheme, look for git-remote-<scheme> on PATH (here git-remote-entire).
Spawn it, write stateless-connect git-upload-pack\n\n (or git-receive-pack for the push side).
The helper does the auth + TLS + context resolution, then proxies the raw pkt-line/pack stream bidirectionally.
git-sync runs its existing v2 protocol logic over that pipe instead of over http.RoundTripper.

The impedance match is good because git-sync already speaks the stateless smart protocol (info/refs + service POSTs over HTTP). stateless-connect is literally the stateless-HTTP-shaped bridge, so the relay/streaming logic carries over.

So this isn't just possible — given what we found, it's arguably the correct design. git-sync's hand-rolled entiredb.go auth has drifted out of sync with the CLI (it knows only the legacy hosts.json + entire:<host> keychain layout; your real creds are in the context model under entire-core:https://eu.auth.entire.io). The helper is the canonical, maintained implementation of "how to authenticate to an Entire git host" — delegating to it means git-sync never has to chase the CLI's auth changes.

The one real caveat

git-sync's --stats byte counting is bolted onto the HTTP transport via countingRoundTripper (syncer.go:407). Over a helper pipe you'd be counting bytes at the pipe instead — same situation the SSH path is already in (docs note SSH omits byte-counted throughput). So a helper transport would either need its own stream counters or would carry that same limitation initially.

Right now, to unblock yourself

The helper transport doesn't exist yet, so for this run pass the token explicitly instead of relying on the stale native lookup:

Want me to (a) dig out the exact entire CLI command that prints a usable token so you can run it today, or (b) sketch/implement the stateless-connect remote-helper transport in newConn so entire:// works natively? I'd lean toward (b) being the real fix — it deletes the auth-drift problem entirely — but (a) gets you running in the next minute.

Done. entire:// (and any git-remote-<scheme>) now works natively in git-sync, validated against the live server.

What I built

On branch feat/remote-helper-transport, three commits:

HelperConn transport (internal/gitproto/helper.go + tests) — a gitproto.Conn that spawns git-remote-<scheme>, drives its stateless-connect bridge, and runs the smart protocol over it. The helper owns auth and network I/O; git-sync still runs the wire protocol.
Scheme dispatch (internal/syncer/syncer.go) — newConn falls back to a remote helper for any non-native scheme, excluding http/https/ssh.
Docs — docs/usage.md + README FAQ.

How it works (the key findings)

I read the helper's own source (devenv/cli/internal/remotehelper/githelper/) and probed it live. The stateless-connect framing differs by service:

upload-pack (fetch): true v2 stateless — ack \n, advertisement, then per-request POST with a trailing 0002 response-end packet (which HelperConn strips so the consumer sees exactly the HTTP byte stream).
receive-pack (push): a raw proxy that reads the whole push request until stdin EOF — so each RPC gets its own helper process, which also makes batched pushes work.

The advertisements come back banner-stripped (like SSH), which git-sync's existing decoders already accept.

Validation

probe of the entire:// repo: authenticated via your eu.auth.entire.io context, negotiated v2, parsed capabilities. ✅
Your exact original command: pushed=55008 refs, protocol=v2, relay-mode=bootstrap, refs/heads/main and all checkpoints created. ✅
go build, full gitproto/syncer tests, and golangci-lint (0 issues) all clean.

One inherited caveat I documented: like SSH, the helper bridge has no per-request byte counting, so --stats omits helper-side throughput (the GitHub side still printed 19.5 MB @ 2.89 MB/s).

Nothing is pushed to a remote branch — the commits are local. Want me to open a PR, or add the efficiency optimization (reusing one process across the v2 ls-refs+fetch round trips instead of one-per-RPC)?

/simplify → 4 cleanup agents in parallel → apply the fixes

You are improving the quality of the changed code, not hunting for bugs. Review it for reuse, simplification, efficiency, and altitude issues, then fix what you find. Do not look for correctness bugs — that is what /code-review is for.

Phase 0 — Gather the diff

Run git diff @{upstream}...HEAD (or git diff main...HEAD / git diff HEAD~1 if there's no upstream) to get the unified diff under review. If there are uncommitted changes, or the range diff is empty, also run git diff HEAD and include the working-tree changes in scope — the review often runs before the commit. If a PR number, branch name, or file path was passed as an argument, review that target instead. Treat this diff as the review scope.

Phase 1 — Review (4 cleanup agents in parallel)

Launch 4 independent review agents via the Agent tool, all in a single message so they run concurrently. Pass each agent the diff and one of the four angles below. Each returns its findings with file, line, a one-line summary, and the concrete cost (what is duplicated, wasted, or harder to maintain).

Reuse

Flag new code that re-implements something the codebase already has — Grep shared/utility modules and files adjacent to the change, and name the existing helper to call instead.

Simplification

Flag unnecessary complexity the diff adds: redundant or derivable state, copy-paste with slight variation, deep nesting, dead code left behind. Name the simpler form that does the same job.

Efficiency

Flag wasted work the diff introduces: redundant computation or repeated I/O, independent operations run sequentially, blocking work added to startup or hot paths. Also flag long-lived objects built from closures or captured environments — they keep the entire enclosing scope alive for the object's lifetime (a memory leak when that scope holds large values); prefer a class/struct that copies only the fields it needs. Name the cheaper alternative.

Altitude

Check that each change is implemented at the right depth, not as a fragile bandaid. Special cases layered on shared infrastructure are a sign the fix isn't deep enough — prefer generalizing the underlying mechanism over adding special cases.

Phase 2 — Apply the fixes

Wait for all four agents to complete, dedup findings that point at the same line or mechanism, and fix each remaining one directly. Skip any finding whose fix would change intended behavior, require changes well outside the reviewed diff, or that you judge to be a false positive — note the skip rather than arguing with it. Finish with a brief summary of what was fixed and what was skipped (or confirm the code was already clean).

2mo ago·1m·7 calls

<task-notification> <task-id>aedbf9eaea46fce56</task-id> <tool-use-id>REDACTED</tool-use-id> <output-file>/private/tmp/claude-501/-Users-soph-Work-entire-devenv-git-sync/4f8f5e9b-9f90-4c22-88b9-988c55f45833/tasks/aedbf9eaea46fce56.output</output-file> <status>completed</status> <summary>Agent "Efficiency review of helper transport" came to rest</summary> <note>A task-notification fires each time this agent comes to rest with no live background children of its own. The user can send it another message and resume it, so the same task-id may notify more than once.</note> <result>Reviewed `internal/gitproto/helper.go` for efficiency only. Findings:

1. (Primary, hot path) `responseEndReader.fill` allocates a fresh buffer per pkt-line — helper.go:335

buf := make([]byte, n) runs for every data packet of the response. A v2 fetch streams the entire pack as sideband pkt-lines (≤~64KB each), so a P-byte pack ⇒ ~P/65520 allocations of up to 64KB. A 4 GB fetch ≈ 65,000 allocations / ~4 GB of GC garbage, all on the streaming hot path. This is exactly the cost PacketReader was changed to avoid (pktline.go:24 "reuses a fixed header buffer and a growable payload buffer to reduce allocations (issue #17)").

Cheaper alternative — reuse a grow-only buffer, mirroring PacketReader. It's safe here: fill() is only entered when len(r.pending)==0, i.e. the previous packet's bytes have already been fully copy'd out to callers, so overwriting the backing array can't corrupt anything.

This drops steady-state allocation to zero. (The append([]byte(nil), header[:]...) flush/delim case at :325 allocates 4 bytes but fires ~once per response — negligible, leave it.)

A more involved option (forward the 4-byte header, then stream the payload via an io.LimitedReader over src with no payload buffer at all) eliminates the copy too, but complicates the state machine; the buffer-reuse fix captures essentially all the win for a few lines.

2. (Confirm — sound) Fresh process per RPC / extra info/refs

The "one extra info/refs per RPC" doc comment is accurate, and the rationale (helper services one stateless-connect session per process; receive-pack reads pack-to-EOF) is genuine — the process can't be reused. One nuance the comment undersells: each RPC also pays a process spawn + the helper's own auth/token resolution, not just an info/refs. For a fetch that's ~3 spawns; for a ref-heavy multi-batch push it's 1 + N batches, so "negligible beside pack transfer" holds for data-heavy pushes but is less true for many-ref/small-object pushes. Not worth re-architecting — flagging only so the claim isn't taken as universal.

3. (Clean — no action) `helperRPCStream` capture and the body-writer goroutine

helperRPCStream is a plain struct, not a closure; it holds ctx/resp/proc/copyErr, all needed for teardown — no oversized scope retained. The go func() in PostRPCStreamBody captures proc, body, copyErr; body (the request/pack reader) is held only until io.Copy finishes, which is required. No leak.

4. (Minor) `dial` does `append(os.Environ(), …)` per RPC — helper.go:155

Allocates + copies the whole environment each spawn. A few RPCs per sync ⇒ immaterial; note only, no change warranted.

No sequential-vs-parallel waste: the body-write goroutine already overlaps with response reads, and dial→advertisement→write→read are protocol-ordered.

Recommend fixing #1 only.</result> <usage><subagent_tokens>272283</subagent_tokens><tool_uses>2</tool_uses><duration_ms>60509</duration_ms></usage> </task-notification>

2mo ago·8m·11 calls

Full suite passes. Here's the cleanup summary.

Applied

Shared pkt-line primitive (reuse + simplification + efficiency, all converging) — added readRawPktLine to pktline.go and routed both readAdvertisement and responseEndReader.fill through it. This:

removes the two hand-rolled copies of pkt-line header parsing the diff introduced;
adds the pktline.MaxSize bound both copies were missing (matching ReadPacket);
eliminates the per-packet allocation on the pack hot path — responseEndReader now reuses a grow-only buffer, so relaying a multi-GB fetch goes from ~1 alloc per 64 KB packet (~65k allocs / 4 GB garbage) to zero steady-state allocations.

finish() dead guard — dropped the !errors.Is(closeErr, os.ErrClosed) check; the sync.Once in closeStdin makes it unreachable. Collapsed to errors.Join(closeErr, p.wait()).

RequestInfoRefs error branches — merged the twin if … != nil arms (identical wrapping) into one errors.Join(readErr, proc.finish()).

Scheme dispatch (altitude) — folded the trailing if ep.Scheme != "http" && … into the switch ep.Scheme, so native schemes are named once (case "http","https" falls through; default tries the helper). Removes the "keep two scheme lists in sync" trap.

Doc honesty (altitude) — tightened HelperConn's comment to state it requires a stateless-connect-capable helper rather than implying any generic git-remote-<scheme> works.

Skipped (with reason)

helperRPCStream ↔ sshRPCStream merge (reuse) — real parallel, but the field-shape mismatch means a true merge needs a shared interface and edits to ssh.go, outside this diff. Low ROI.
finish/cleanup teardown merge (simplification) — the agent itself rated it marginal; the drain-vs-stdout-close difference doesn't clearly justify a shared teardown(bool).
Close()'s os.ErrClosed guard — unlike finish(), stdout has no sync.Once; the guard is defensive against the documented StdoutPipe/Wait double-close interaction, so removing it is riskier than the win.
gitProtocol hardcoded-v2 / extra info-refs per RPC — behavior-sensitive design points, not quality cleanups; the per-RPC cost is already reasoned about in the doc comment.

All changes are behavior-preserving; the existing helper tests (including the two-packet responseEndReader case) still pass. Changes are in the working tree, not yet committed — want me to commit them onto the branch?

Honestly: no — and I deliberately tightened the doc comment to not claim otherwise. It works for helpers shaped like git-remote-entire, not arbitrary ones. Here's the precise breakdown of what's portable and what's still assumed.

What is reasonably generic

Fetch (upload-pack) is close to spec-conformant. Any helper that supports stateless-connect for upload-pack and speaks v2 will work: ack \n, advertisement, per-request POST framed with a trailing 0002 — that's the gitremote-helpers(7) contract, and responseEndReader strips exactly that. git-remote-curl-style helpers fit this.
Receive-pack read-to-EOF turns out to be the more compatible choice, not less: entire ends the push response at EOF (no 0002), but a spec-conformant helper that does append 0002 still works too, because the report-status decoder stops at its own flush and ignores trailing bytes.

What is still entire-shaped (the real answer to "any helper?")

No capabilities handshake. I send stateless-connect <service> directly instead of doing git's capabilities → pick-connect-vs-stateless-connect negotiation. A helper that only advertises the legacy connect capability (a raw pipe, no stateless framing) gets an unsupported-command error or replies fallback, which I treat as a hard error rather than falling back to connect. So connect-only helpers don't work at all.
Push over stateless-connect is assumed to exist. It's not universal — it's something the helper must specifically implement for git-receive-pack. entire does; not every helper will.
GIT_PROTOCOL=version=2 is hardcoded and the caller's gitProtocol arg is ignored.
Args: <url> <url> instead of <remote-name> <url>. A helper that reads remote.<name>.* git config by name wouldn't find its config.

To make it truly generic

It'd need the negotiation half that's currently missing: send capabilities, read the advertised set, use stateless-connect when present and fall back to driving connect (a raw bidirectional pipe — a different, stateful code path) otherwise. That's a real feature, not a cleanup — the connect path alone is roughly what entire's own handleFetch/handleConnect bridge does (~200 lines).

So today it's accurate to call it "a transport for stateless-connect-capable helpers, validated against git-remote-entire," which is what the doc now says. Want me to add the capabilities probe + connect fallback to make the generic claim real, or leave it scoped to stateless-connect helpers (which covers entire and modern git-remote-curl)?

Done. The capabilities probe is in and validated against the real helper (probe negotiated v2 and now sees the 2 refs from the earlier push — the new probe-first dial path works end-to-end).

What changed

dial now sends capabilities first and reads the advertised set before attempting stateless-connect. If stateless-connect isn't advertised, you get:

instead of a cryptic unsupported-command or fallback error mid-handshake. It's a local exchange — no network round trip happens until after the probe passes, so the cost is negligible.

I kept the scope exactly where you wanted it: probe and fail clearly, no connect fallback. So the honest capability statement is now:

A helper without stateless-connect → clean, actionable error.
A helper with stateless-connect (entire, modern git-remote-curl) → works.

Added TestHelperConn_NoStatelessConnect_MeaningfulError (fake helper advertising only connect), updated the fake to do the capabilities exchange, and all existing helper tests still pass. Build, full test suite, and lint (0 issues) are green.

These cleanup + probe changes are uncommitted in the working tree, on top of the three feature commits. Want me to commit them — as a single Refine remote-helper transport commit, or fold into the existing transport commit?

PR opened: https://github.com/entireio/git-sync/pull/96

Final commit history on `feat/remote-helper-transport`

Three feature commits (transport, syncer wiring, docs) followed by the two post-review commits — the cleanup pass and the capabilities probe, kept separate since they're distinct concerns. I split the mixed helper.go cleanly by reconstructing the cleanup-only state for commit 4 and restoring the probe for commit 5, verifying build + tests + lint at each step.

Everything is pushed and the working tree is clean. go test ./..., go vet, and golangci-lint are all green, and the feature is validated end-to-end against the live server (the 55,008-ref push). The PR body documents the what/how, the scope limitations (stateless-connect-capable helpers only; no --stats throughput), and the testing.