Integrate EntireDB Commands into CLI

What would you like to clarify? Happy to dig into any of the tradeoffs — the dep model (Go module dep vs vendoring vs leaving it as a plugin), the command placement, or anything about the surface area / scope of what the PR actually exposes.

What would you like to clarify? I can dig into any of these — what code is actually private vs already public, what the proto contract looks like, what the migration cost is per option, or anything else about the split.

All clean — 0 issues, no drift. Branch soph/vendor-entiredb-repo has 4 commits ready for review.

Summary

Smoke-verified entire repo --help, entire repo clone --help (examples rebranded), entire labs (lists it), and entire --help (doesn't). Lint, unit tests (5,614 pass), and integration tests (340 pass) all green.

Clean history, 5 commits, lint passing. The branch is now:

The bogus "Drop redundant nolint:ireturn directives" commit is gone — those //nolint comments stay where they belong.

On your original report: entire repo ls help was Cobra reading help as a positional arg (the <repo> slot); entire repo ls --help or entire repo help ls both show docs without touching auth. The real bug was the misleading error message — fixed in commit 0afbde7a7. And since you're already logged in (entire auth status shows you on https://entire.io with 5 active tokens), the bridge in 51bb9e9bd now plumbs that token through ENTIRE_TOKEN automatically — verified the gRPC server actually responds to a real token instead of dying locally with "no context".

Known follow-up: entire login doesn't capture refresh tokens, so eventually access tokens expire. The vendored auto-refresh path will log a warning and fall back to the existing bearer — same UX as the cli has always had. Proper refresh-token plumbing is a separate piece of work.

Quick clarification before I write it — the JWT's iss claim is the auth host (e.g. us.auth.entire.io), not the data host. So "use iss directly" still wouldn't reach the right cluster. The conventional derivation is to strip auth. from iss → us.entire.io. That's the heuristic I'll implement, falling back to iss as-is for single-host setups (where iss == entire.io == data host).

What would you like to clarify? Happy to dig into how entiredb actually handles this in practice, what contexts.json looks like end-to-end, what entire.io's production deployment expects, or anything else about how URLs flow.

Wait — let me check what entire repo list and entire org are in your installed binary, because those aren't in this branch's vendored tree.

Branch is at:

Lint clean, tests passing. To use it:

Now the cli mirrors what your coworker has installed:

• Lifecycle verbs (list, create, delete, grant) hit entire-core directly — no cluster URL needed
• Data-plane verbs take the cluster as part of the positional (<cluster>/et/<org>/<repo>) — no ENTIRE_BASE_URL env var, matches entiredb's documented model
• Auth flows through entire-core auth login writing ~/.config/entire/contexts.json — same store the standalone entire-repo plugin reads, no bridge needed

Known follow-up: the cli's own entire login and entire-core auth login still use separate keyring slots. Users currently need entire-core installed and authed for entire repo to work. Bridging is deliberately deferred — the previous attempt got messy and the user-facing UX is fine for now (same as anyone using the production entire-repo binary).

The earlier vendor work (legacy cmd/entiredb/cli, auth bridge, etc.) has been reset out of the branch. The current diff against main is just the two commits above.

Branch is now:

What changes for the user flow:

Before: needed both entire login (cli's own) AND entire-core auth login (for entire repo to find a context).

After: just entire login — it now does both writes (cli's own entire-cli keyring + entiredb's entire-core:<iss> keyring + ~/.config/entire/contexts.json entry + flips current_context). The bridge is keyed on the JWT's own iss/handle claims, so single-host and split-host deploys both Just Work without extra config.

Known limits:

• Bridge skips silently if the device-flow token is opaque (not a JWT) or missing iss/handle.
• Refresh tokens aren't captured by device flow, so the vendored auto-refresh path logs a warning and falls back to the existing bearer on expiry — same behavior the cli has always had.
• Failure to write the entiredb half is non-fatal (warning, not error); cli login still completes so non-entire repo surfaces are unaffected.

Test it next time you entire login — entire repo list should work without touching entire-core at all.

The error's from the new plugin's positional parser — that's the design, not a regression from our work. The format is <cluster>/<surface>/<org>/<repo> where cluster is the data-cluster host (separate from auth host):

gh/entireio/cli got parsed as cluster=gh, path=entireio/cli — hence the surface-prefix complaint about entireio/cli.

You need the data-cluster host. From entire auth status you're logged in via us.auth.entire.io (auth), so your data cluster is probably us.entire.io or similar. Quickest way to find out:

Each entry should carry a cluster host. Or your coworker can tell you what they use — for the EU example they showed, it'd be e.g. eu.entire.io/gh/entireio/cli or whatever the EU cluster is named.

If the cluster-discovery UX is rough, that's a real complaint we should bring to whoever owns the entiredb CLI — but it's the upstream design, not something our vendor introduced. Want me to dig deeper or is this enough to get you unstuck?

Here's the surface of each:

entiredb's kubectl-style plugins on main

All four read auth from the same ~/.config/entire/contexts.json that entire login now populates (after the bridge we just landed), so they'd all "just work" with one login.

What it'd take to vendor the other three

They're much smaller than entire-repo and share the same shared-package set already vendored (cliauth, cliapi, cliauth, clidocs, client/*). Adding entire-org, entire-project, entire-grant is mostly:

1. Copy 4 dirs (~830 LOC total: 2 files for org, 2 for project, 5 for grant + the main.go shims)
2. Rewrite imports — same as before, one sed pass
3. Three thin wrappers in cmd/entire/cli/ (one per plugin) — each ~20 lines, identical shape to repo.go
4. Add three labs entries

No new transitive deps; no surgery needed. The org/project/grant plugins are lifecycle-only (entire-core HTTP, no gRPC, no scoped JWT exchange), so they're simpler than entire-repo.

Want me to vendor them in the same branch?

Branch is now:

entire labs now lists all four:

After entire login (which now also writes contexts.json), the full kubectl-style surface should work without entire-core or the separate plugin binaries installed. ~830 LOC of new vendored source, no new module deps.

Branch is now 6 commits. entire project create (and any other vendored arg-count failure) now prints the failing subcommand's usage block before the error — matching how entire foobarbaz and entire --bogus-flag already worked.

Test it:

Other error paths (unknown command, unknown flag, runtime errors) are unchanged.

The matcher is a string substring on "arg(s)" — cheap, stable across cobra's stock validators, doesn't reach into cobra internals. The switch from ExecuteContext to ExecuteContextC is the only structural change; it gives us back the deepest matched command so we can show its usage rather than the root's.

Draft PR opened: https://github.com/entireio/cli/pull/1294

6 commits, +11,384 LOC (most of it the vendored entiredb plugin tree + the shared cliauth/cliapi/client/* infrastructure). Body covers the layout, the auth-bridging work, the UX polish (entire login → contexts.json, usage-on-arg-error), what got trimmed vs upstream, new module deps, the test plan I ran locally, and three known follow-ups.

No response requested.

oapi-codegen 2.4.1 doesn't support OpenAPI 3.1's nullable-array shorthand (type: [array, null]). Let me survey how much 3.1-specific syntax is in the spec before deciding how to handle it.

Only 5 occurrences, all the same construct (type: ["array","null"] — huma's 3.1 nullable shorthand). Let me see exactly which schemas, then decide between stripping them or normalizing.

Done. The OpenAPI-first approach is complete and committed on soph/entire-control-plane-client (2 commits, off origin/main):

What this delivers vs the vendor approach

The pipeline (all reproducible via go generate)

1. spec/core.openapi.json — pristine upstream spec
2. spec/normalize.go — collapses huma's type:["array","null"] → "array", and replaces per-operation 4xx/5xx with a single default so ogen emits convenient errors ((*Org, error) not sum types)
3. ogen (client-only) → 24K LOC typed client

Verified working

• entire org list → [] against the live control plane (real authenticated round-trip)
• Errors surface as clean problem-detail messages via APIError
• entire labs lists all four; --help works on each; hidden from root help
• go build / go test ./... (54 pkgs) / go vet / lint all clean; unit test on the error mapper

On the two open questions from earlier

• ogen vs libopenapi: ogen won — libopenapi is a spec toolkit, not a client generator. The 3.1 nullable blocker turned out orthogonal to the generator choice and is handled by the normalizer (which becomes a no-op once entiredb's server emits non-null arrays — your "we'll fix the spec" path).
• Base-URL discovery: the control plane is served at the login host (<auth-host>/api/v1), so api.AuthBaseURL() is correct and consistent with the existing auth commands. A user on the bare entire.io default needs ENTIRE_AUTH_BASE_URL set — same constraint entire auth status already has.

Want me to open this as a draft PR (and/or close the vendor PR #1294), add more coreapi tests, or wire any of the remaining Core API surface (service-accounts, mirrors, audit, me)?

Cleanup complete. Here's what the four agents found and what I did.

Applied (1 fix)

Altitude — centralized the repeated RunE preamble. All 14 command bodies repeated the same invariant: SilenceUsage = true → build client → call → renderCoreError → printJSON. Extracted two helpers in corecmd.go:

• runCoreJSON(cmd, fn) — for the read/create commands that render JSON
• runCore(cmd, fn) — for delete/revoke/remove that print a message

The four command files now express only their per-command logic (the API call + flags). Removed ~4 lines × 14 sites of boilerplate and the now-redundant newCoreClient wrapper.

Follow-on: the refactor surfaced wrapcheck on closures returning bare coreapi errors (wrapcheck allows direct return f() tail-calls, which is why create/get weren't flagged, but not x, err := f(); return err). Since the whole point of renderCoreError is to map those errors centrally, I declared that intent once via wrapcheck.ignore-package-globs: [.../internal/coreapi] rather than scattering 6 fmt.Errorf wraps that renderCoreError would immediately unwrap.

Skipped (with reasons)

• Efficiency — "STS exchange per request" in BearerAuth (flagged High): false positive. Verified auth-go's tokenmanager caches exchanged tokens, and each CLI command makes exactly one API call — so there's nothing to amortize.
• Efficiency — double AST pass in normalize.go: build-time-only (runs in go generate), and the two transforms walk different subtrees (all-schemas vs paths→operations); merging would hurt clarity for zero runtime benefit.
• Reuse — printJSON duplicates an inline pattern in auth.go/sessions.go: there's no existing shared helper to call, only the same inline idiom repeated. Extracting a repo-wide one would touch unrelated files, out of scope for this diff.
• Simplification/Altitude — collapse the per-resource command structure or optional-field wrapping: both agents agreed the per-command repetition is idiomatic cobra and the type diversity (3 distinct input bodies, different Opt enums) makes a generic builder more complex than the repetition. Left as-is.

Build, lint (0 issues), and tests all green; entire org list still round-trips live, help output and the delete message-path are intact. Only the 5 intended files changed (.golangci.yaml + the 4 command files + corecmd.go); the generated client and unrelated files are untouched. These are working-tree edits on top of the two existing commits — want me to amend them into the command commit, or leave them staged for you to review?

Scope is clean now — only my intended files. entire repo list (and every other arg-count error) now prints the failing subcommand's usage block before the error:

The change (in main.go): switched ExecuteContext → ExecuteContextC to recover the deepest matched command, and added an isPositionalArgError branch that routes cobra's accepts N arg(s) failures through the existing showSuggestion path — same mechanism unknown command/unknown flag already used. Unknown-command and normal errors are unchanged; build + lint green.

This sits on top of the /simplify working-tree edits. Current uncommitted set:

• main.go — usage-on-arg-error (this change)
• corecmd.go / org.go / project.go / repo.go / grant.go — the runCore/runCoreJSON refactor
• .golangci.yaml — wrapcheck ignore-package-globs for coreapi

Want me to fold these into the existing two commits (the refactor into the "Surface …" commit, main.go as its own), or leave them staged for your review?

The generated client already has all four mirror operations (they came free from the spec) — there's just no CLI command in front of them. So entire mirror doesn't exist on this branch yet.

To be precise about what's where:

One caveat worth flagging: the vendor's entire repo mirror was a mixed data-plane/placement command (it could repoint your local git remote at a mirror). The Core API's /mirrors is the control-plane view — managing mirror records (create/list/get/delete). So it's the same domain but not a 1:1 behavioral replacement; the "repoint my local clone" part isn't a Core API operation.

Wiring it is cheap though — a thin cmd/entire/cli/mirror.go using the existing helpers (runCoreJSON/runCore), exactly like the other four. Want me to add entire mirror with list / create / get / delete? If so, let me check the CreateMirrorInputBody shape to get the flags right — and I'd add it to the labs list alongside the others.

What would you like to clarify? I can dig deeper into any part — e.g. exactly what the STS repo-scoped token exchange does and whether auth.TokenForResource could cover it, what git-remote-entire requires, whether the readiness poll is essential or optional, or how the entiredb CLI expects users to invoke this end-to-end.

Both wire the same four record operations (create POST, list, get, remove). The only difference is what mirror create does after the POST succeeds.

What the increment actually is

Option 1 — mirror create returns the moment the placement record exists:

That's one POST /api/v1/mirrors and done.

Option 2 — after the POST, it polls until the initial GitHub→EntireDB clone is actually usable:

Why that poll is expensive to add here

The record (option 1) lives entirely on the control plane — the single host this branch is built around, reached with the generated client and the login bearer. The readiness poll (option 2) talks to the data-plane cluster, which is a different host with a different auth requirement, and that's precisely the boundary this branch was designed to not cross. Concretely, the poll needs three things the control-plane client doesn't:

So option 2 reintroduces a slice of exactly the data-plane/STS machinery the OpenAPI-first pivot removed — the architectural cost is bigger than the ~30 lines of poll loop suggest.

The UX side — why it's not just polish

Without the poll, create succeeds but the mirror may not be clonable yet (initial sync still running). A user who immediately runs git clone entire://… hits a confusing failure/hang and has to retry blindly. The poll is what makes "create succeeded" mean "you can clone now." That's why entiredb's CLI does it by default (with --no-wait to opt out).

The honest framing

• Option 1 is complete and clean for "I want the mirror record to exist" (and for mirrors that are already synced). It keeps the branch a pure control-plane client. Its weakness is the silent gap between "record created" and "actually clonable."
• Option 2 is the faithful entire repo mirror create behavior, but it punctures the control-plane-only boundary — you're now maintaining a repo-scoped token exchange and a data-plane probe in a branch whose whole pitch was "single host, generated client, no data-plane."

A reasonable middle path: ship option 1 now, and make the success output set the expectation honestly (e.g. "mirror registered; initial sync may take a few minutes before git clone works"). Add the readiness wait later as a deliberate, separate piece if/when the data-plane auth helper is worth bringing over — rather than smuggling it in with mirror.

Which way do you want to go?

Yes — exactly. The auth the readiness poll needs is the same credential a git clone of that repo needs.

Here's the precise chain from the code:

• The poll hits GET https://<cluster>/gh/<owner>/<repo>/info/refs?service=git-upload-pack — that's literally the first request git clone makes (the smart-HTTP ref advertisement).
• The data-plane git endpoint (AcceptGitAuth) rejects the login bearer with 403. The code comment is explicit: "only accepts repo-scoped JWTs — a raw login JWT is rejected with HTTP 403. Exchange the login JWT via core for a repo-scoped token first."
• So it calls ExchangeRepoScopedToken(…, repoSlug, "pull", "https://"+clusterHost, …) — an RFC 8693 exchange that mints a token scoped to that one repo, action pull, audience the cluster. That token is then used as basic-auth for the probe.

So the missing auth isn't mirror-specific at all — it's the per-repo, data-plane "pull" token that gates every read against a cluster's git endpoint: clone, fetch, info/refs, ls, log, diff. The readiness poll just happens to be the one control-plane command that reaches across into data-plane territory and therefore needs it.

This is the same thing we dropped when we scoped to control-plane-only. Recall the split:

What we have today (auth.TokenForResource) only ever exchanges for the control-plane audience. The repo-scoped one is a different exchange shape (per-repo resource, action scope, cluster audience). The cli's lower-level auth.Token(ctx, TokenRequest) can set a custom audience/resource/scope, so it's plausibly buildable without vendoring cliauth — but it would need to match exactly what the server's repo-exchange validator expects, which I'd want to verify against the server before relying on it.

Two implications worth naming:

1. mirror create with the wait (option 2) and a future entire repo clone need the identical building block. If we build the repo-scoped exchange once, it unlocks both — the readiness poll and the whole data-plane read surface we set aside.
2. Note git clone entire://… already works without us — git-remote-entire does that exact repo-scoped exchange internally. So users can clone today; what we'd be adding is the CLI doing the exchange itself (in Go) to probe readiness rather than shelling out to git.

So: option 1 stays purely control-plane. Option 2 means building (or vendoring) the repo-scoped "pull" exchange — which is the real prerequisite for ever doing data-plane git reads from this CLI directly. Worth deciding whether that's a door you want to open now or keep closed until there's a clear reason to do clone/ls/log here too.

The repo-scoped token exchange is built, tested, and committed. Branch now at 5 commits:

What I built

auth.RepoScopedToken(ctx, clusterBaseURL, repoSlug, action) in cmd/entire/cli/auth/repo_token.go — the credential the data plane requires for any git read (clone/fetch/info-refs), which the control-plane login bearer can't satisfy.

Key decisions, grounded in what I verified:

• Built on auth-go's sts.Client (the same package the tokenmanager already uses), not a hand-rolled POST and not the tokenmanager itself. The tokenmanager would have sent a stray resource param alongside audience; sts.Client lets me send audience+scope only — matching entiredb's known-good form byte-for-byte.
• Reuses the cli's own identity — I confirmed the cli's v2 provider already uses client_id=entire-cli and STSPath=/oauth/token, identical to entiredb's exchange, so a successful entire login implies a usable exchange. No new config, no vendored auth.
• Explicit requested_token_type because sts.Client (unlike the tokenmanager) applies no default.
• A test locks the wire contract (grant_type, both token-types, audience, scope=repo:pull, client_id, and the absence of resource) via an injected transport — so the form the server accepts can't silently drift.

What this unlocks

This is the foundation we identified: the same per-repo "pull" token gates mirror create's readiness poll and any future entire repo clone/ls/log against the data plane.

I committed the helper on its own (and folded in the earlier /simplify refactor + usage-on-arg-error as their own commits, per your "meaningful commits in between"). I have not yet wired it into a consumer — next step would be entire mirror with the create-time readiness wait (option 2). Want me to wire that now, or pause here with the exchange primitive landed?

There isn't one yet — on this branch nothing mirror-related is wired into the CLI. entire mirror just falls through to the root help.

What exists so far is only the plumbing, not a command:

• the generated coreapi client has CreateMirror/ListMirrors/GetMirror/DeleteMirror/LookupRepoByMirror
• auth.RepoScopedToken (the data-plane credential the create-time readiness poll would need)

No cmd/entire/cli/mirror.go, no labs entry. I stopped at the exchange primitive and asked before wiring the command — so there's currently no way to create a mirror from this CLI.

When wired (the option-2 path we discussed), it would be:

— POST /api/v1/mirrors, then poll the cluster's info/refs (using auth.RepoScopedToken) until the initial clone is ready, matching entiredb's entire repo mirror create.

Want me to wire entire mirror now (create with the readiness wait, plus list/get/remove)?

entire repo mirror is wired and committed. Branch now at 6 commits.

What you can run now

create registers the placement (POST /mirrors), then — unless --no-wait — blocks until the initial GitHub→EntireDB clone is usable, so a follow-up git clone actually works. That readiness wait is what the repo-scoped token exchange we built last step unlocked: it mints a repo:pull token via auth.RepoScopedToken and polls the cluster's smart-HTTP info/refs, decoding the advert with go-git packp until HEAD resolves.

Verified

• entire repo mirror list returns real mirror records live against the control plane (full path: ogen client + login bearer)
• All four subcommands show correct help; repo now lists mirror alongside create/list/get/delete
• fmt/lint/tests green; scope limited to the three intended files

What I deliberately left out

The mirror use / --disable local-remote rewrite from entiredb — that's a git-config + git-remote-entire concern outside the control-plane API, and git clone entire://… already works through git-remote-entire without the CLI. Noted in the commit so it's an explicit decision, not an oversight.

One caveat worth flagging: I tested list live (read), but not an actual create with the readiness poll end-to-end — that needs a real GitHub repo to mirror and write access, which I didn't want to do against your account without asking. The wire form of the token exchange is locked by a unit test, and the probe reuses go-git's well-tested packp decoder, but the create→poll happy path hasn't run against a live cluster. Want me to walk through a real create to confirm, or is that something you'd rather drive yourself?

Pushed and opened as a draft: https://github.com/entireio/cli/pull/1299

What's in it

6 commits on soph/entire-control-plane-client off main:

The diff is +29.5K, but only ~18 files are hand-written — the bulk is the ogen-generated oas_*.go + the committed spec/normalized-spec. The PR body calls that out and explains the generation pipeline so reviewers know what to actually read.

Heads-up for review

• The body's test plan is honest about the one gap: repo mirror create's clone-readiness poll hasn't been exercised against a live cluster (needs a real GitHub repo + write access). The token-exchange wire form is unit-tested and the probe reuses go-git's packp, but the create→poll happy path is unverified end-to-end. Flagged as an unchecked box.
• It's a draft, so CI can run before you decide on review.

This is a separate PR from the vendor approach (#1294) — they're alternative implementations of the same goal. Worth deciding whether to close #1294 now that this exists, but I left it alone.