Add Trueup Provenance CLI Command

Claude Code·Opus 5.5·Jennycruzy·5d ago·4min·1 Checkpoint·15 file changes·+616/-7·14.9K tokens

You are working in the Trueup repository (this folder). Run every command in the foreground. Do not commit, push or post anything. Never print or write any key. Do not mention any other project. Plain words in anything a person reads. Work fast: finish within 25 minutes.

Goal: every fix Trueup made carries its own Entire record ("how this fix was made"), and the live demo shows it.

1. trueup provenance

Add a CLI command trueup provenance [<finding-id>|--all] (src/trueup/provenance.py + cli wiring + tests) that, for each folder in fixes/ that has a proof.json, writes fixes/<id>/PROVENANCE.md and fixes/<id>/provenance.json:

  • the commit that first added fixes/<id>/patch.diff (git log --diff-filter=A --format=%H -- fixes/<id>/patch.diff, oldest), its date and subject;
  • its Entire-Checkpoint trailer if present (git log -1 --format='%(trailers:key=Entire-Checkpoint,valueonly)' <sha>), and the Entire link https://entire.io/gh/Jennycruzy/trueup/commit/<full sha>. If the commit has no trailer, say plainly "no Entire record for this commit" and do not invent one;
  • the fixer's tries from fixes/<id>/tries.json if present: how many, which failed and the one-line reason, model and cost if recorded;
  • the verifier result from proof.json: before/after lead totals and new mismatches;
  • the upstream link if one exists (upstream/<id>/ or the finding's upstream field; the posted PRs are Arize-ai/openinference#4024, traceloop/openllmetry#4610, neatlogs/neatlogs#207).

Also run entire explain for each checkpoint found (try entire explain --help / entire checkpoint --help first and use the read-only command that prints a checkpoint's summary; pass </dev/null) and include its first ~15 lines as "What the session did" when available. Never include anything that looks like a key or a private path; scrub /root and /home.

Run it with --all and commit nothing. Tests: one with a temporary git repo containing a commit with and without the trailer (assert the link and the "no Entire record" wording), and one that the scrub removes private paths.

2. Show it in the live demo

In live/static/index.html, the three "Where the fixes went" cards: add a small line under each, "How this fix was made: Entire record →", linking the Entire commit page for the fix it describes, ONLY if provenance.json for that fix has a trailer. Cards and fixes: Arize card → fixes/openinference-run_status-openai-agents-top-level-span-status-ok; neatlogs card → the neatlogs Responses/Anthropic stream fixes if they have trailers (else say "recorded build: Entire record" linking the commit 79210a4 "Add fixer, verifier and upstream drafts" which has a trailer). Read the facts from the provenance files; do not hard-code a trailer you have not seen. Add a footer line "Build history recorded with Entire: <link to the latest commit with a trailer>".

3. README

One line under the live link: "Every fix links to the Entire record of the session that made it: see fixes/<id>/PROVENANCE.md." Keep README at or under 305 words.

Run uv run pytest -q until green, then uv run trueup bundle and uv run trueup verify. Print a short summary: files changed, which fixes have Entire records, tests passing.

5d ago·3m

Everything is in place. All tests pass (417), trueup bundle rebuilt the evidence folder, and trueup verify reports OK. Nothing is committed.

Files changed

  • New: src/trueup/provenance.py and tests/test_provenance.py.
  • Changed: src/trueup/cli.py (adds trueup provenance <id>|--all), live/static/index.html and README.md.
  • Generated by --all: PROVENANCE.md and provenance.json in each of the 5 fix folders that have a proof.json.

Which fixes have Entire records

FixEntire record
openinference agents root span statusyes, commit 79210a4
openllmetry agents LLM span countyes, commit 79210a4
neatlogs Anthropic messages.stream statusyes, commit 79210a4
neatlogs Responses stream error eventno, commit 2555765 has no trailer
neatlogs chat refusal outputno, commit 2555765 has no trailer

All three recorded fixes come from the same session, checkpoint 01M4M8M1GSQ2BM1RX2JKDJHPD6. Its Entire summary hasn't been written yet, so "What the session did" shows only the header: session, date, tokens and the session's goal. Each file also lists the tries (for example, Arize: 3 tries, 2 failed, $0.63), the lead totals before and after (19 → 14, no new mismatches) and the upstream link. Only #4024 and #4610 count as posted. The other fixes point to their drafts in upstream/<id>/.

Live page

  • Arize card: "How this fix was made: Entire record →" links commit 79210a4, taken from that fix's provenance.json.
  • neatlogs #207 card: this card is about the Agents fix. That fix was added in 953ce26, which has no trailer and no proof.json. Linking the Anthropic stream fix's record as "how this fix was made" would be misleading, so I used your fallback: "Recorded build: Entire record →" linking 79210a4.
  • Email card: no line, because it doesn't describe a fix.
  • Footer: "Build history recorded with Entire" links 7c92dc4, the latest commit with a trailer.

README The new line sits under the live link. To get back to exactly 305 words I cut 11 words from other sentences:

  • "and watch the record" after the live link
  • "proposed" before "patches" in the line above the table (the next paragraph still says these are proposed patches)
  • "updated with their maintainer's suggestions" is now "updated per their maintainer"
  • the line about which tracing code recorded the runs is shorter
  • "added up what it was given" is now "summed what it got"

Please check you're happy with those wording changes.

Other notes

  • The command runs entire checkpoint explain --short <checkpoint> with no input. That is Entire's read-only summary command; plain entire explain doesn't exist.
  • No key or private path appears in any output.
  • The new tests check the Entire link and the "No Entire record for this commit." wording using a temporary git repo, and check that private paths and keys are removed.

The Paybox and arvix connectors need to be authorized in your claude.ai connector settings before they can be used. This task didn't need them.