Test Live Server Safety Rules

Claude Code·Opus 5.5·Jennycruzy·5d ago·5min·1 Checkpoint·4 file changes·+287/-12·13.4K tokens

You are working in the Trueup repository (this folder). Run every command in the foreground. Do not commit, push or post anything. Never print or write any key; keys are only in the environment. Do not mention any other project. Use plain words in anything a person reads.

Two jobs.

Job 1: tests for the live demo server (live/server.py)

The live demo at live/server.py runs examples/shop_agent.py when a visitor presses one of three buttons. Write tests/test_live_server.py that exercise its safety rules WITHOUT making real calls or starting the real subprocess:

  • Only the three breaks (normal, cut, agent) and the two modes (released, fixed) are accepted; anything else gets 400. The body is capped (Content-Length is limited when read).
  • A second request from the same visitor inside the cool-down gets 429.
  • While a run is in progress (_run_lock held), a new run gets 429.
  • When today's count reaches the daily cap, a run gets 429, and _count refuses to count past the cap.
  • The state file counts runs and "light on" per mode:break, and a new day starts at zero.
  • /api/job/<id> never returns the visitor's IP.
  • /download/<name>.png sends Content-Disposition: attachment, and a path outside docs/x is not served (try ../ tricks).

Start the server's Handler on a ThreadingHTTPServer on 127.0.0.1 port 0 inside the test, point LIVE_STATE at a tmp file (set the module's STATE_FILE), and monkeypatch threading/subprocess so no real run happens. Import the module from live/server.py by path (it is not a package). If a rule above turns out NOT to hold in the code, fix the code minimally and say so. Keep the server behaviour otherwise unchanged. Run uv run pytest -q until it is fully green.

Job 2: cfo.ai inputs from today's measured costs

Update docs/cfo-inputs.md so every number traces to a file in the repo:

  • Live real-traffic cost: evidence/witness/before.json gate_spend_usd for 40 normal calls + 4 breaks (state the call count).
  • Model cost per fix, per planner run: from hackathon.md / evidence (cite the file and the line or key).
  • Live demo cost per visitor run: estimate from the above (label it ESTIMATE and show the arithmetic), and the daily cap of 300 runs in live/server.py.
  • Witness overhead per call: evidence/witness/before.json wrapper_overhead_microseconds.
  • Proposed prices clearly labelled PROPOSAL: free for small teams; a team plan priced per traced run volume; a vendor plan for tracing companies (the lab re-run on every SDK release, with fixes).
  • The questions for cfo.ai's assistant (Ari): runway at 3 price levels, break-even customers, and how cost per audit changes with volume.

Keep it under 400 words, plain words. When both jobs are done, print a short summary: files changed, tests passing, any code fix you made in live/server.py.

5d ago·4m

Both jobs are done. The full test suite passes: 415 passed (19 deselected), including 37 new tests for the live demo server. Nothing was committed, pushed or posted.

Files changed

  • tests/test_live_server.py (new)
  • live/server.py (two small fixes, below)
  • docs/cfo-inputs.md (rewritten, 389 words)

Fixes in live/server.py

  1. The body cap could be bypassed. A negative Content-Length made the server read with no limit, so the request just hung. It now reads at most 200 bytes and never less than 0.
  2. Some bad requests crashed the handler. A list sent as the break (for example {"break": ["normal"]}), or a body that isn't a JSON object, made the server close the connection with no reply instead of returning 400. It now returns 400. Nothing else in the server changed.

What the tests cover: every rule in your list held apart from those two gaps, and each rule has its own tests. The tests start the server on 127.0.0.1 port 0 and keep their state in a temporary file. They replace the server's thread and subprocess calls with stand-ins, so no real run happens.

Cost doc

  • Live traffic: $0.0123794 for 44 real calls (40 normal + 4 breaks), from before.json.
  • Planner run: $0.09–$0.10 each, or $0.21 per pair at list price (hackathon.md lines 172 and 307–308).
  • Fix run: about $0.52 each, which is $5.73 over 11 runs (line 204).
  • Witness overhead: 2.9 microseconds median per call.
  • Demo (ESTIMATE): one model call per visitor run, so $0.0123794 ÷ 44 = $0.00028 per run. At the cap of 300 runs a day that is $0.084 a day, about $2.53 a month.
  • Prices: all three plans are labelled PROPOSAL.
  • Questions for Ari: runway at three price levels, break-even customers, and how audit cost changes with volume.

Something to check: during the session, hackathon.md gained a new section at the end titled "What Entire recorded, and what it did not". I didn't write it and left it as it is. It doesn't move the lines the cost doc cites.

Not available: the claude.ai Paybox and arvix connectors need you to sign in through your claude.ai connector settings. They weren't needed for this work.