Humans vs LLMs Quiz Web App

use superpowers for this project i am just starting now. make a web app game called Humans vs LLMs: Humanity's Last Exam. The webpage asks "can you beat LLMs on Humanity's Last Exam? Take the quiz here!" The app
- pulls 10 Qs (of different subject matters) from https://agi.safe.ai/
- Make multiple choice questions the user can answer
- provide a social media link and graphic to share how they did. this can be copy-and-pasteable or use the X API and should link to this app (which will be deployed on Cloudflare Workers)
- after someone takes the quiz, they are added to the Human ranking durable leaderboard.
- visitors to the webapp can click a button to see the leaderboard regardless
- after someone submits their answers to the quiz, show the correct answers and what they submitted. Say something like: on average, {Fable 5/Opus 4.8/Muse Spark 1.1/GPT 5.5/models on the benchmark} out-performs humans on this question by x%. The x % changes each time someone takes the quiz. Do the math to calculate that number.
- whenever someone refreshes the page or revisits the page, they get different questions
Using superpowers since you've explicitly asked for it here. Since this is a "let's build X" task, I'll start with the brainstorming skill to nail down design decisions before writing any code.
Base directory for this skill: /Users/lizziesiegle/.claude/plugins/cache/claude-plugins-official/superpowers/6.1.1/skills/brainstorming
Brainstorming Ideas Into Designs
Help turn ideas into fully formed designs and specs through natural collaborative dialogue.
Start by understanding the current project context, then ask questions one at a time to refine the idea. Once you understand what you're building, present the design and get user approval.
<HARD-GATE> Do NOT invoke any implementation skill, write any code, scaffold any project, or take any implementation action until you have presented a design and the user has approved it. This applies to EVERY project regardless of perceived simplicity. </HARD-GATE>Anti-Pattern: "This Is Too Simple To Need A Design"
Every project goes through this process. A todo list, a single-function utility, a config change — all of them. "Simple" projects are where unexamined assumptions cause the most wasted work. The design can be short (a few sentences for truly simple projects), but you MUST present it and get approval.
Checklist
You MUST create a task for each of these items and complete them in order:
- Explore project context — check files, docs, recent commits
- Offer the visual companion just-in-time — NOT upfront. The first time a question would genuinely be clearer shown than described, offer it then (its own message); on approval its browser tab opens for you. If no visual question ever arises, never offer it. See the Visual Companion section below.
- Ask clarifying questions — one at a time, understand purpose/constraints/success criteria
- Propose 2-3 approaches — with trade-offs and your recommendation
- Present design — in sections scaled to their complexity, get user approval after each section
- Write design doc — save to
docs/superpowers/specs/YYYY-MM-DD-<topic>-design.mdand commit - Spec self-review — quick inline check for placeholders, contradictions, ambiguity, scope (see below)
- User reviews written spec — ask user to review the spec file before proceeding
- Transition to implementation — invoke writing-plans skill to create implementation plan
Process Flow
The terminal state is invoking writing-plans. Do NOT invoke frontend-design, mcp-builder, or any other implementation skill. The ONLY skill you invoke after brainstorming is writing-plans.
The Process
Understanding the idea:
- Check out the current project state first (files, docs, recent commits)
- Before asking detailed questions, assess scope: if the request describes multiple independent subsystems (e.g., "build a platform with chat, file storage, billing, and analytics"), flag this immediately. Don't spend questions refining details of a project that needs to be decomposed first.
- If the project is too large for a single spec, help the user decompose into sub-projects: what are the independent pieces, how do they relate, what order should they be built? Then brainstorm the first sub-project through the normal design flow. Each sub-project gets its own spec → plan → implementation cycle.
- For appropriately-scoped projects, ask questions one at a time to refine the idea
- Prefer multiple choice questions when possible, but open-ended is fine too
- Only one question per message - if a topic needs more exploration, break it into multiple questions
- Focus on understanding: purpose, constraints, success criteria
Exploring approaches:
- Propose 2-3 different approaches with trade-offs
- Present options conversationally with your recommendation and reasoning
- Lead with your recommended option and explain why
Presenting the design:
- Once you believe you understand what you're building, present the design
- Scale each section to its complexity: a few sentences if straightforward, up to 200-300 words if nuanced
- Ask after each section whether it looks right so far
- Cover: architecture, components, data flow, error handling, testing
- Be ready to go back and clarify if something doesn't make sense
Design for isolation and clarity:
- Break the system into smaller units that each have one clear purpose, communicate through well-defined interfaces, and can be understood and tested independently
- For each unit, you should be able to answer: what does it do, how do you use it, and what does it depend on?
- Can someone understand what a unit does without reading its internals? Can you change the internals without breaking consumers? If not, the boundaries need work.
- Smaller, well-bounded units are also easier for you to work with - you reason better about code you can hold in context at once, and your edits are more reliable when files are focused. When a file grows large, that's often a signal that it's doing too much.
Working in existing codebases:
- Explore the current structure before proposing changes. Follow existing patterns.
- Where existing code has problems that affect the work (e.g., a file that's grown too large, unclear boundaries, tangled responsibilities), include targeted improvements as part of the design - the way a good developer improves code they're working in.
- Don't propose unrelated refactoring. Stay focused on what serves the current goal.
After the Design
Documentation:
- Write the validated design (spec) to
docs/superpowers/specs/YYYY-MM-DD-<topic>-design.md- (User preferences for spec location override this default)
- Use elements-of-style:writing-clearly-and-concisely skill if available
- Commit the design document to git
Spec Self-Review: After writing the spec document, look at it with fresh eyes:
- Placeholder scan: Any "TBD", "TODO", incomplete sections, or vague requirements? Fix them.
- Internal consistency: Do any sections contradict each other? Does the architecture match the feature descriptions?
- Scope check: Is this focused enough for a single implementation plan, or does it need decomposition?
- Ambiguity check: Could any requirement be interpreted two different ways? If so, pick one and make it explicit.
Fix any issues inline. No need to re-review — just fix and move on.
User Review Gate: After the spec review loop passes, ask the user to review the written spec before proceeding:
"Spec written and committed to
<path>. Please review it and let me know if you want to make any changes before we start writing out the implementation plan."
Wait for the user's response. If they request changes, make them and re-run the spec review loop. Only proceed once the user approves.
Implementation:
- Invoke the writing-plans skill to create a detailed implementation plan
- Do NOT invoke any other skill. writing-plans is the next step.
Key Principles
- One question at a time - Don't overwhelm with multiple questions
- Multiple choice preferred - Easier to answer than open-ended when possible
- YAGNI ruthlessly - Remove unnecessary features from all designs
- Explore alternatives - Always propose 2-3 approaches before settling
- Incremental validation - Present design, get approval before moving on
- Be flexible - Go back and clarify when something doesn't make sense
Visual Companion
A browser-based companion for showing mockups, diagrams, and visual options during brainstorming. Available as a tool — not a mode. Accepting the companion means it's available for questions that benefit from visual treatment; it does NOT mean every question goes through the browser.
Offering the companion (just-in-time): Do NOT offer it upfront. Wait until a question would genuinely be clearer shown than told — a real mockup / layout / diagram question, not merely a UI topic. The first time that happens, offer it then, as its own message:
"This next part might be easier if I show you — I can put together mockups, diagrams, and comparisons in a browser tab as we go. It's still new and can be token-intensive. Want me to? I'll open it for you."
This offer MUST be its own message. Only the offer — no clarifying question, summary, or other content. Wait for the user's response. If they accept, start the server with --open so their browser opens to the first screen automatically. If they decline, continue text-only and don't offer again unless they raise it.
Per-question decision: Even after the user accepts, decide FOR EACH QUESTION whether to use the browser or the terminal. The test: would the user understand this better by seeing it than reading it?
- Use the browser for content that IS visual — mockups, wireframes, layout comparisons, architecture diagrams, side-by-side visual designs
- Use the terminal for content that is text — requirements questions, conceptual choices, tradeoff lists, A/B/C/D text options, scope decisions
A question about a UI topic is not automatically a visual question. "What does personality mean in this context?" is a conceptual question — use the terminal. "Which wizard layout works better?" is a visual question — use the browser.
If they agree to the companion, read the detailed guide before proceeding:
skills/brainstorming/visual-companion.md
can you load the dataset with from datasets import load_dataset
dataset = load_dataset("cais/hle", split="test")
Good — the two sources disagree meaningfully on exact numbers (different eval dates/methodologies), which confirms these figures are volatile. I'll use Scale AI's leaderboard (the benchmark's own official site) as primary source, with a visible "as of [date], source: Scale AI" note, stored in an easily-refreshable config file rather than baked in as unlabeled fact.
Here's the full design:
Architecture
Single Cloudflare Worker, no separate backend:
- Static SPA (vanilla HTML/CSS/JS, no framework) served via Workers Static Assets — homepage, quiz, results, leaderboard views.
- API routes (
/api/*) on the same Worker, using Hono for routing. - One Durable Object (
LeaderboardDO, singleton instance, SQLite-backed storage) holding the persistent human leaderboard — this is the "durable leaderboard" you asked for. - No live HF/agi.safe.ai calls at request time. I already have working HF access here, so I'll write a one-off script now that loads
cais/hle, filters to the plain-text multiple-choice subset we agreed on, and bakes the result intosrc/questions.json(question, choices, correct letter, category) — checked into the repo, regenerable later by rerunning the script.
Quiz flow & anti-spoiler mechanism
GET /api/quiz/new— Worker picks 10 questions (1 from each of the 8 categories + 2 more), shuffles choice order per question, strips correct answers, and returns the questions plus an encrypted token (AES-GCM, key = a Worker secret) containing the question IDs + correct letters. The client never receives answers in cleartext — inspecting network traffic doesn't reveal them.- Every page load/refresh calls this fresh — satisfies "different questions each visit."
POST /api/quiz/submit { token, answers }— Worker decrypts the token, grades the submission, and for each question: for each of ~5 curated models, rolls a weighted random check (crypto-random, weighted by that model's real published overall HLE accuracy) to simulate whether it got that question right. Aggregates into the "models out-perform you by X%" stat — clearly labeled as a simulated estimate, not real per-question data, since none exists publicly. Returns score, per-question correct/your-answer/correct-answer, and the X% stat.- Response also includes a short signed
resultToken(score + timestamp, HMAC-signed) so a leaderboard submission can't be forged with an arbitrary score from the browser console.
Leaderboard & sharing
POST /api/leaderboard { nickname, resultToken }→ Worker verifies the signature,LeaderboardDOstores{nickname, score, date}.GET /api/leaderboard→ top 50 by score desc, earliest date as tiebreaker. Visible via a button on the homepage regardless of whether you've played.GET /api/share/:resultId.png→ generates a PNG result card server-side (viaworkers-og, i.e. Satori+resvg) showing score and the X% stat — used both as the page'sog:image(so pasting the app link unfurls nicely) and as a downloadable/copy-pasteable image. Paired with pre-filled share text and a link back to the app. X API OAuth posting is a later phase, not in this build.
Testing
Vitest + @cloudflare/vitest-pool-workers for: token encrypt/decrypt round-trip, grading logic, model-simulation math, leaderboard DO methods. Manual smoke test via wrangler dev: play a full quiz, confirm leaderboard updates, confirm refresh yields different questions, confirm share image renders.
Does this look right overall, or anything you want to change before I write it up as a spec?
I did a self-review pass — no placeholders, contradictions, or ambiguity found, and scope is focused enough for a single implementation plan.
Spec written and committed to docs/superpowers/specs/2026-07-14-humans-vs-hle-design.md. Please review it and let me know if you want any changes before we move to the implementation plan.
Base directory for this skill: /Users/lizziesiegle/.claude/plugins/cache/claude-plugins-official/superpowers/6.1.1/skills/writing-plans
Writing Plans
Overview
Write comprehensive implementation plans assuming the engineer has zero context for our codebase and questionable taste. Document everything they need to know: which files to touch for each task, code, testing, docs they might need to check, how to test it. Give them the whole plan as bite-sized tasks. DRY. YAGNI. TDD. Frequent commits.
Assume they are a skilled developer, but know almost nothing about our toolset or problem domain. Assume they don't know good test design very well.
Announce at start: "I'm using the writing-plans skill to create the implementation plan."
Context: If working in an isolated worktree, it should have been created via the superpowers:using-git-worktrees skill at execution time.
Save plans to: docs/superpowers/plans/YYYY-MM-DD-<feature-name>.md
- (User preferences for plan location override this default)
Scope Check
If the spec covers multiple independent subsystems, it should have been broken into sub-project specs during brainstorming. If it wasn't, suggest breaking this into separate plans — one per subsystem. Each plan should produce working, testable software on its own.
File Structure
Before defining tasks, map out which files will be created or modified and what each one is responsible for. This is where decomposition decisions get locked in.
- Design units with clear boundaries and well-defined interfaces. Each file should have one clear responsibility.
- You reason best about code you can hold in context at once, and your edits are more reliable when files are focused. Prefer smaller, focused files over large ones that do too much.
- Files that change together should live together. Split by responsibility, not by technical layer.
- In existing codebases, follow established patterns. If the codebase uses large files, don't unilaterally restructure - but if a file you're modifying has grown unwieldy, including a split in the plan is reasonable.
This structure informs the task decomposition. Each task should produce self-contained changes that make sense independently.
Task Right-Sizing
A task is the smallest unit that carries its own test cycle and is worth a fresh reviewer's gate. When drawing task boundaries: fold setup, configuration, scaffolding, and documentation steps into the task whose deliverable needs them; split only where a reviewer could meaningfully reject one task while approving its neighbor. Each task ends with an independently testable deliverable.
Bite-Sized Task Granularity
Each step is one action (2-5 minutes):
- "Write the failing test" - step
- "Run it to make sure it fails" - step
- "Implement the minimal code to make the test pass" - step
- "Run the tests and make sure they pass" - step
- "Commit" - step
Plan Document Header
Every plan MUST start with this header:
Task Structure
No Placeholders
Every step must contain the actual content an engineer needs. These are plan failures — never write them:
- "TBD", "TODO", "implement later", "fill in details"
- "Add appropriate error handling" / "add validation" / "handle edge cases"
- "Write tests for the above" (without actual test code)
- "Similar to Task N" (repeat the code — the engineer may be reading tasks out of order)
- Steps that describe what to do without showing how (code blocks required for code steps)
- References to types, functions, or methods not defined in any task
Remember
- Exact file paths always
- Complete code in every step — if a step changes code, show the code
- Exact commands with expected output
- DRY, YAGNI, TDD, frequent commits
Self-Review
After writing the complete plan, look at the spec with fresh eyes and check the plan against it. This is a checklist you run yourself — not a subagent dispatch.
1. Spec coverage: Skim each section/requirement in the spec. Can you point to a task that implements it? List any gaps.
2. Placeholder scan: Search your plan for red flags — any of the patterns from the "No Placeholders" section above. Fix them.
3. Type consistency: Do the types, method signatures, and property names you used in later tasks match what you defined in earlier tasks? A function called clearLayers() in Task 3 but clearFullLayers() in Task 7 is a bug.
If you find issues, fix them inline. No need to re-review — just fix and move on. If you find a spec requirement with no task, add the task.
Execution Handoff
After saving the plan, offer execution choice:
"Plan complete and saved to docs/superpowers/plans/<filename>.md. Two execution options:
1. Subagent-Driven (recommended) - I dispatch a fresh subagent per task, review between tasks, fast iteration
2. Inline Execution - Execute tasks in this session using executing-plans, batch execution with checkpoints
Which approach?"
If Subagent-Driven chosen:
- REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development
- Fresh subagent per task + two-stage review
If Inline Execution chosen:
- REQUIRED SUB-SKILL: Use superpowers:executing-plans
- Batch execution with checkpoints for review
I'll pull in Cloudflare-specific guidance before writing exact code, so the plan uses current, correct APIs rather than possibly-stale training knowledge.
Base directory for this skill: /Users/lizziesiegle/.claude/skills/durable-objects
Durable Objects
Build stateful, coordinated applications on Cloudflare's edge using Durable Objects.
Retrieval Sources
Your knowledge of Durable Objects APIs and configuration may be outdated. Prefer retrieval over pre-training for any Durable Objects task.
Fetch the relevant doc page when implementing features.
When to Use
- Creating new Durable Object classes for stateful coordination
- Implementing RPC methods, alarms, or WebSocket handlers
- Reviewing existing DO code for best practices
- Configuring wrangler.jsonc/toml for DO bindings and migrations
- Writing tests with
@cloudflare/vitest-pool-workers - Designing sharding strategies and parent-child relationships
Reference Documentation
./references/rules.md- Core rules, storage, concurrency, RPC, alarms./references/testing.md- Vitest setup, unit/integration tests, alarm testing./references/workers.md- Workers handlers, types, wrangler config, observability
Search: blockConcurrencyWhile, idFromName, getByName, setAlarm, sql.exec
Core Principles
Use Durable Objects For
| Need | Example |
|---|---|
| Coordination | Chat rooms, multiplayer games, collaborative docs |
| Strong consistency | Inventory, booking systems, turn-based games |
| Per-entity storage | Multi-tenant SaaS, per-user data |
| Persistent connections | WebSockets, real-time notifications |
| Scheduled work per entity | Subscription renewals, game timeouts |
Do NOT Use For
- Stateless request handling (use plain Workers)
- Maximum global distribution needs
- High fan-out independent requests
Quick Reference
Wrangler Configuration
Basic Durable Object Pattern
Critical Rules
- Model around coordination atoms - One DO per chat room/game/user, not one global DO
- Use
getByName()for deterministic routing - Same input = same DO instance - Use SQLite storage - Configure
new_sqlite_classesin migrations - Initialize in constructor - Use
blockConcurrencyWhile()for schema setup only - Use RPC methods - Not fetch() handler (compatibility date >= 2024-04-03)
- Persist first, cache second - Always write to storage before updating in-memory state
- One alarm per DO -
setAlarm()replaces any existing alarm
Anti-Patterns (NEVER)
- Single global DO handling all requests (bottleneck)
- Using
blockConcurrencyWhile()on every request (kills throughput) - Storing critical state only in memory (lost on eviction/crash)
- Using
awaitbetween related storage writes (breaks atomicity) - Holding
blockConcurrencyWhile()acrossfetch()or external I/O
Stub Creation
Storage Operations
Alarms
Testing Quick Start
Base directory for this skill: /Users/lizziesiegle/.claude/skills/wrangler
Wrangler CLI
Your knowledge of Wrangler CLI flags, config fields, and subcommands may be outdated. Prefer retrieval over pre-training for any Wrangler task.
Retrieval Sources
Fetch the latest information before writing or reviewing Wrangler commands and config. Do not rely on baked-in knowledge for CLI flags, config fields, or binding shapes.
| Source | How to retrieve | Use for |
|---|---|---|
| Wrangler docs | https://developers.cloudflare.com/workers/wrangler/ | CLI commands, flags, config reference |
| Wrangler config schema | node_modules/wrangler/config-schema.json | Config fields, binding shapes, allowed values |
| Cloudflare docs | Search tool or https://developers.cloudflare.com/workers/ | API reference, compatibility dates/flags |
FIRST: Check if Wrangler is installed, and if not, install it
Check if Wrangler is installed by running:
If Wrangler is not installed, you should install it by running:
Wherever possible, you should use Wrangler instead of manually constructing API requests.
Key Guidelines
- Use
wrangler.jsonc: Prefer JSON config over TOML. Newer features are JSON-only. - Set
compatibility_date: Use a recent date (within 30 days). Check https://developers.cloudflare.com/workers/configuration/compatibility-dates/ - Generate types after config changes: Run
wrangler typesto update TypeScript bindings. - Local dev defaults to local storage: Bindings use local simulation unless
remote: true. - Profile Worker startup: Run
wrangler check startupto measure startup time and detect scripts that exceed the startup time limit. - Use environments for staging/prod: Define
env.stagingandenv.productionin config.
Quick Start: New Worker
Quick Reference: Core Commands
| Task | Command |
|---|---|
| Start local dev server | wrangler dev |
| Deploy to Cloudflare | wrangler deploy |
| Deploy dry run | wrangler deploy --dry-run |
| Generate TypeScript types | wrangler types |
| Profile Worker startup time | wrangler check startup |
| View live logs | wrangler tail |
| Delete Worker | wrangler delete |
| Auth status | wrangler whoami |
Configuration (wrangler.jsonc)
Minimal Config
Full Config with Bindings
Generate Types from Config
Local Development
Start Dev Server
Remote Bindings for Local Dev
Use remote: true in binding config to connect to real resources while running locally:
Recommended remote bindings: AI (required), Vectorize, Browser Rendering, mTLS, Images.
Local Secrets
Create .dev.vars for local development secrets:
Deployment
Deploy Worker
Manage Secrets
Security: Never pass secret values as command arguments or pipe them via
echo. Use the interactive prompt (preferred), pipe from a file, or usesecret bulk. Never output, log, or hardcode secret values in commands.
Versions and Rollback
KV (Key-Value Store)
Manage Namespaces
Manage Keys
Config Binding
R2 (Object Storage)
Manage Buckets
Manage Objects
Config Binding
D1 (SQL Database)
Manage Databases
Execute SQL
Migrations
Export/Backup
Config Binding
Vectorize (Vector Database)
Manage Indexes
Manage Vectors
Config Binding
Hyperdrive (Database Accelerator)
Manage Configs
Config Binding
Workers AI
List Models
Config Binding
Note: Workers AI always runs remotely and incurs usage charges even in local dev.
Queues
Manage Queues
Config Binding
Containers
Build and Push Images
Manage Containers
Manage Images
Manage External Registries
Security: Never hardcode registry credentials in commands. Use environment variables.
Workflows
Manage Workflows
Manage Workflow Instances
Config Binding
Pipelines
Manage Pipelines
Config Binding
Secrets Store
Manage Stores
Manage Secrets in Store
Config Binding
Pages (Frontend Deployment)
Observability
Tail Logs
Config Logging
Testing
Local Testing with Vitest
vitest.config.ts:
Test Scheduled Events
Troubleshooting
Common Issues
| Issue | Solution |
|---|---|
command not found: wrangler | Install: npm install -D wrangler |
| Auth errors | Run wrangler login |
| Startup time limit exceeded | Run wrangler check startup to profile startup and generate CPU profiles |
| Type errors after config change | Run wrangler types |
| Local storage not persisting | Check .wrangler/state directory |
| Binding undefined in Worker | Verify binding name matches config exactly |
Debug Commands
Best Practices
- Version control
wrangler.jsonc: Treat as source of truth for Worker config. - Use automatic provisioning: Omit resource IDs for auto-creation on deploy.
- Run
wrangler typesin CI: Add to build step to catch binding mismatches. - Use environments: Separate staging/production with
env.staging,env.production. - Set
compatibility_date: Update quarterly to get new runtime features. - Use
.dev.varsfor local secrets: Never commit secrets to config. - Test locally first:
wrangler devwith local bindings before deploying. - Use
--dry-runbefore major deploys: Validate changes without deployment. - Never embed secrets in commands: Use interactive prompts (
wrangler secret put), file-based input (wrangler secret bulk), or secure CI environment variables. Never echo, log, or pass secret values as CLI arguments.
Plan complete and saved to docs/superpowers/plans/2026-07-14-humans-vs-hle.md. Self-review passed (full spec coverage, no placeholders, consistent types/signatures across all 10 tasks). Two execution options:
- Subagent-Driven (recommended) — I dispatch a fresh subagent per task, review between tasks, fast iteration
- Inline Execution — Execute tasks in this session using executing-plans, batch execution with checkpoints
Which approach?