Anthropic spend: audit, fixes, and the path to −80%
Every figure below comes from the Anthropic admin cost/usage report pulled today (July 1–31 MTD, as of Jul 30) or from merged PRs. Anything forward-looking is labeled est. and is an estimate, not a measurement.
1 · Baseline (verified)
Frozen snapshot from the admin cost report, July 1–31 MTD as of Jul 30. This is the number every later claim is measured against.
By model
By token type
Uncached input is 64.8% of the bill, output 14.7%, 1h cache writes 11.6%, cache reads 5.0%, 5m cache writes 3.9%. We are paying to write caches roughly 3× what we get back from reading them.
By key
| Key | Share of token volume | What's actually on it |
|---|---|---|
| propflow-prod | ≈90% | 1.086B Sonnet tokens — prod Clara plus leaked CI, preview, and local lanes, all mixed together on one key |
| fede-local | ≈5% | Eval workspace, $100.25 |
| nightly-tests | ≈4% | Nightly suites |
| github · cloudflare-agent · staging | ≈0 | Effectively unused |
Daily shape: typical prod-key days run 20–40M tokens; eval-heavy days spike to ~190M (Jul 5–6), and local gauntlet bursts hit 37–40M (Jul 8, 14). Those spikes are test traffic wearing production's badge.
2 · The leak
PR #5055 had already found five on-PR metered workflows. The audit found the rest of the surface:
| Leak | Detail |
|---|---|
| Sixth on-PR metered path | Vercel preview deployments — pipeline-lab-evals.yml, 98 successful July runs; wo-creation-e2e.yml the same shape |
| Two sibling repos | Claude PR reviewers running on metered keys in appfolio-browser-agent and propflow-ai-app |
| Production code bypass | 1 site — rent-roll llm-fallback constructing a bare new Anthropic() |
| Demo harness | Pulled the production key straight from AWS Secrets Manager |
| 8 script judges | Self-built clients, armed by evals/run-all.sh and a vitest EVAL_KEYS path that exported the key out of .env.local |
| Nightly sms-stress | Had no credential at all — its quality/FHA judge failed open (passed:true). Nightly SMS quality was green off a judge that never ran. |
3 · Shipped today
| PR | What changed | Proof |
|---|---|---|
| #5055 | Five on-PR eval lanes → subscription bearer; fail-closed policy; drift test | Live-proven credential = bearer |
| #5068 | All script bypasses → shared client. The fail-open FHA judge now refuses to start credential-less. classify-trades keeps maxRetries: 6; drift guard matches real imports. | Merged |
| #5070 | All 8 nightly lanes → subscription (+ fallback account); sms-stress finally got the bearer its judge lacked | Merged |
| Sibling repos | Reviewers swapped to OAuth; ANTHROPIC_API_KEY secrets deleted | Live-proven |
| #5073 open | cache_control on the Clara loop's growing message tail, dashboard chat, and reasoning-brain; removed a Haiku cache marker that never worked (4096-token floor) | Open at time of writing |
One lane deliberately stays metered: intent-evals trace-replay. Its gate is measured against a metered-captured baseline, so moving it to the bearer drifts the comparison. Tracked on Trello 4t5QU5hu.
4 · Why caching was only 5%
Of 86 Anthropic call sites, exactly 3 set cache_control. Clara's agent loop cached only tools and the system prompt, then re-sent a growing message history uncached on every tool-loop iteration — up to 10 per turn.
Worse: volatile content (open-WO lists, renewal state, per-person notes) was interleaved into the cached system prefix. Clara doing her job invalidated her own cache. The $386/mo of 1-hour cache writes were, in the main, never read back.
5 · Projected path to −80%
Every dollar figure in this section is an estimate. None of it is measured yet — that's what the proof plan is for.
| Lever | Status | Est. monthly effect |
|---|---|---|
| CI / nightly / local lanes off metered | Shipped | ~190M-token spike days + nightly ~4% leave metered entirely |
| Caching tier 1 (PR #5073) | Open PR | est. $850–1,550 |
| System-prompt reorder (stable prefix, volatile tail) | Eval-gated follow-up | est. $250–450 |
| Tool sorting | Eval-gated follow-up | est. — unsized |
| 1h → 5m cache TTL decision | Measure first | est. $0–145 |
| Postcall extractor consolidation (5–7 calls re-send the same transcript) | Eval-gated follow-up | unsized, likely large |
Risk flag — AGENT_MODEL_DEFAULT moved to claude-opus-5 on 2026-07-28 ($5/MTok input vs Sonnet 4-6's $3). Uncached volume is 1.67× pricier in August, which raises the stakes on the caching work rather than lowering them.
6 · Proof plan
The baseline above is frozen. Proof is the same admin cost report (scripts/anthropic-cost-report.ts) re-run daily, looking for three things:
- Immediately: eval-day spikes disappear from the
propflow-prodkey. - Once #5073 deploys: cache-read share rises, uncached share falls.
- Target: metered run-rate at or below ~$670/mo equivalent.
Open items needing Fede
- Bearer token on the Vercel preview and development targets (plus rotation coverage) — this is what closes the preview lane.
- Per-surface key split in the Anthropic console, so attribution is one screenshot instead of an audit.
- Console confirmation of which keys the deleted sibling-repo secrets actually were.