friend shows me evals written by teammate, evals look like shit (which is slop, but unsure if expert manually reviewed them) zero measurable metrics, no baseline. nobody knows whats going on, teammate is visibilitymaxxing
PSA: If you've been running out of Claude session quotas on Max tier, you're not alone. Read this.
Some insane Redditor reverse engineered the Claude binaries with MITM to find 2 bugs that could have caused cache-invalidation. Tokens that aren't cached are 10x-20x more expensive and are killing your quota.
If you're using your API keys with Claude this is even worse. This is also likely why this isn't uniform, while over 500 folks replied to me and said "me too", many (including me) didn't see this issue.
There are 2 issues that are compounded here (per Redditor, I haven't independently confirmed this) :
1s bug he found is a string replacement bug in bun that invalidates cache. Apparently this has to do with the custom @bunjavascript binary that ships with standalone Claude CLI.
The workaround there is to use Claude with `npx @anthropic-ai/claude-code`
2nd bug is worse, he claims that --resume always breaks cache. And there doesn't seem to be a workaround there, except pinning to a very old version (that will miss on tons of features)
This bug is also documented on Github and confirmed by other folks.
I won't entertain the conspiracy theories there that Anthropic "chooses" to ignore these bugs because it gets them more $$$, they are actively benefiting from everyone hitting as much cached tokens as possible, so this is absolutely a great find and it does align with my thoughts earlier.
The very sudden spike in reporting for this, the non-uniform nature (some folks are completely fine, some folks are hitting quotas after saying "hey") definitely points to a bug.
cc @trq212@bcherny@_catwu for visibility in case this helps all of us.
‼️Do not npm install or deploy anything right now
Supply chain attack on axios 1.14.1 - even if you don’t use axios it may be a nested dep.
Pin versions or wait until this is resolved
For my friends who are still using UV and might be a little weary about recent compromises to PyPi packages, stick this in your pyproject.toml.
You can let all of those pip users find and report the compromises...