Sure, here are the raw counts from the API usage logs (full table in the image).
Claude Code (Opus 5.5), whole run, 757 API calls:
- input (uncached): 1.5k
- cache write: 1.9M
- cache read: 154.9M
- output: 598k (incl. thinking)
- total: 157.4M
Codex (GPT-5.6 Sol, high reasoning) for the native part of #9:
- input (uncached): 630k
- cache read: 31.7M
- output: 108k (incl. 55k reasoning)
- total: 32.5M
~98% of all tokens are cache reads. Every call re-sends the full context, and nearly all of it comes from cache.
What it would have cost at API list prices instead of a flat subscription:
- Claude Code: ~$58 (cache reads $31, cache writes $15, output $12)
- Codex: ~$17
- Total: ~$75 for the whole CTF
#9 alone was ~$29 across both tools, #2–#8 together ~$43.
#2–#6 shared one session and are split by prompt timestamps, so treat those per-challenge numbers as approximate.
No tuning on my side: default settings, no context optimization, no caching tricks. Just a short CLAUDE.md with my tools. So this is a baseline, not a best case. Would be curious to see other people's token counts, especially anyone who solved it with a leaner setup.
Flare-On used to take me several days. This year it took an afternoon.
Challenge 1 I solved by hand, the other eight with Claude Code running Claude Opus 5.5 and one with ChatGPT Sol 5.6 when Opus was stuck. What I took away:
1. No special harness. A short CLAUDE.md with my preferred tools and how to drive them was enough, with GhidraSQL as the workhorse. The capable model did the rest.
2. The instructions grew with the run. After each challenge, the lessons went back into CLAUDE.md. One example: use the real reverse engineering tools instead of rebuilding them in Python, which the model likes to do. Another: log dead ends too, because those are the expensive part.
3. Refusals happened. On two challenges Opus 5.5 refused to continue. I finished one with Sonnet 5, the other with Opus 5 and then 4.8.
4. So did getting stuck. Twice Opus 5.5 hit a wall. Once a hint from me was enough. The other time I switched to ChatGPT Sol 5.6, which burned a full day's token allowance but solved it. The remaining seven took about 1.5 days' worth of Opus token limits, typically 8–9% of a 5-hour usage window per challenge and around 25% for the hard ones.
5. The expensive mistakes weren't reasoning errors. They came from tool choice and missing folklore. An hour went into a full auto-analysis of a huge binary, and 25 minutes passed before the agent tried a password every CTF player tries first. Experience still mattered, just after the fact, as rules in the instructions.
6. The agent took a convincing decoy at face value. Only submitting the flag exposed it. Without external validation, nobody would have noticed.
7. It fetched its own tools, within bounds. When a format wasn't supported, it proposed a Ghidra plugin and rebuilt it in about five minutes, asking first as instructed. Apart from two cases I explicitly approved, it never ran challenge code on my host: static analysis and emulation only.
8. Days became hours. What used to take me days, sometimes one or two weeks, took 3–4 hours, with very little interaction.
9. My knowledge wasn't really needed. The agents solved the challenges autonomously. I learned little about the underlying reverse engineering this time. What really counted: what to put in the instructions, when to step in, when to switch models.
If you want to learn from an agent's work, make it document its findings and its approach, including the dead ends. Otherwise the knowledge disappears with the context window.
That leaves the big open question: what do we as humans still need to know and take away, and what do we leave entirely to the AI?
#FlareOn #ReverseEngineering #AI #Cybersecurity
“Not powered for mortality” is a fair point for individual trials. But “nothing to conclude” from pooled mortality data or a null dose-response meta-regression is too strong.
More importantly, the diabetes claim is simply inconsistent with randomized evidence. The 2024 CTT individual-participant meta-analysis states that statins cause a moderate, dose-dependent increase in new diabetes diagnoses, with small increases in glucose and HbA1c. Yes, many cases occur in people already near the diagnostic threshold, but that is not the same as “revealing pre-existing diabetes.”
And “other side effects aren’t more prevalent than placebo” is also too broad: blinded RCT meta-analyses show small but real excesses in muscle symptoms and liver-enzyme abnormalities, with stronger signals at higher intensity.
Statins can still have a favorable net benefit. But denying measured harms is not evidence-based medicine either.
🤔 Interesting definition of “evidence-based medicine.”
2.3 percentage points fewer major CV events over ~6 years.
No significant reduction in all-cause mortality.
No significant improvement in death/dementia/disability.
And, if we frame harms the same way you frame benefits:
+22% musculoskeletal adverse events
+57% diabetes-related events
+100% hepatobiliary adverse events
See how impressive relative percentages suddenly sound?
That’s exactly why benefits and harms should be reported as absolute AND relative risks.
Cherry-picking only the favorable RRRs isn’t evidence-based medicine either.
RRR is apparently good enough for benefits, but not for harms. Interesting standard.
Shai-Hulud is back. 🪱
A massive software supply chain attack just tore through npm: 2K packages compromised with hundreds of millions of combined downloads.
🚨 Notable victims include flat-cache, file-entry-cache and packages under namespaces [@]workbench-stack and [@]adminide-stack.
Malware records from RL's https://t.co/nuVv6DoVR8:
https://t.co/Od8ugBK44g
https://t.co/6qqr3NDa7C
https://t.co/kwXyndS6O1
https://t.co/ohkRc11nBn
Full research/analysis to come...
🚨 NEW from ReversingLabs: Copy, Paste, Compromise: The Tale of ClickFix
Full #ClickFix attack chain research + an open-source #YARA rule validated against 4K+ samples — incl. 123 lures that evaded every #Antivirus engine.
Detect the lure before the user acts: https://t.co/qStZxwH7xl