Had asked Claude Code to do some research on some computational benchmarks. Wrote a summary that included some references to some papers. The final analysis had some, let's say, "questionable" statements in it.
I asked it to go find the paper and actually read it.
Its response: "Holy shit, the original search-result summary was completely wrong about this paper. Reading it for real...That number was hallucinated from a search-result snippet..."
I also read the paper...round two was on the money, round one was completely false.
LLMs can be great for research when they actually, you know, do the research...
“ARC-AGI-3 focuses entirely on evaluating fluid adaptive efficiency on novel tasks, while avoiding language and external knowledge…Our testing shows humans can solve 100% of the environments, in contrast to frontier AI systems which, as of March 2026, score below 1%.”
There's an ongoing battle in every developer and on every development team between whether to rebuild/refactor/rewrite vs not. It all stems from the goal of fixing "bad" code and doing things right.
Joel Spolsky's classic post, Things You Should Never Do, Part I, talks about avoiding rewriting code. His take is heavily against the rewrite. And I think he's right...for the most part. I know, who am I to say that I disagree with Joel Spolsky. He's significantly more accomplished and experienced than I am. But there is a distinction he doesn't make that I think matters.
I split "bad" code into two types.
1. It works but isn't the best (ugly code)
2. It doesn't work as intended (wrong code)
Spolsky's post focuses heavily on (1), the ugly code. I want to talk about (2), the wrong code.
For ugly code, I've had to accept that it's better to improve with small changes over time. It works and should be allowed to continue to work.
For wrong code, where the code really doesn't do what it needs to, I lean heavily into the rewrite. In order to properly diagnose the problem, you have to dig deep. And if you stop too early, you're building on unstable foundation or unnecessarily inheriting patterns from the approach that doesn't work.
But there's a trap...you will find ugly code while you're reading the bad code. You'll want to rewrite it...don't. It's so easy to end up overdoing it, taking out things that work by justifying that the ugly code is part of the wrong code. That's what ends up having a 6 week refactor end up being a 12 month rebuild.
If it's wrong "bad", fix it, and fix it right. If it's ugly "bad", just leave it alone.
Everyone is focused on ship, ship, ship.
Everyone is focused on MVP.
Everyone wants to get out the door as quickly as possible.
You sacrifice so much by doing that. You end up coupled to an exponentially complex network of services that you have to manage indefinitely. You have tech debt you will never be able to get out from underneath. You released quickly but you're fragile forever.
"Good things take time," is a cliché for a reason.
Everything worthwhile doing is hard.
The final 10%, 5%, 1% get exponentially harder. But it's those minute details, that last bit of polish, that makes the difference.
People can tell.