Ryan McConville

CTO & machine learner building real time AI code review at | senior lecturer in AI at University of Bristol | prev @wandb

[Ryan McConville @ryanmacconville]

2025 AI coding takeaway: models are getting better, but they still ship enough bugs that automated code review is worth having in the loop every day. 9/9

[Ryan McConville @ryanmacconville]

we build an automated code review tool, so we get a unique view of how AI-generated code behaves in real workflows across IDEs and models. insights from our 2025 sample.

bug rates vary widely across models: 2.7 to 19. 5 bugs per 10k lines (7. 2x spread). 1/9 🧵

[Ryan McConville @ryanmacconville]

security, logic, and quality issues: highest observed rates were Claude Sonnet 4 with ~4.2 security, ~5.1 logic, and ~2.1 quality issues per 10k lines.

for reference, Sonnet 4.5 still shows security + logic + quality totaling ~6.5 issues per 10k lines. 8/9

[Ryan McConville @ryanmacconville]

Spellchecking did not start as a subtle, always-on feature. The early spell checkers were basically batch programs: take a document, compare tokens against a dictionary, print a list of suspicious words

On PCs in the 1980s, spellchecking becomes more mainstream, but it is still mostly a deliberate step. Many systems either shipped as separate utilities or ran in a proofing mode inside word processors that you had to run yourself.

What is easy to miss is why it changed. I think that a major leap was not just better detection of mistakes but the innovations to make spell checking fast enough to feel interactive: compressing dictionaries with affix stripping (store stems plus prefix and suffix rules), keeping a frequent word list in (limited!) main memory to avoid slow disk accesses, and using hashed / bitmap based membership tests to quickly answer "is this a word?" without storing the full wordlist verbatim.

In other words, once it got fast enough, the interface flipped. Spellcheck moved into the background and became continuous. When the editor can underline mistakes while you type, you stop thinking of it as a separate tool and it becomes part of writing. Microsoft Word made this style mainstream in the mid-1990s, right around the Windows 95 era, when I was coming of age as a young computer nerd.

I think automated code review hasn't yet made this shift. It is treated like a batch job, and the latency is high enough that people naturally push it to the end.

I wrote a short blog post about what changes when review is fast enough to stay in flow, and why timing matters as much as what a tool finds.

That work also enabled a feature we just shipped at https://t.co/oqB8fvkYZh: Ambient Background Review in your IDE. It runs continuously and only interrupts when there is actually something worth saying. Like spellcheck in Microsoft Word in 1995.

At the risk of sounding like this post is written by AI, I think this is a game changer.

Attached picture of the Microsoft Windows 95 manual that is usually on my bookshelf, which I recently found myself flipping through again.

https://t.co/mhaYiN39ol

[Ryan McConville @ryanmacconville]

@theo AI code reviews on your PR are for cavemen. reviews should be instant, in your IDE, or better yet, integrated into the code writing agent like a pair programmer.
https://t.co/g4wtz2vy7q

[Ryan McConville @ryanmacconville]

If you look at how developers work today, something interesting has happened. Writing code has become incredibly fast. AI tools changed that almost overnight. People are producing more code than at any point in the history of software.

But the rest of the workflow did not speed up with it. In fact, some parts slowed down. When you generate more code, you also generate more to review, and the review step starts to feel heavier. PRs get larger, context switching gets worse, and teams end up with a strange situation where the easiest part is writing the code and the hardest part is checking it.

This is the problem we have been working on with https://t.co/oqB8fvkYZh. Most code review tools were designed for the pull request stage, which means they only look at the code after the fact. Some newer tools have started moving review earlier, closer to where the code is written, but they still feel like a separate step. We wanted something different. We wanted review to happen the moment the code exists, and for it to be fast in a way that would reset expectations. We regulararly see it close to a hundred times faster.

It works for all code, human written or AI generated. And because it learns the patterns already in your codebase, it keeps everything consistent in a way that feels natural rather than enforced. It also catches the subtle mistakes that appear more often now, even when an AI assistant does something different from what the developer intended.

I think this shift toward earlier review will become normal. It removes friction. It keeps quality high. It fits the new pace of software development.

[Ryan McConville @ryanmacconville]

spent hours debugging why i was having mongodb connection issues, timeouts pushing to aws us-east (claude helpfully suggested migrating to a aws eu region), accessing docker hub and debian repos.. turns out my ISP is under attack.

happy friday 🙃

[Ryan McConville @ryanmacconville]

someone cloned our VS Code extension, swapped order of two words in the name, injected suspicious code, and bought fake installs to outrank. we reported it over 24 hours ago using official channel yet still live on VS Code marketplace…

how is extension squatting with injected code not an immediate takedown? has anyone else dealt with this?

real: klusterai.kluster-verify-code
fake: klustfix.kluster-code-verify

@code @burkeholland @msftsecresponse

[Ryan McConville @ryanmacconville]

@finbarrtimbers have you tried any AI based code review tools?

[Ryan McConville @ryanmacconville]

it's great to see that pebble software is now 100% open source. it was my favorite smart watch ever -- it was the right balance.

it was also the first (and last) piece of tech that I bought from a vending machine

https://t.co/06NYQiUC4p

[Ryan McConville @ryanmacconville]

@EthanL_v5 yeah, review should always follow the code gen stage. even the best writers need an editor. same thing applies to ai coding.

[Ryan McConville @ryanmacconville]

@HelloAlberuni yeah, we see this a lot. the bottleneck has shifted from code writing to code review (where increasing amount of time is being spent and problems found).

[Ryan McConville @ryanmacconville]

@ethanmonkhouse @vitruvius_ai what do you think about ai code review? we built https://t.co/pVfWOKpKJr because we encountered the exact problem you mention. bottleneck has simply shifted downstream now from code gen to code review and ultimately delivery. productivity gains tend to neutralize otherwise

[Ryan McConville @ryanmacconville]

@GradyGaugler we built something like this at https://t.co/pVfWOKpKJr - a multi agent sys that does real time code review. one thing that differs from your tweet is that we don't focus on the PR stage, instead we do it in the IDE, as the code is written. I think better suited there than PR.