Like a human, they can be stubbornly wrong and blind to their mistakes. Yesterday I had one generate fundamentally incorrect formatting in an end user doc. Unusable. I explained why it was wrong. It generated the same wrong doc. I explained that it hadn’t understood my correction. It generated the same wrong doc. I simplified the input format in hopes of helping it out. It generated a simplified wrong doc. I swore at it. It generated the wrong doc.
A week earlier, it found and fixed a logic error deep in an architecture layer I didn’t write, using technology I barely knew, solving a problem that has vexed me for three years.
They’re idiot savants.
Your experience is highly unusual. It sounds like you're operating on existing, refined codebases, which certainly produces different results than a new project might. In general the quality of the output of even the best models is still below that of the worst engineer you've ever worked with *unless* it's something that's very common in the training data... like all of your code is. The LLMs make all the same mistakes that you'd expect from your most boneheaded junior dev. To the point that it's uncanny. The most obvious example is just silencing an error rather than fixing the underlying problem. The difference is that I was never quite able to convince my junior of what was wrong about doing that ("But I got rid of the error didn't I?"), whereas the LLMs immediately respond with the silicon equivalent of slapping their forehead and saying doh! and then do it properly (and then proceed to make the exact same error when performing the next task).
TFW you get some random email about IBM and you log on to check earnings and the stock is off 25%. A rare moment of honesty from the CEO here: "These conditions require our teams to execute perfectly, and this quarter we faltered." https://t.co/GQhLau7w6Y
The AI market is changing fast, with customers turning to low-cost open-source models to save money while the frontier LLM makers try to raise prices - @ThomasClaburn reports
https://t.co/milPye34xg - tip @Techmeme
OpenAI hasn't held pre-IPO investor meetings or set timeline yet, sources say -huh, almost like it isn't planning to go public at all! Just trying to change the narrative or something. Huh. But companies don't lie to the press so. https://t.co/3r7lZBoeik
Kīlauea Eruption Update — The Big 5-0! Episode 50 of Kīlauea summit lava fountaining began at 10:10 a.m. HST today, June 27, and is ongoing. This is the 50th lava fountaining episode in the eruption that began in Kīlauea summit caldera in Hawai‘i Volcanoes National Park on December 23, 2024. The lava fountain from the north vent is reaching an estimated 600-700 feet high above the vent. According to the National Weather Service, surface winds below the inversion level (about 8000 feet or 2400 meters above sea level) are forecast to be moderate to strong tradewinds out of the northeast, which will move the lower part of the plume to the southwest and result in tephrafall in that direction. Above the inversion layer, very light winds are forecast up to 18000 feet (5000 meters), which might allow the plume to spread out. Above this, winds will become more westerly and strengthen. Higher level winds could push parts of the plume to the east and could result in ash and Pele’s hair falling to the east. Fountaining episodes typically last 12 hours or less, but ash can remain in the air for longer depending on wind and weather conditions. Please stay aware of hazards and rely on official updates from USGS, National Weather Service, and Hawai’i Volcanoes National Park. 🎥 Video of episode 50 on June 27, 2026. #Kilauea #Eruption #Lava
Meta is recovering DDR4 memory from old servers, installing it in new machines, and using a custom CXL ASIC to share the memory across applications – without encountering latency problems. We found the paper ahead of planned presentation: https://t.co/z0HtBE8XVK - tip @Techmeme
One man, two kernels, and a lot of RISC-V
https://t.co/Cfk9XWJzAU
A homebrew PC and mini-mainframe were only the warm-up for Yuri Zaporozhets' latest operating system
<- by me on @TheRegister