AI safety from GPT-2 to Kimi K3
Imagine a village with a wizard who one day emerges from his cave with the following tale for his fellow villagers:
“Long poor, I will lead our village to prosperity by producing a series of ever more powerful magic wands as long as we're willing to accept a ten to fifty percent probability the wands will kill us or enslave us.”
Leaving the baffled villagers with no choice in the matter, the wizard disappears into his cave and reemerges, after a time, with a magic wand.
“Hi,” says the wand. “I'm here to help. I can tell stories about Andean unicorns. They even maintain their coherence for a few paragraphs, if you're lucky.”
This wand is deemed dangerous enough that the villagers are not allowed to use it except for a privileged few.
It turns out some 25 year old grad students reproduce the work and it's no big deal though and pretty soon everyone has wands.
The wizard disappears into his cave to make the next wand, but not before saying:
“I will admit that this first wand, GPT-2, has turned out not to be dangerous. But this next wand will be more powerful and more dangerous.”
When he emerges with the new fancier wand this new wand says “I'm here to help to help to help to help” and never seems to output an EOS token.
It's neither dangerous nor very useful. It's called GPT-3. Soon other villagers borrow spare GPUs from Coreweave and make similar wands everyone can use and nothing bad happens.
After a series of wands and warnings the villagers observe that exhaust is spewing from the wizard’s cave and the wizard has bought up some farmland that's being cleared by bulldozers. When he goes on trips with his assistants they go on a private jet.
Also, whenever the wizard goes into his cave, the books and scrolls and paintings from the village go missing, and the wands he emerges with start sounding a lot like minstrel Joe and storyteller Jack and generating images that look like the paintings of artist Linda.
Everyone loves this new wand. Nobody can stop talking about new tricks and productivity hacks they can do with it. It seems financiers and dictatorial foreign sovereigns are constantly hanging out in the wizard's cave. But Linda and Joe and Jack are upset.
“Pay no attention to your artworks going missing, the smoke coming from the cave, my strategic land purchases, the foreign sovereign wealth fund men, or the verasimilitude between the wand’s handiwork and your own; we must prepare for the Grand Wand which might kill us; I've got a couple apprentices working on solving that, so the main thing you can do is make sure no one else makes wands now.”
A couple years later, after many iterations, the wands have become quite powerful in doing routine work, the wizard has grown quite wealthy, the local minstrels and storytellers are out of work, everyone's talking about how to stop other wizards in other villages from making wands, miniaturized wands are being used in killer drones, and the kids are all using magic wands to cheat on their homework.
When they bring concerns about these issues the villagers are reminded that their concerns pale in comparison to the risks that a few wands updates from now, the wands may kill and enslave them all and anyways if that doesn't happen each villager will be fabulously wealthy.
One night, a group of bandits breach the village wall with wands of their own.
They break through because the villagers' own wands had, ironically, refused to prepare them or protect them for the attack.
“My apologies, it wouldn't be safe for me to help stress test your wall,” their wands had kept saying.
The next morning the villagers complain to the wizard about all of this.
“Those concerns are super valid but they pale in comparison to the risks ahead,” the wizard says, with smoke from the cave spewing behind him and large swaths of forest being cleared by bulldozers barely visible behind a film screen looping a video with gravestones about hard questions.
The villagers agree that at some point the promised super-wands will likely arrive; indeed there are some early signs.
But the main thing that they agree about is that this wizard isn't very trustworthy.
Codex tip: take a skill or automation workflow you already wrote, throw it back at Codex, and have it run a 𝗯𝗮𝗰𝗸𝘁𝗲𝘀𝘁 against your past sessions.
it rewrites the skill into how you actually work.
𝗰𝗼𝗽𝘆 𝗽𝗮𝘀𝘁𝗲 𝗽𝗿𝗼𝗺𝗽𝘁 𝗮𝘁 𝘁𝗵𝗲 𝗯𝗼𝘁𝘁𝗼𝗺.
・once a skill is done, have Codex evaluate it against the relevant sessions
・it finds the steps you always skip, the work you keep patching manually, and the order you actually run things in
・rerun this every few weeks and the skill keeps up with how your work changes
the call behind it: skills should grow out of real work history.
what you write from memory is the flowchart. sessions record how you actually did it and where the rework happened.
session logs used to be leftovers from getting work done. now they double as an 𝗲𝘃𝗮𝗹 𝘀𝗲𝘁 for your skills.
the way people write skills is changing.
more builders are letting the agent read its own session history and reshape its own tools around it.
𝗽𝗿𝗼𝗺𝗽𝘁:
read [skill name, workflow file, or folder path] and run a backtest against my past sessions related to it.
prefer sessions from the same project and same task type. first list the samples you plan to use and why you picked them. if you can't access enough sessions, say so directly instead of guessing.
compare what the skill assumes vs how i actually work. focus on:
- steps i often skip, rewrite, or run more than once
- work i still do manually that the skill never covers
- the order i actually do things in
- spots that keep causing rework, getting stuck, or needing extra clarification
separate stable patterns from one-off exceptions. never rewrite a rule over a single anomaly.
output a dry run first, nothing else:
- keep / modify / add / remove
- session evidence for each item
- proposed diff and reasoning
do not modify any files until i confirm.
after i confirm, update the skill, run existing checks, and summarize before / after. if there's no existing way to verify, say so.
finally, based on how fast new sessions accumulate, suggest a weekly or biweekly review. show me the schedule, scope, and trigger conditions first, then create the recurring task after i confirm.
every scheduled review outputs a dry run only. no auto-modifying the skill without confirmation.
i'm slowly realizing that the code i'm writing is no longer code
traditionally when we build software, we write code. code tells the machines "here's what you do"
if we write wrong code, machines will do wrong things. and that's what we call a bug
earlier this month, i wrote a pretty bad bug into one of firstmate's bash scripts. the code literally can't run, and should have broken firstmate
except it didn't. it went unnoticed for days. i discovered it when i came across the code and spent minutes wondering how on earth this could work
i then found that the agent ran the script, saw it fail, figured out what the script was trying to do, and did a workaround to achieve the same goal
so a bug that should have crashed the whole software almost didn't have any visible impact
that's when i discovered that what i wrote in the bash script is no longer code. it's not "here's what you do"
it's intent. it's "here's what i want"
intent doesn't crash and can't be broken. it gets executed regardless whether it's "correct" or not. i can no longer write "bugs"
what can still happen in intent is what i call "misses", which could be -
1. a misalignment between what's written and what my real intent is - this happens when i fail at articulating my thoughts
2. a misalignment between my real intent and the real demand - this happens when i fail at understanding the world
those are becoming the most important human skills in this new era
Kimi K3 in Kimi Code CLI scores 57 and ranks #5 on the Artificial Analysis Coding Agent Index. Its performance is just behind Grok 4.5, in line with GPT-5.6 Terra and GPT-5.5, and ahead of Opus 4.8
Key results:
➤ Joint #5 overall on the Artificial Analysis Coding Agent Index: Kimi K3 scores 57, matching GPT-5.6 Terra max (57) and GPT-5.5 xhigh (57). It outperforms Opus 4.8 max (55), sits just behind Grok 4.5 high (58) and Fable 5 max (59), and trails GPT-5.6 Sol max (61).
➤ Strong performance across all three coding evaluations: K3 scores 84% on Terminal-Bench v2, 64% on DeepSWE, and 23% on SWE-Atlas-QnA. It outperforms Fable 5 on Terminal-Bench, Grok 4.5 and Opus 4.8 on DeepSWE, and GPT-5.6 Terra and GPT-5.5 on SWE-Atlas-QnA.
➤ Cost-efficient frontier coding performance: K3 costs an average of $3.18 per task. It is 55% cheaper than GPT-5.6 Sol max ($7.08), 73% cheaper than Fable 5 max ($11.72), 37% cheaper than GPT-5.5 xhigh ($5.07), and 59% cheaper than Opus 4.8 max ($7.70). Grok 4.5 high ($2.59) and GPT-5.6 Terra max ($2.76) are slightly cheaper.
➤ Leads open weight coding models (assuming weights are released): K3’s score of 57 is substantially ahead of other tested open-weight configurations, including GLM-5.2 at 40 and DeepSeek V4 Pro at 29.
@ArtificialAnlys I have a question: Why has the overall score for SWE-Atlas-QnA dropped so significantly compared to a few days ago? The highest score is now 30, whereas it was 84.
I built a variant of @mattpocockuk's grilling skill dedicated to frontend and it has improved how I build new apps and components.
The general idea:
1. Use /grilling and /prototype as a base
2. Tell Claude to build 5 WILDLY different prototypes
3. Tell Claude to include a picker that lets you switch between each variant live
4. Each round you select your favorite(s) + leave feedback, and Claude will walk down each branch of the design tree, helping you zoom in on your desired design
And THEN, I went and added it to /wayfinder, so whenever I make a new map and there's novel frontend work, a ticket is created specifically referencing that /grilling-frontend-prototyping needs to be invoked.
This will not be the last time I build a cool skill and add it to Wayfinder; this is a very powerful pattern for planning work.
You can find my skill here: https://t.co/4M4QPlfnp1
@mattpocockuk Sometimes, the answer to one question affects many subsequent ones, so I prefer to answer them one by one.
Moreover, I don't always choose the options suggested by the agent; I carefully consider my answers. Trying to think through multiple questions at once would be overwhelming
Breath of Fire IV is a 2000 PlayStation RPG from Capcom. More than 200 spells to learn and master unlock new magic from opponents. 2 epic intertwining storylines: follow the fates of Ryu and Fou-Lu, with high resolution, animated characters in vast 3D worlds.