LLMs are good at finding exploits and bad at system design, which captures their general strengths/weaknesses: good at relentless repetition of bounded tasks; bad at holistic understanding. Good at producing working code (including exploits); bad at knowing what is "good".
Nathan: Do you see why this is funny?
Elizabeth: Because all these certificates have to be revoked?
Nathan: Yes. But do you understand why 4 is a terrible secret key?
Elizabeth: Well... it's equally likely as any other value drawn from the uniform distribution.
When an LLM says:
"Don't turn assumptions implicit in an everyday scenario into explicit input restrictions"
I have some intuition for how it got there instead of just writing:
"Don't state obvious assumptions about inputs"
and that scares me a bit.
the AI era has brought a new world of pain regarding "do I need to hit shift+enter or alt+enter to insert a newline", where choosing the wrong one often submits the text instead of inserting the newline, and I am genuinely mad about this in real life
I've been back on codex this week for reasons, and it makes me miss pi. Pi is simple, has very little UI jank, and it doesn't tell me to "visit the community forum" and other nonsense. It waits for me, then does exactly what I said, which is how terminal tools should work.
This is my experience too. It's a huge, obvious effect. Now put this alongside the utterly confident claims that "AI writes better code than you ever did." https://t.co/f7Qw9roc4F
I always listen to the confident, loud declarations about the future of programming (don't want to get left behind)! That's why I always start with a UML diagram, then let tooling generate aspect-oriented WS-* services, with the data stored in a network database.
blockchain
I fully appreciate the feeling, and I feel it too, but which "value"?
"What will someone pay me a wage to do?"
"What do I enjoy doing?"
"What will improve other people's lives?"
"What will reduce the probability of Dijkstra haunting my home?"
https://t.co/s6Vy8gJQfB
I spent 25 years getting good at web dev.
I memorized syntax, browser APIs, accessibility techniques, and performance optimizations, security patterns, and learned dozens of libraries and frameworks.
Now I'm trying to decide what knowledge still has value.