award winning ai researcher (2 so far) | just graduated cs phd | author how to build conscious machines | musician | @bennettsrazor | i do not work @AnthropicAI
two failures really;
1. in interactive settings you really have two interpreters, rather than one, so complexity becomes subjective. see hutter and leike 2015, bad universal priors.
2. in that setting, if you allow for all interpretations, then everything is equally simple (see attached, from 2024)
You can use an LLM to code declaratively, like an old-school SAT problem. Define the boundary conditions, let it rip, pursue goal until satisfied. Unlike an old search algo, you can intervene and direct; you ARE the heuristic
Another thing: what you get from writing things yourself isn't just the code. It's an improved understanding of what the code does. That mental model is what lets you come up with further improvements, or invent a different way of doing things. You can't come up with ways to improve a blackbox you don't understand.
For most projects this doesn't really matter, because the code is the only thing you need. But if you're doing something novel, if you're doing research, the code is not the most important part. Understanding what the code does is the most important part.