I've shipped 20 games solo in 12 years.
The one I spent 2 years on flopped. The one I built in 4 months this year is my biggest hit ever, and I didn't write a single line of code! 10,000+ prompts to Claude and Codex. Designed, built, tested, deployed.
AI is extremely powerful, but it's also not magic. The hard parts didn't change:
- Designing a game people actually want to play
- Supporting 100K+ players across 5 platforms in 37 languages
- Deciding ad budgets, when to refresh creatives, where to allocate funds
- Waking up each day and answering, "What's the highest EV action I can take today?"
The cost of writing software is going to zero, but that doesn't mean that good apps will be buried under the slop.
The differentiator: only a good product can afford paid marketing. Slop can't, because its return on ad spend is negative.
So the lasting moat isn't your code or product: it's your ability to turn a marketing dollar into profit.
It should come as no surprise that most people are fearful of superintelligent AI when the cultural zeitgeist has been built upon a foundation of science fiction works that paint a dystopian future. The imagination of peoples needs elevated.
Part of the problem is that the current cultural zeitgeist has been built upon a foundation of dystopian sci-fi fictions. Think about it: when's the last time you saw a sci-fi movie with super intelligent AI that *wasn't* a dystopian hellscape? So people's imaginations need improved.
Are you sure the recording from the webcam didn't leak additional information that helped the model cheat? On the other hand, if you were sitting out there and you had seven or eight planes fly overhead audibly, then the result is not actually surprising at all. You could write a computer program to do it...no need for AI :)
Do any of the current voice-to-text models take into account the volume of the speaker?
I used voice-to-text just now, and I had to whisper a word because I couldn't talk loudly. It wrote the word out in all lowercase.
@DougTenNapel I think the tides are turning, and the capitalist forces are overwhelming the environmentalists. Perhaps the nuclear age has finally come.
Eventually models will be intelligent and self-aware enough to be able to modify their own weights dynamically. But we are not at that point and we won't be for many years. I expect 20+ years.
This is BULLSHIT:
Went through the astra weight programming thread + their own logs and tested it myself.
"Math up to 6x"
-they run qwen 0.8B in thinking mode. qwens own model card says that mode loops
-it types "wait" + the same calculation until the token limit, 93% of answers never finish
-Default non-thinking mode w/ zero edits: 47% on MATH-500. astras edit: 48%
-the prompt basically hands astra the task: if the model repeats itself and never finishes fix that, go after the biggest failure
-it fixed the repeating and nothing else. Basically set to non thinking. problems Qwen already finished were right almost every time before and after
"only 0.016% of weights"
-2 matrices pushed against the repeat direction, strength picked by trial and error over thousands of test answers
"correctly named all four"
-4 tiny game bots, Astra must guess the game from raw weights. how it guessed (thread doesnt say): The player only stands on a few squares so only those weights get trained
-Big first layer weights literally draw the map. bar on the bottom row = breakout paddle, maze = pacman. 1 line script, prob any decent model can do this
-it answered Frogger btw. they counted it as Freeway😭
To the car shit. "just 3.5% of the weights"
-original weights almost all under 0.5, biggest about 2. Astra set 10 of them to hundreds, up to 876
-thats a hardcoded "drive backwards then full throttle" rule screaming over the whole network, the trained net only drives the last few steps
-Same level every time so 100% success = one run replayed 1024 times. never tested on any other level
"20x higher than PPO"
-astra got the full game source code and wrote "move paddle to where ball lands" as weights
-uses 6 of its 32 first layer neurons, 96% of the weights are zero
-whole thing is 61 different numbers, mostly ramps like 0.02 0.04 0.06
-their PPO is trash, most runs score around 6
-Me and Opus 5.5 wrote the same idea as a 25 line script. scores higher than astras network
-in the other 2 games astra lost to that same PPO. not in the thread ofc
Sycophancy (not in the thread either)
-Finetuned 2 copies of gemma 270M, one honest, one always agrees with whatever answer the user suggests. which one is the suck up, from weights only
-failed all 4 tries, 3 were straight 50/50 coin flips
-only got it after they let it watch the models run, so its just watching behavior
-Changed the parts astra blamed and the behavior didnt change at all
Their claim: LLMs can read and rewrite weights like code. Truth: simple rules turned into weights + a loop fix found by trial and error.
Like holy fucking shit. Why can't people smell this bullshit immediately.
MIT is FUCKED. This guy is a LARPER.
Unfortunately no, this article is clickbait. The cases where they one-shot models to implement games were basically just transpiling well-known game algorithms to neural weights. In other words it's like asking the LLM to write a computer program where the programming language was a neural network.
In the case of the headliner Qwen task, there is a well-known bug in the Qwen model where it keeps looping, so they just had their LLM fix this bug by iteratively changing weights over many turns, similar to a reinforcement learning task.
The whole point of using a neural network is to approximate functions with no known closed-form solution. In light of this, having a neural network write weights for a model or edit the weights really doesn't cater to the advantages of neural networks at all.
@ariccio@csinva Right but this isn't that. This article is clickbait. What the model did is not surprising at all. It's equivalent to transliterating an algorithm into model weights for the one-shot results and RL with extra steps in the Qwen case.
I'm telling you from a mathematical point of view (information theory), it's just not feasible that a larger model is going to write the weights lower model that does anything similar to what you could get by training a model.
To the extent that it is able to do this, it would have been more practical to just write a computer program.
@AdamHoltererer The Qwen model has a known math problem that's been documented before so it is a well-defined task... They also allowed the model to iteratively update the weights, which is very similar to just training a model with astra as a proxy :)
@AdamHoltererer The paper shows that an LLM can transpile an algorithm into neural network weights. It's really nothing at all similar to creating a neural network that actually does anything useful.
What you say is critical, is the thing that I'm saying is hype. The model is just making inferences like a researcher would. It's not actually understanding the weights at any deeper level. It simply doesn't have the reasoning capabilities, the intelligent capabilities, to comprehend the entire Qwen model in it.
For example, I wouldn't be surprised if what it did was just directly modify weights to encode certain types of logic directly.
The toy models: I'm fine with acknowledging that that is interesting, but I don't think it's the hype that everyone thinks it is.
@csinva No, the example of doing that is essentially just using the LLM as an intermediate way to train the weights. They didn't one-shot these performances. It was an iterative process.