You really, really need to be more ambitious
Things are gonna get really strange over the next year, and if you're not using these new models to do bigger greater things your going to have a very very bad time
Working on my Fable 5.2 vid rn, the runs keep going longer and better than expected and it's making me believe this even more. We have barely scratched the surface of what these things are capable of...
what the hell did they do with gemini 4 neon to make theo actually like it
seriously how tf did we go from the gemini carbon experience to him texting me this
Capability = "at it's best, what can it do"
Coherency = "does it understand what it's doing"
Best method I've come up with to explain the current gap in models. Different model families have never felt so different before
I've been really harsh on GPT 6.2 Astra and I stand by that. The behavior is still really weird, it's communication is painful, and it's quite lazy
It's also really capable. I just finished recording my video on it and felt weird. I bitched about it a lot, but also the stuff I've built with it is insane:
I've got a giant box of parts it sourced and ordered for me sitting in my living room, my 3d printer is working on building out the full body, I've got a custom physical recorder that it fully designed and printed, massive solid custom dungeons in majora's mask, an among us simulator bench (that fable did have to go cleanup it made a lot of mistakes), some crazy 3d models, and the foundation of a game that fable has taken off with
It's a hyper capable model, it's just that those capabilities are locked behind such a shitty experience. I look at the outputs now and I'm impressed, then I go back and look at the threads and use it more and get pissed off fast when it randomly stops working or asks me the most insane bullshit in a different language
That's the frustration I and others have with this model. I wanted this to be the Opus 5.5 killer, and it's not which just entirely poisons my feelings on it. The experience of using the model is at this point equally important as the raw capability (because u will never get to it's limit if pushing it is a chore)
For this generation I think for me the openai models are getting relegated to reviewer and computer use jail. I just bought my 4th claude sub, it is still just better. Hopefully the next model makes it effortless to get these insane things out of it, and I can finally go back to primarily driving codex
I've been accused of being paid off by OpenAI for over a year straight. ngl, I honestly thought my transparency around GPT-6.2 Astra would help tone that down.
Instead it just flipped to "paid off by Anthropic"
Never underestimate the stupidity of the average poster.
I took this to heart and spent the last few hours doing another deep dive on GPT-6.2 Astra
tl;dr - things are better than when I tested, but IMO still "directionally, very bad"
I'm going to keep using the same example I've hyperfixated on since the start: "why is creating a worktree so slow?"
I had a long thread trying to diagnose perf issues while using t3 code on a particular VPS. 2 of the 3 were fast, 1 was really slow.
I asked astra outright "why is this so slow?", and it said a bunch of gibberish. I followed up with "I feel like I'm going insane, can you just tell me the cause of the slowness?"
I'll give you the answer upfront: this VPS uses a virtual disk that is slow as shit, my other VPSs have physical NVME drives that are faster.
Image 1 is from before the fixes Tibo mentioned. It shows an unnecessary graph, focuses too much on pnpm and file systems, bolds irrelevant text, and brings up ram just as much as it (subtly) mentions drive speed.
Image 2 is of Opus asked same thing. It opened with the important part in bold: "Slow disk, made worse by VDO". Ideal!
Image 3 is 6.2 Astra post-fixes Tibo mentioned. Better? Absolutely. Much more concise. Sadly, it still focuses way too much on second-order problems that are a small % of the cause. "XFS over a compression and deduplication layer on a virtual disk." - so close, but still a miss.
I think this perfectly encapsulates the problem I have with 6.2 Astra. It doesn't miss anything. It's SO thorough. But in it's thoroughness, I find it misses the most important thing: the actual point.
It wants to know everything, and then it wants to share everything. It doesn't understand what is and isn't important.
Did it say anything wrong in my screenshots? Not really. But IMO, it still entirely failed to miss the point of my question, and gave an answer that was too focused on accuracy and not enough on clarity.
In the wise words of Michael Seibel: "Most aim for 100% accuracy, and they end up with 50% clarity. Aim for 100% clarity, 80% accuracy".
Silly way of putting it, but I genuinely wish OpenAI models would lie to me a LITTLE more so I could better understand the vibe, instead of having my time wasted by all these useless details.
the difference in how the two have handled reasoning has mattered infinitely more than I expected
OpenAI went so hard on efficiency that they made their models miserable to work with (and also to understand for monitoring/safety stuff)
I have zero clue how hard it is to reverse out of that or even if it's important to do so tbh. Completely above my paygrade or understanding lmao
Thus far Anthropic has always been the one to make the big step function jumps that change how we can use AI:
- Sonnet 3.5
- Opus 4.5
- Opus 5.5
Usually others catch up and often surpass them (like how GPT overtook Claude last spring), then they jump again.
I'm expecting this to repeat (OpenAI or someone else will crack the insane usability + capability combo Ant got with Opus 5.5) sooner or later
I was really, really hoping that would be GPT 6.2 Astra but it's just not. I would however bet money that the next big GPT release will be almost entirely focused on being the best possible experience to interact with + good behavior
As long as this cycle continues as it has I think things are fine, but I need to see/feel someone actually catch Anthropic as capabilities continue to skyrocket or things are probably gonna get really shitty. Anthropic's been on their best behavior since OpenAI became a much more credible threat this year, that must continue next year.
An Anthropic not suffering from competitive pressures would be horrible for the world...
Why are they all posting reviews at the same time: there's an embargo.
Why are the reviews all negative: the model is bad.
It's pretty simple, guys.
There's no conspiracy. We're just giving our opinions.
On a more serious note this is the reason we love OpenAI so much
The fact that they can give us early access to a model, be super receptive to very brutal candid feedback, and I don't feel any stress putting out a scathing review is remarkable
(also look at how aggressive all the early access testers are being about the model being bad, that is not normal for this kind of thing when dealing with a trillion dollar company)
They have great people running things behind the scenes and really are one of the best companies to work with that I've ever experienced
They definitely know what the problems are, hopefully they can bring things back with whatever they make next
Everyone thinks Gpt 6.2 Astra is bad... but Theo and Ben are still arguing over it
0:00 Three New Titans
3:30 Gemini 4 Carbon Is Slow
7:23 Mythos Security Surprise
12:50 Google's Death Timer
21:08 Fable 5.2 Subscription Math
29:55 Fable's Shortcuts
46:49 Astra Crash Out
50:53 Astra Can't Explain
59:16 Fable + Opus Build a Game
1:09:58 Why OpenAI Is Losing
I'm doing ~30-40% of my work entirely from my phone now
T3 Code and Grok Bot both have excellent apps on the duo, to the point where over the weekend while I was at Twitch con I got a ton of actual work done (burned down 3 $200 claude sub weekly limits) entirely from my phone while being out doing other stuff
I'm at the point where when working on a project one of the first things I do is get the verification/dev system setup to work perfectly on the duo so I can test stuff on the go and merge PRs without having to go to my computer
Since getting my iPhone Duo, I keep falling more in love with it.
Wasnβt sure if I would daily drive it. Now I cannot imagine going back.
Favorite piece of hardware Iβve gotten in years. Last time I enjoyed a new gadget this much was AirPods Pro.
Even though lots of folks are hating on Astra 6.2, this is comforting to me.
It means that when folks say they like it, they mean it.
And when they dont like it, they also mean it.
I appreciate people who get early access not being forced to withhold negative thoughts.
I was saving this for my video but I'll spoil it now.
Ben and I both made new levels in games using 6.2 Astra during the early testing. He did Majora's Mask, I did Outer Wilds.
We both prompted ourselves, but it was along the lines of "make a creative new level for [game]"
We both got water levels with music themes, specifically around bells.
Both were named "Bellwether".
This model regresses to the mean like nothing I've ever seen.
Go ask it to make a new level for a game you love, lmk if it chooses "bellwether" as the name.