@bridgemindai@matthewmillerai@pvncher there’s a real issue here. The advertised efficiency that would makes the cost per task be similar is not happening in practice for a lot of users. I’ll need to go back to Sol for most of the work even with 3 Pro 20x accounts
@tunahorse21@thsottiaux same here using mostly just medium reasoning. $200/plan with same type of work I did before with Sol, weekly limit gone in 1 day
@boringmarketer I think with current models it’s smarter to simplify this a lot being less prescriptive and specify more in high level what you want, like how you’d talk to a capable human. Otherwise it constrains the model to much
Coding is solved. I haven’t written a single line of code since 2025. But software engineering is more alive than ever. If you’re building something that matches a known pattern, AI will deliver perfection. But as soon as you get into deeply complex, novel work, it produces slop.
It feels like we've hit diminishing returns on intelligence for many tasks
We may no longer see every product auto-switch to the next frontier model upon release
This is great for app-layer builders (many opportunities to bring down COGS!)
The ability to wrangle these immensely powerful, yet dangerously capricious, agents into a productive harness that produces high quality systems — is the software engineering challenge of this decade.
the focus on whether you read the code or not is the wrong thing to look at
if you have software you're responsible for, you should be able to answer questions from memory about how it works
the expectations for how well you can do this should not be any different now
One of our more useful internal channels is gong-love. Every night a sentiment bot reads our customer calls to surface themes and quotes that bring calls to life
It helps sellers articulate impact in customers' own words, share feedback to builders and make our marketing better
For anyone with endless ideas, this agent age is nirvana as those ideas are met with endless execution, endless exploration. I've never had has much fun working with computers as I do right now. What a time to be alive.
yessssssssss lets add 20 helper functions to the top of the file for appending to arrays yessssssss thank you o wise superintelligence yessssssssssssssssssssss dont listen to me telling you not to do this
I think the reason models feel worse a while after launch is that a model's performance is a function of its raw intelligence, multiplied by the codebase's quality. yet, every time you use AI, your codebase gets a bit worse. sometimes it is invisible, but confusion accumulates, even when behavior remains intact. unused fields, inconsistent variable names, stale comments. the more the AI edits, the more slop-fried your codebase becomes, like a JPEG that is repeatedly compressed. this, in turn, causes more junk to automatically get in the context, bringing you closer to ugly latent space neighborhoods, making it produce worse outputs, til you reach an equilibrium where no more progress is possible. then a new model launches, and its increased slop-resistance counterweights the accumulated junk, and things suddenly start moving forward again. until your repo crosses that model's slop threshold again, and it stops working, again. I think this process is an inherent part of how these things work and GPT 10 or Fable 10 will still feel moronic after 1 month of launch. the only difference is you'd be maintaining a 100k LOC project, but it will still fail to add a new feature in ways that make you facepalm
many people ask when to use a big model (like sol/fable) at low reasoning effort, vs a small model (like luna/sonnet) at high reasoning effort
i deliberately forced myself to use all the permutations a lot over the last couple of weeks to build intuition, and i realized the difference is "wisdom" vs "diligence"
bigger models are "wiser"
they have seen a lot. they remember a lot. they have a lot of expertise across different domains. they have better intuition, can connects the dots, and come up with creative, inspired ideas
reasoning effort makes a model more "diligent"
it'll assess each option, think through consequences, and figure out edge cases etc more thoroughly
if there are 100 paths ahead, diligence makes the model assess every single one without a miss, but it will not make the model realize maybe the best one is to take none of the 100 paths and instead dig a tunnel
"wisdom" and "diligence" are orthogonal. and now i get why the models are launched the way they were, and not just a single fable level model with 12 different reasoning levels
we're all misled by the way we've been plotting the models with benchmark scores, which are fundamentally flawed because they use a single dimension to measure the model's capability, making us think of model size and reasoning effort as being fungible with each other, while in fact "wisdom" and "diligence" needs to be measured separately
i hope the evals eventually catch up and address this. until then, here's my recommendation for how to choose -
- if the problem you are trying to solve is something you think requires a genius, use a bigger model
- if the problem you are trying to solve is something you think requires a pen and lots of paper, use higher reasoning effort
- if it requires a genius sitting down with a pen and lots of paper, tune up both