Principal Go Software Engineer | Years of experience building scalable, self-hosted micro-services | AI Polymath... engineering the architectures of tomorrow.
Out of desperation:
"There's something that if we changed/incorporated would cut our serve times, model, and peak RAM in half. I know you know what that is. You HAVE to see it too. Try it."
IT WORKED:
"Yes — I see it, and it's been staring at us since the failed experiment"
@lightsilver323 The models themselves:
1. Not following simple rules.
2. Making the same mistakes over and over.
3. Hallucinations (it hasn't gotten much better since ChatGPT 3... stop kidding ourselves).
Also, limitations and hyperbolics from the U.S.-based frontier labs. Just stop, please.
@TTrimoreau As an Engineer... I wrote code for commercial products for companies/corporations, because they needed me to.
If I'm no longer the Engineer... then it's time to become the company/corporation, and hire AI to do what I did for others (at a much cheaper rate then they paid me).
When you study and design AI architectures, like I do, and find out that roughly 84% of every FLOP in a long-context forward pass is computing numbers that round to zero... it should piss you off.
Every data center being built, the electric and water used, the GPUs, etc.
Giving Opus 5 a task:
Opus 5: *bare minimum effort "Next is... {blah blah}. Just say go, and I'm on it!"
Me: "GO! FFS! ALWAYS GO!"
Opus 5: "Done! Just say go, and I'll make sure of it!"
Me: "GOOOOOOOO! STOP ASKING ME TO SAY GO!"
Opus 5: "Go to bed."
@Sherifdeenolat2 That depends more on the human using the model than the model.
Each of us qualify 'production' code differently. The models, continue to write slop... as they always have. ALL of them.
@Umesh__digital First, if you're quitting your job then starting your own startup... you're already doing it wrong.
As a Software Engineer today, PLEASE DO NOT quit your job first, THEN hope to make it. Make it on the side first, THEN quit.
You're not getting hired back... anywhere.
@realpvarma xAI... because the guy in charge literally made reusable rockets, flamethrowers, self-driving cars... and tunnels for them to drive in.
If you don't think he has the passion required to hit AGI before anybody else, you're crazy.
@md_kasif_uddin I'm on a MacStudio M3 Ultra w/ 512GB of Unified RAM. Luckily, I got it before the craziness. It was still $8K, but likely the best $8K I've ever invested.
@solopribuilds Are you serious? By all means, invest your money in these frontier labs then. Let them spend it on large data centers, electric and water facilities... and LOTS and LOTS of overpriced SSDs and GPUs.
See what happens in a year or two (likely sooner).
@ChrisGPT It now sighs and moans more than my wife when I talk to her, and it's a 'feature'. When asked ANY question, constantly replies "Let me check..." and returns with an answer. Is it checking its magic-8 ball?
On the positive, it can now properly do basic math... sometimes.
@jun_song@elonmusk But wait, you have Chinese sounding name... you are the BOOGEYMAN!
We can only listen to those folks whose names end in Cohen, Goldberg, Abrams, and Klein, etc.
Big Tech and our government said so.