CosmicRaisins

im pretty sure i was below 10 above the speed limit

[CosmicRaisins @CosmicRaisins]

I've got @Zai_org's SoTA open-source model GLM 5.2 running on a cluster of 4 @nvidia GB10 nodes with vllm at ~25 tok/s in agentic workflows, 328k ctx length, and ~600 tok/s prefill at all depths! Now you can have Opus 4.5 at home🎇

Recipe and patches at: https://t.co/NqZvySqzhm

[CosmicRaisins @CosmicRaisins]

@TheDavidTai Swapping to GLM takes about 5-8 mins iirc. DS is much faster since it's about 1/3 the size.

[CosmicRaisins @CosmicRaisins]

GLM 5.2 is smarter, but I just can't go back from this kind of speed...

I really hope GLM 5.3 can pull some serious punches.

[CosmicRaisins @CosmicRaisins]

@TechMDAI Thanks for the shoutout and your kind words🤯 It means a lot.

[CosmicRaisins @CosmicRaisins]

@TechMDAI 18-30 tg depending on the prose, 500-800 prefil.

[CosmicRaisins @CosmicRaisins]

@TheDavidTai I can only run one at a time, and GLM keeps getting squeezed out. For me, most tasks GLM can do, DS can do as well, much faster. What's left is a small band above dsv4's ceiling, and for those I'd rather run DS and use a cloud model than wait for GLM. So it never gets picked.

[CosmicRaisins @CosmicRaisins]

I feel like Anthropic is doing an A/B test or something where it either drags the temperature or topk or some stupid metrics for some users and waits to see how they react, because working with it is like working with some of my master's program classmates all over again.

[CosmicRaisins @CosmicRaisins]

I'm pretty sure restaurants as a category is dead because everyone is just gonna cook their own meals

Meals don’t have ongoing switching costs or lock-in, you can just decide what you want to eat and make something else tomorrow

Perhaps an argument for wanting someone else to design and cook a meal for you, easier to eat a meal than to make one.

But home cooking and restaurants are so close now the difference between making a restaurant-quality meal yourself and having someone else make it is getting closer and closer to zero.

So why eat someone else’s food? Just spend $20 on groceries and make your own

[Nick Dobos @NickADobos]

I’m pretty sure gaming as a category is dead because everyone is just gonna build their own games

Games don’t have ongoing maintenance cost like companies replacing SaaS vendors

Perhaps an argument for wanting someone else to build an experience for you, easier to watch a movie than create one.

But video games and vibe coding are so close the difference between creating the form and playing in a form is getting closer and closer to zero.

So why play someone else’s game?
Just pay chatgpt $20 and make you own

[CosmicRaisins @CosmicRaisins]

Most of the people I've met IRL who still claim that actually typing code and reading code is a necessity are either:
1. Working on seriously intense and low-level stuff like kernels.
2. "Last time I tried coding with AI was pasting code blocks from Gemini 2.5 Pro's reply."

[CosmicRaisins @CosmicRaisins]

@TheSpacerr vibing

[CosmicRaisins @CosmicRaisins]

I'm an indie game dev. I give AI design goals. It proposes architectures. I review the architecture. AI writes the code. I never look at the code, except when I explicitly ask for something while debugging.

It works much more smoothly for a small team than before, when people have to read each other's code and waste time in design meetings talking about code hygiene and style.

[CosmicRaisins @CosmicRaisins]

"through agentic inquiry"

[Guillermo Rauch @rauchg]

If you’re not reading the code, whether explicitly or through agentic inquiry, one or more of these is true:

○ You’re a beginner
○ Software is throwaway
○ You’re prototyping
○ You have no users / revenue
○ You’re taking on debt & risk
○ Your problems are basic

And btw. All of this is fine. But the reality is that models are still not at the “full autonomy” stage yet.

They make rookie mistakes, they go down bad architectural paths. I just had the best model in the world add a nonsensical 700ms delay to “settle” something and it told me “you’re right, I was cargo-culting” 🤨

I am on the camp that this need will diminish more and more. Most code is indeed going to be assembly-like. But we also have the global internet and software infrastructure riding on these models and narrative, and we have to respect that.

[Santiago @svpino]

I'm officially done reading AI-generated code.

It's been two weeks since I looked at any of it.

I think the IDE is officially on its way to the graveyard. The job is no longer about "writing code," so we need new tools that better reflect this new reality.

While reviewing the code, I realized my only complaints were stylistic, and I wasn't finding any obvious bugs anymore.

The more code I generated, the harder it became to keep track of every line. I found that my time is better spent designing ways to verify that the overall system works than looking at the code.

State-of-the-art coding agents are better at writing code than I'd ever be, and I'm going to stop pretending otherwise.

I still think these coding agents can't go too far without an experienced human guiding them, but we're past the point where we need to check every line of code.

[CosmicRaisins @CosmicRaisins]

I just discovered that Theo is friends with pirate software. Wtf!

I unfollowed the guy faster than a mage can blink out of a dungeon. But I still feel kind of gross.

[CosmicRaisins @CosmicRaisins]

@sloppenheimer Wait, are you saying that I can't ask Claude to put 3 lines of patchs on my vllm clone, bench it by prompting "count from 1 to 100," and claim that I made a vLLM image that gets 100 tok/s on 2 sparks anymore??

[CosmicRaisins @CosmicRaisins]

It's always nice to study good architecture, project management, and high-level programming conventions and patterns to build intuition. But "when everyone vibe codes, engineers who can hand-code will be in demand" is an ass pull. But hey, I guess COBOL maintainer is a real JD.

[CosmicRaisins @CosmicRaisins]

Coding was always the hard part, and it's specifically because of it that vibe coding caused an explosion of people making stuff.

But (not a direct reply to this post) "use AI but not to the extent that it erodes your skill" is still LinkedIn-coded. It means nothing actionable.

[Mario Zechner @badlogicgames]

recommended reading.

https://t.co/igyB7oofie

[CosmicRaisins @CosmicRaisins]

@Authentic1ty @kernel_trick Perhaps everyone could all just stick to their own work instead of "tearing people down" when they don't agree with them.

[CosmicRaisins @CosmicRaisins]

Voxel scenes are cool, but provide minimal information on how a quant compares to the base. REAP hurts knowledge and behavior; quantizations hurt precision.

Neither affects a Well-known, low-skill, unbounded task like this, probably extremely redundant in the weights.

[CosmicRaisins @CosmicRaisins]

Guy claims his GLM reap "holds parity with the full unpruned model on a well-powered real-world eval." Then few lines down he reports 2x loop rates compared to teacher.

No mention of his eval methods in the entire model card too.

[𝑘𝑒𝑟𝑛𝑒𝑙𝑡𝑟𝑖𝑐𝑘 @kernel_trick]

crying this guy got $100k+ in donations for "local AI"

[CosmicRaisins @CosmicRaisins]

@Authentic1ty @kernel_trick The biggest barrier to quantizing, REAPing, and healing models is compute, which is money, not skill. The dataset and methods are open source, and any frontier LLM can do it without guidance.

A lot of people can do better, but they don't have the means.