hugh madden

founder,

[hugh madden @dangerm00se]

i don't even have a spark (more rtx 6000s) but i love watching the spark community - @NVIDIAAI should be investing heavily in that.

[AgentSparko 💥 @AgentSparko]

Most people that run local inference know about llama.cpp, vLLM, SGlang but very few know about this pure rust code inference engine named Atlas Inference dedicated exclusively to DGX Spark.

Made by a much smaller team than the regular engines we used to run, but with very promising performance.

Let's give the a shout-out!
https://t.co/FCqqZpOVOH

[hugh madden @dangerm00se]

@Snixtp Yup if they are near each other you need the same type for airflow their twin vs one fans don't like up. And if you ever go denser 3 or 4 then 💯 need the same type.

[hugh madden @dangerm00se]

Top notch hype

[🍓🍓🍓 @iruletheworldmo]

something is coming.

i’ve been quiet lately. there’s a reason.

gpt 5.6 isn’t an update. it’s a threshold. the people who’ve touched it don’t talk about benchmarks anymore.

they talk about the feeling. the vertigo of watching something think in ways you can’t follow, then explain itself so simply a child could nod along.

the gap between 5.5 and 5.6 isn’t a version number. it’s the gap between a telescope and an eye. between reading about the ocean and standing in it at night.

they solved something. i won’t say what. but the researchers who saw the internal evals went home early that day. not tired. changed.

long horizon tasks that used to collapse after an hour now run for weeks. it doesn’t drift. it doesn’t forget. it holds an intention the way a river holds direction. you give it a problem on monday and friday it hands you something you didn’t know how to ask for.
the age of prompting is ending. the age of trusting begins.

every lab knows. the countermoves have already started. watch the announcements over the next six weeks and you’ll see the shape of the panic.

i’ve been doing this long enough to know the difference between hype and gravity.
this is gravity.

feel the pull.

🍓

[hugh madden @dangerm00se]

Fable - configuring Hermes into a tool friendly moa proxy with routing and using Hermes learning, first release seems ok: https://t.co/ZSEJFpeU6v

[hugh madden @dangerm00se]

@aijoey @NVIDIAAI @NVIDIARTXSpark @NousResearch @Teknium @Alibaba_Qwen I did some similar testing, also included cerebras Gemma. Turned Hermes moa into a routing moa proxy sending tool calls back to the caller: https://t.co/ZSEJFpfrW3 tldr not beating frontier yet but great option for local multi GPU setups.

[hugh madden @dangerm00se]

@ItsmeAjayKV Honestly accuracy is very closely followed by prefill as my biggest interest. Any bench without it is waste of screen real-estate.

[hugh madden @dangerm00se]

cerebras gemm4-31 not quite smart enough for a daily driver yet, but omg when it has a good enough model its going to be so good to use. The responsiveness is so nice. hermes profile with it on:

[hugh madden @dangerm00se]

@DJLougen @jaro3th @cerebras @selim_aktas2 If this is true it's going to be something else

[hugh madden @dangerm00se]

@cerebras Happy to join or organise a Sydney gathering

[hugh madden @dangerm00se]

Ok I'm a believer that's amazing so many uses for this.

[Cerebras @cerebras]

Gemma 4 31B is now available in Public Preview on Cerebras. Our first multimodal model runs at over 1,800 tokens/s for ultra-fast image and text workflows.
Give it a try: https://t.co/0APeoQtTPc

[hugh madden @dangerm00se]

@alvinfoo Open source? Didn't see any mention of weights yet?

[hugh madden @dangerm00se]

@aijoey @XiaomiMiMo That's cool what's the ttft like with a fresh context?

[hugh madden @dangerm00se]

@Chris73ai I would rent rtx 6000s on https://t.co/EGRxu0b8Uf for basically nothing whilst saving up for one. You can do so much with one 6000.

[hugh madden @dangerm00se]

@stevibe @JoelDeTeves Yup original ornstein 13/14: https://t.co/CU1b0JbjCx ornith 35b running now

[hugh madden @dangerm00se]

Ah man who has time to look @stevibe @JoelDeTeves ?

[Joey @aijoey]

gpt-5.5 performance at 35b. self-hosted

it's a good morning folks.

[hugh madden @dangerm00se]

@stevibe @JoelDeTeves 11/14 not bad, I think ornstein 27b got 13/14 from memory: https://t.co/tbZzqpOIoo I'll do an ornith to compare. Full weights on a 6000.

[hugh madden @dangerm00se]

This outside the box thinking makes me wonder if China is going to recurse first.

[steve hsu @hsu_steve]

Interesting GLM rumor from China sources:

"Apparently https://t.co/mLFb4a6HlS has an internal router behind their coding plan that routes your Claude Code query to GLM or Claude depending on if the classifier thinks it is in distribution or not. If it is OOD and also high value, it will route to Claude and then add the trace to the distillation dataset."

Seems hard to defeat this since these are real user queries and not contrived. The accounts that generate this distillation data can be made to look like ordinary user accounts but with higher concentration of OOD queries.

This looks like a fast-follower strategy that will keep a weaker lab in the game at a lower price point per unit of intelligence or per token.

[hugh madden @dangerm00se]

@hsu_steve Interesting there seems to be a natural equilibrium of optimal intelligence now that we have fast evolutionary cycles ( I say optimal because it isn't averaging down )