Ryan Wright

Building Local AI Inference & Agentic AI in Public 🛠️ | ex @SentinelOne 1st SRE Go Team & @Comcast VIPER Startup | Local Inference Club |

[Ryan Wright @IQReactorAI]

Bad 👎: Didn't think Strix Halo would drop in $ w/ Gorgon Halo ( see 🧵for RAM data )

Good 👍: 2nd Strix Halo on the way. Planned vLLM, Ray, RDMA RoCe, with k3s cluster on top for AI Inference and other fun stuff.

Wait till you see what I deploy in it.

Decision data 🧵

[Ryan Wright @IQReactorAI]

Getting smaller and smaller:

Zuck's new "Muse Glimmer" 2-bit GGUF did 100+ tools calls on 14GB RAM.

[Unsloth AI @UnslothAI]

2-bit Muse Glimmer GGUF managed to call 100+ tools on just 14GB RAM. 🔥

Muse Glimmer did a complete repo bug hunt for 5 mins nonstop with: evidence, repro, fix, tests and a PR writeup.

Run and train it in Unsloth.

GitHub repo: https://t.co/aZWYAtakBP

[Ryan Wright @IQReactorAI]

@SifLeandro46268 super cool 😎

[Ryan Wright @IQReactorAI]

@drose101 great fun! 😎

[Ryan Wright @IQReactorAI]

@xubujie @pidotdev Love this very true

[Ryan Wright @IQReactorAI]

@randomingspace lol basically 😎

[Ryan Wright @IQReactorAI]

@jurlycat That’s a great point, true

[Ryan Wright @IQReactorAI]

Truth:

ULTRA SMART (Frontier LLMs): unlocking novel reasoning. zero-days, scientific synthesis, complex architecture.

ULTRA CHEAP (SLMs / MLMs): Commodity intelligence operating near zero marginal cost (comparatively) for everyday workflows.

Prediction: Distillation & quantization will bring frontier-grade capabilities to consumer edge hardware faster than people think.

[Bindu Reddy @bindureddy]

The future will be dominated by two types of models

ULTRA SMART - large extremely intelligent 10T models that can do very complex work

ULTRA CHEAP - near free intelligence that can handle pretty much everything else

Today’s equivalents are Fable and Luna / Deepseek Flash

[Ryan Wright @IQReactorAI]

Offically on Pi coding agent now. I don't need "y" or "n" .

I `/edge-case` my well researched Spec > plug the holes > let it rip.

Sys prmpt tune by A/B testing diff prompts & checking code for new models ( see vid )

[Ryan Wright @IQReactorAI]

@_alexspring We laugh at this till it gets humanoid MCP and starts racking servers and running cables.

[Ryan Wright @IQReactorAI]

@alexocheema @llm_wizard The holiest of molies needed for this weeks AI endeavors

[Ryan Wright @IQReactorAI]

@paulmillr Can’t wait to run flash

[Ryan Wright @IQReactorAI]

@rozario_ru83489 Ha same ! I can’t keep up

[Ryan Wright @IQReactorAI]

Meta’s Muse Glimmer is out. Open Source still cookin people!

30B open (Apache 2.0)

Unsloth says “strongest agentic model in its class”

Quant ~17-20GB, strong tool use, reasoning, recovery, multimodal.

Weights on Hugging Face, Ollama (MLX) + Unsloth/llama.cpp support live, full integrations rolling out. And Muse Spark 1.2 open weights coming soon per the Zuck (see photos)

Who’s testing it?

#Agents #LocalAI #OpenSourceAI

[AI at Meta @AIatMeta]

Introducing Muse Glimmer, an open-weight 30B-parameter model optimized for local, always-on agent workflows.

Muse Glimmer delivers strong performance on key agentic use cases and benchmarks compared with leading models in its size category, and is designed to run entirely on consumer hardware like a Mac or PCs with performant GPUs.

In keeping with our long tradition of sharing fundamental AI research, we’re releasing model weights under a permissive Apache 2.0 license.

🧵👇

[Ryan Wright @IQReactorAI]

I really like it. Esp even though I installed web search, it refuses to use it and instead pulls the package down and searches the code.

Come to find out this is a lot of people’s exp. What I noticed is it’s faster than checking the Docs. I have a lot of other processes around how I use it that I’ll publish soon. But loving it with Laguna S2.1

[Ryan Wright @IQReactorAI]

@HotAisle Exactly my line of thinking for what I’m setting up today w/ multi model bias.

Competing model biases to fight over how the code doesn’t live up to the criteria.

Great point for local too, route to lessor model for quicker edits.

[Ryan Wright @IQReactorAI]

@DegenApeDev 😁

[Ryan Wright @IQReactorAI]

Pi is so good. I was blown away by the fact that it doesn’t want to guide the model to searching the web and instead the harness will search the code base of the packages it pulls locally to figure things out.

Extremely fast and get things right the first time with my local Laguna S2.1 model

[Ryan Wright @IQReactorAI]

@Keldrik It’s actually engineering genius because instead of searching the web for structured data which totally messes up your prefix cashing and spec decoding it instead searches the package it downloaded locally. Pi is crazy simple and crazy good 👍

[Ryan Wright @IQReactorAI]

@rozario_ru83489 Gotta latch on too and interact with ppl in your niche then it starts getting good.

Reply to people in the comments you like their tech, whatever they are doing and your whole feed becomes that

[Ryan Wright @IQReactorAI]

@rozario_ru83489 Gotta try omp. Everyone going for it/liking it

[Ryan Wright @IQReactorAI]

@sirstripy Hi, love pi, but good points actually, I didn’t think about.