Sudo su

GPU/local LLM. more RAM and OSS... everywhere

[Sudo su @sudoingX]

i should have posted this a long time ago

hey, i'm sudo, and if you own a gpu and want to get your hands dirty with local ai, follow along

i run hands on experiments with local models on every tier of hardware, 8gb, 12gb, 16gb, 24gb, 32gb, 64gb, 128gb, all the way up to 256gb and climbing, and i've been in the local ai space for a while now

i also drop recommendations for the uncensored models and the weird ones, odd models and odd setups, so you can experiment and actually understand what's happening on your own hardware

my goal is to get as many normal people as possible started in local ai, and give them a taste of what owning your thinking feels like on your own metal

so if this moves something in you, listen to it, tell me in the comments what you want to see and i'll do my best to get you the answer, and if i can't, someone else will, the people in local ai are the most helpful crowd i've ever seen

[Sudo su @sudoingX]

someone smarter than me explain this.

why do we need a whole new GPU generation just to get more VRAM?

can't manufacturers use the same chip and put more memory next to it?

[Sudo su @sudoingX]

if you are not harness engineering you probably not gonna make it anon

[Sudo su @sudoingX]

when claude suddenly starts reasoning in turkish (or some other random language), that’s Anthropic quietly serving you the nerfed 4bit quant of opus 5.5.

[Sudo su @sudoingX]

someone smarter than me explain this.

why do we need a whole new GPU generation just to get more VRAM?

can't manufacturers use the same chip and put more memory next to it?

[Sudo su @sudoingX]

i might be stupid but

if my SSD can store 4TB of data, why can't i use part of it as extra RAM when i run out?

i know it's slower, but what exactly stops us from making this seamless?

[Sudo su @sudoingX]

@md_argv let's go

[Sudo su @sudoingX]

which friend would build an entire startup just to avoid paying $20/month?

[Sudo su @sudoingX]

i'm falling for my own agent harness

xpress agent is the harness i built for myself, it's rust and i can feel it in the speed, and offrail 27b is the model behind it, running on my laptop right now

it pulls images off the web and lay them out right under its answer, i slid through them and opened the one i wanted, no more clicking through folders to find what it downloaded, and this one feature saved me so much time

the model does the thinking, the harness decides what it sees and what it remembers, and you spend your whole day inside the harness, so it should fit the way you work

> one binary under 10 mb
> 14 mb of ram for the whole loop
> finds my local model by itself
> self learning loop

it writes its own skills as it works, and on that same run it rewrote its image skill without me asking

i think eventually you all will have to build your own agent harness the way you like it and the future of agents is rust

[Sudo su @sudoingX]

yesterday this was a tweet asking a grokbot to build it.

since last night it's the only agent i open xpress agent v0.1.0.

local first, learns as it works, written in rust grokbot prototyped it, i took it into cursor and engineered every inch of it, and it's already the best local-first agent i've ever used.

built by me, for me the moment you launch it, it finds the model server on localhost by itself and configures itself. no config file, no setup.

my 27b is just there, 128k context, ready to work /model to switch between local and cloud. free openrouter models chained as fallbacks so it never dies mid task. permissions ask by default, shift+tab to change. /resume any past chat. @ any file.

web search built in and the part i love most: it learns as it works. hermes agent's self-improving loop is the best idea in agents right now, so i rebuilt it in rust my way one binary. no python waking up.

it loads faster and answers faster than hermes agent and openclaw on the same model. and my api keys never leave my machine this era is insane.

if you're still waiting for someone to ship the tool you want, you're already behind.

build it yourself with @bot, in a day

[Sudo su @sudoingX]

i might be stupid but

if my SSD can store 4TB of data, why can't i use part of it as extra RAM when i run out?

i know it's slower, but what exactly stops us from making this seamless?

[Sudo su @sudoingX]

here are the 1017 weights and the recipe, the launch build stays up next to them

> weights: https://t.co/0qEYt67Pai
> recipe: https://t.co/KyX225scKE

196 gb of weights plus deepseek's own 1017 engram shards, the launch ones differ, and the heavier dense and head runs sit in the repo's benchmarks with their speed

[Sudo su @sudoingX]

i rebuilt my deepseek v4.1 flash weights on the 1017 update, v1.2 is live on 2x dgx sparks 🐋

> structured output: 141.3 tok/s
> code: 100.7 tok/s
> prose: 59.2 tok/s

and the quality held, my 2.70 bit weights scored against native 1017 on my own kl panel, the one other 1017 packs now report on

> kld 0.0243 from native 1017
> 20% closer than libertaidai's 2.9 bit pack
> top-1 agreement 95.47%
> humaneval+, mbpp+, ifeval within noise
> ifeval replies: theirs wins, 0.1135 vs 0.1199

deepseek shipped 1017 as a minor update, so i diffed it against the checkpoint my build came from, the config, tokenizer and tensor layout match byte for byte and every shard's content is new, in the slices i sampled about 27% of the 4 bit expert codes flipped and the built-in drafter moved the most

there was nothing to patch, so i pulled the 510 gb checkpoint again, measured all 15,360 experts, ran a 16 hour encode with one spark down for almost 3 hours while the other carried on solo, then dense layers, head, drafter and every eval, about 25 hours from download to serving

it runs about as fast as my launch build, which did 103.0 tok/s on code and 58.0 on prose, a verify round on code still takes 42.8 ms on both, the new weights just let the drafter land 4.31 tokens a round on that reply instead of 4.41

by the time mine finished, henryous and 0xsero had shipped 1017 packs built on my recipe, and henryous reports 0.0201 at 2.75 bits with heavier dense layers and head, closer to native than mine

so i tried that mix on my own experts, dense at 5 bits and the head at 6 got me to 0.0219 for about 5% of my speed, i'm shipping the fast build and the heavier numbers are in the repo

same engine as v1.1, so you keep the image and swap in the 1017 weights plus deepseek's 1017 engram shards

next i'm training my own drafter for it to push more

[Sudo su @sudoingX]

holy shit, llama.cpp rpc actually works

this is 4x dgx sparks hosting ling 3.1 flash, a 560b moe, at q4 across all four gb10s, about $28k of sparks at today's $6,950 a box

no one shipped an official fp4 yet, so i had to try the gguf over rpc and it's possible. it's super slow because of the engine, but i really like the model's output, it has weight to its answers

i couldn't stop myself from running this one, i ran ling 3.0 flash on a single dgx spark before qwen 3.8 took over, and this one feels like a real improvement, but it doesn't fit on my 2x dgx sparks, so @niklaslenz_ai, a friend here on x, lent me his 4x dgx spark just to run it

it took about 12 hours to complete a build and i'll share it next, stay tuned anon, looks like @AntLingAGI just had an ai leap

[Sudo su @sudoingX]

yesterday this was a tweet asking a grokbot to build it.

since last night it's the only agent i open xpress agent v0.1.0.

local first, learns as it works, written in rust grokbot prototyped it, i took it into cursor and engineered every inch of it, and it's already the best local-first agent i've ever used.

built by me, for me the moment you launch it, it finds the model server on localhost by itself and configures itself. no config file, no setup.

my 27b is just there, 128k context, ready to work /model to switch between local and cloud. free openrouter models chained as fallbacks so it never dies mid task. permissions ask by default, shift+tab to change. /resume any past chat. @ any file.

web search built in and the part i love most: it learns as it works. hermes agent's self-improving loop is the best idea in agents right now, so i rebuilt it in rust my way one binary. no python waking up.

it loads faster and answers faster than hermes agent and openclaw on the same model. and my api keys never leave my machine this era is insane.

if you're still waiting for someone to ship the tool you want, you're already behind.

build it yourself with @bot, in a day

[Sudo su @sudoingX]

i owe the new people here an intro

hey, i'm sudo, i run local models on my metal every day, my 24gb laptop, a 128gb strix halo and 2x dgx sparks, and i post the numbers and the exact commands so you can run the same thing at home

i do most of this for free, the tools too:
> a one-line installer, 8gb and up
> serve recipes for the big models
> my own model builds on huggingface

last week that was an uncensored 27b at 50.8 tok/s on a 12gb card, the whole model in a 6gb file anyone can download

and i help as many people as i can get started, in the replies and dms, so if you're new here anon, reply with your gpu and how much vram it has and i'll point you at the first model to run on it

[Sudo su @sudoingX]

@DigitalNav the first company to ship it gets my wallet and half the local ai crowd's

[Sudo su @sudoingX]

30 years ago you upgraded vram with your thumb, then they took it away like the headphone jack

[很好吃 @ned714]

@sudoingX Actually, slots for adding more VRAM to GPUs existed very early on. Let me show you an “antique.”

[Sudo su @sudoingX]

@styleshifter1 30 years later and the gpu wars never changed, only the logos did

[Sudo su @sudoingX]

@KStoyanov80 the card they called a 3d decelerator, and you still bought it more ram, that's love

[Sudo su @sudoingX]

@tolewis and the headphone jack while we're at it

[Sudo su @sudoingX]

@MPECSInc i'm too young for irq jumpers and the photo still gives me anxiety

[Sudo su @sudoingX]

can i ask a DUMB tech question?

why can't we just add more VRAM to a GPU like we add more RAM to a PC?

what exactly makes this so difficult?