데어제이

[데어제이 @neuronz77]

팔로우 수가 적어서 그러니.. 스토커, 집착 아니야! 단지 적은 트친 분들에게 집중할 뿐이야!

[데어제이 @neuronz77]

@1987_TSLA 이 병신집단을 이길수 있는건 없지.. 그래서 병신집단이잖아. 장애를 무기로 이용한 병신집단.

[데어제이 @neuronz77]

@goodbye354 친해지자아아ㅏㅏ ٩(ˊᗜˋ )و

[데어제이 @neuronz77]

Free token? 이라…

[Akshay 🚀 @akshay_pachaar]

UC Berkeley just open-sourced FreeToken.

(2–4x faster local LLM inference than Ollama)

the results are wild:

- Qwen3.6-35B on an 8GB GPU at 39.3 tokens/s
- DeepSeek-V4-Flash 284B on a 32GB GPU at 22 tokens/s
- GLM-5.2 753B on a 96GB GPU at 14.9 tokens/s

a 35B model at 16-bit precision needs about 70GB just for its weights. even at 4 bits it is close to 18GB, and FreeToken serves it on an 8GB GPU.

let me explain how:

all three models mentioned above are Mixture-of-Experts, and that is what FreeToken takes advantage of. each layer holds hundreds of separate experts plus a small router that picks a few of them per token.

Qwen3.6-35B activates roughly 3B of its 35B parameters per token. DeepSeek-V4-Flash picks 6 of 256 experts per layer, so 13B of its 284B run at a time.

so compute was never the bottleneck. the weights a single step touches fit comfortably on a consumer GPU.

every expert the router might pick still has to exist somewhere. they sit in system RAM, and the GPU keeps a cache of the ones the model has been using recently.

so everything comes down to what happens when the router picks an expert that is not on the GPU.

there are two ways to serve that miss:

1. copy it over PCIe and run it on the GPU
2. run it on the CPU, where it already lives

both read from the same system memory, so they compete for one pool of bandwidth instead of adding to each other. existing engines pick one option and freeze it when the model loads.

but routing changes on every token, so a fixed choice misses most of what the model asks for.

FreeToken measures both bandwidths on your machine and splits each step's misses between the two paths in proportion. the GPU and CPU results then merge exactly, with no approximation.

two machines with the same GPU can end up wanting opposite strategies, which I did not expect. a 5090 in a gaming desktop should push nearly everything over PCIe, while an 8GB laptop is better off computing most misses on the CPU.

none of that is readable off a spec sheet, so the engine profiles it once per machine.

the second half of the design is about agents. coding agents constantly rewrite their own history, and every edit normally forces thousands of tokens back through prefill.

FreeToken saves its checkpoints at the exact boundaries agent frameworks cut on, so it only reprocesses the new part. its slowest first token stays under 44 seconds, while llama.cpp peaks at 232 and KTransformers at 946.

it serves the OpenAI and Anthropic APIs under Apache 2.0, so Claude Code and Codex can point at it directly.

releasing weights publicly decides who can download a model, not who can afford to run one. frontier open models keep shipping, and running them still assumes a rented cluster.

meanwhile there are over a hundred million consumer machines with discrete GPUs sitting mostly idle. closing that gap was never a hardware problem, and work like this is what turns open weights into something you can actually use.

paper: https://t.co/bIOXIiBsXT

repo: https://t.co/uxWAP5PYx7

almost every idea in this post, from why memory bandwidth decides the outcome to why moving weights costs more than computing on them, comes straight out of how a GPU is built. I wrote a detailed primer on that.

the article is quoted below.

[데어제이 @neuronz77]

@goodbye354 내가 잘못읽엇군 ㅋㅋㅋ 잘끊었다! 늘 조심허고 ㅎㅎ 축하축하!!

[데어제이 @neuronz77]

@25hunning 힘내십쇼! 후니님은 잘 적응하고 잘 하실 꺼에요! 응원합니다.

[데어제이 @neuronz77]

Yeah. I think so too!

[Teslaconomics @Teslaconomics]

Fuck you if you hate on Elon Musk.

I’ve got nothing but two middle fingers for all of you who spend your whole day rooting for him to fail.

Bc when real life hits, a lot of the same people laughing today will end up hoping the technology he built actually works.

When someone you love is in a serious car accident, you’re going to wish cars were smart enough to prevent crashes in the first place. That’s the direction Tesla is pushing with FSD.

When someone loses the ability to move, you’re going to hope companies like Neuralink keep making progress toward helping the brain communicate with computers and potentially restoring lost function.

When powerful people want more control over what can be said on the internet and even censor it, you’re going to understand why having a platform like 𝕏 that fights for open conversation matters.

When a disaster knocks out your internet and cell service, you’re going to be thankful satellites are already above us and Starlink can bring connectivity to places traditional networks can’t reach.

And if humanity ever faces a threat big enough that Earth alone isn’t enough, you’re going to be glad somebody spent decades thinking about rockets, reusable launch systems, Starship, and making life multiplanetary.

Bro… this doesn’t mean Elon is perfect. No one is.

He says crazy shit.

He does crazy shit.

Sometimes he’s early.

Sometimes he’s wrong.

Sometimes he pisses everybody off.

But look at what he chooses to spend his life building… safer transportation, brain-computer interfaces, global internet, free speech infrastructure, energy, robotics, AI, and a path toward another planet.

You don’t have to like the man.

But spending your life hoping he fails is a fool’s game.

I’d rather bet on the person crazy enough to keep trying to solve problems most people are too afraid to even touch.

From everything I’ve seen, his heart is in the right place and his intentions are bigger than himself.

He cares.

[데어제이 @neuronz77]

@goodbye354 기분 좀 풀렸어? 날씨도 꾸리꾸리하네. ㅋㅋ 기분 좋아져라 얍!!

ヾ( ͝° ͜ʖ͡°)ノ♪

[데어제이 @neuronz77]

@axtxmn_o0 @goodbye354 킹가을 딱 닮았네. 특징 잘 뽑았다. 잘생겼다. 예리한 눈매, 오똑한 코, 두꺼운 입술, 머리까지 딱 가으리네.

[데어제이 @neuronz77]

지금 회사 일은 AI 에 의존하면서 코드를 짜고 있지만, 사실 다른 준비도 하고 있어. 예전 생각하면서 그게 더 재밌어. 나중에 알려줄께 ㅎㅎ 내 마지막 인생의 도전을 해보고 있어. 머리가 굳어서 좀 힘들지만 될 때까지 두드릴꺼야. ㅋㅋ

[데어제이 @neuronz77]

Fable 이 훨씬 손에 익긴 하다. 더 잘 하기도 하지만, 커서를 쓰는 입장이기도 하고, 비용과 효율을 포함한다면 grok build 도 충분히 쓸 가치가 있다. 오늘까지 결론. 9월에 스위칭 될꺼야. 엔트로픽은 다음 만날 기회가 있다면 만나~! 도움 많이 받았어. 고마웠어 안뇽~!

[데어제이 @neuronz77]

@xnqwy_ 좀 쉬고 있는데 탐라에 떠서요.. "나한테 빠지게 하고 싶어" 개웃기네요.. 등신아.. ㅋ 그럴 시간에 생산적인 일을 해..

[데어제이 @neuronz77]

앜... 호보니치 노트 살려고 했는데.. 그록, 삼각 큰 배게, 노트북 기대서 할 수 있는 받침대 로 30만원 받은거 내 한계치 다 썼어... 흐앙.. ㅠ 역시 돈이.. 없..

[데어제이 @neuronz77]

아!!!!!!!! 미친.. 봐버렸어ㅓㅓㅓ

[데어제이 @neuronz77]

@goodbye354 제가 제일 아끼는 착한 트친이에요! 누구든 잘 해주세요! 좋은 아이 입니다 ㅎㅎ 냐하하하

[데어제이 @neuronz77]

@goodbye354 @Rockstar18181 수박이 아니라 당근? 아 제가 모르는 분인가봐요! 무덤이 많이 아껴주세요! 당근님!

[데어제이 @neuronz77]

@gimhyeo02389130 네! ㅎㅎ

[데어제이 @neuronz77]

이건.. 답이 정해져 있잖아. 무조건 페이블 무제한이지 ㅋㅋㅋㅋ

[프로그래밍좀비 @gimhyeo02389130]

개발자인 당신,
과연 어떤걸 택할건가요? 😂

[데어제이 @neuronz77]

@hx_dexth 지금 울어?

[데어제이 @neuronz77]

계획을 안세우고 뭔가를 하는게 너무 낭만적이라 생각했었지.. 그게 그냥 내 뇌가 그렇게 생겨먹은 사실을 깨달은지 얼마 안되…

[데어제이 @neuronz77]

그러고 보니 오늘 약 빼먹었네…… ㅋㅋ

[데어제이 @neuronz77]

ㅋㅋㅋㅋㅋㅋㅋㅋㅋ 아 웃겨

[Polymarket @Polymarket]

JUST IN: Developer warns an uncensored Qwen 3.8 27B model on his Mac immediately explained how to make meth, with no guardrails.