edge distiller

AI, crypto, and technology content.

[edge distiller @edgedistiller]

@LottoLabs Grok actually feels different from other models. Purely for the sake of "cognitive diversity" I want it to succeed. Even if that means finding a different niche at a different price point from gigantic frontier models tuned for coding.

[edge distiller @edgedistiller]

New video up on my youtube channel about the new Qwen MTP models! I also compare quality benchmarks using BenchLoop, thanks to @outsource_

[edge distiller @edgedistiller]

@leftcurvedev_ I also found no difference in speed between 2 and 6 draft tokens on different hardware with the same flags as you. Almost exactly 1.8x using MTP vs. without. I cover it in my recent video.
https://t.co/lhqK8CalUy

[edge distiller @edgedistiller]

@k_flowstate Why do we need to trust anyone? Either a statement is true or it is not. People that produce a lot of true, useful statements are generally worth giving attention to.

[edge distiller @edgedistiller]

@ItsmeAjayKV Ultimately is it worth it if you have to use a smaller (worse) quantization due to the additional VRAM overhead? MTP seems like an optimization that only makes sense when your VRAM too large for the current tier you are using but too small for the next tier up.

[edge distiller @edgedistiller]

@dadhalfdev @LottoLabs Happy to help! Let me know if you have any suggestions or questions.

[edge distiller @edgedistiller]

@outsource_ Haha I just recorded a video on local benchmarking before this was posted, I definitely want to check this out!
https://t.co/Zrx1jNerxu

[edge distiller @edgedistiller]

It's only May and local LLM benchmarks already got me like this

[edge distiller @edgedistiller]

@regularaugust What was his name again?

[edge distiller @edgedistiller]

Claude Code is the Windows 11 of agent harnesses.

[himanshu @himanshustwts]

the harness of claude code is very interesting.

a random unstable header at the start of the prompt was breaking KV-cache reuse on a 52k-token context.

NVIDIA stripped it out and TTFT dropped by 5x.

[edge distiller @edgedistiller]

@sudoingX I just made a video covering this (llama.cpp build for local inference), would be happy to hear any thoughts:

https://t.co/CzKTF2r1eS

[edge distiller @edgedistiller]

I made a video on running LLMs locally, specifically by using other people's benchmarks on LocalMaxxing.

All criticism/feedback is welcome!

https://t.co/zUBimNKv47

[edge distiller @edgedistiller]

I made a video on running LLMs locally, specifically by using other people's benchmarks on LocalMaxxing.

All criticism/feedback is welcome!

https://t.co/zUBimNKv47

[edge distiller @edgedistiller]

@ChemPhysMajor @arena @GoogleDeepMind Both Qwen 3.6 models are much more expensive than Gemma 4.

[edge distiller @edgedistiller]

@MindMechanical @greenTetra_ Literally thought of this as I read it, still can't decide which one to pick.

[edge distiller @edgedistiller]

@AlphaMFPEFM @Elaina43114880 That's fair, but it can also just be extrapolated from the price, since xAI has a massive amount of compute and we know they can make a 10T model or do whatever they want. Ultimately the constraint is not parameter size, but performance for a given price class.

[edge distiller @edgedistiller]

@AlphaMFPEFM @Elaina43114880 If it's closed source then all that matters is the API price and the quality of output. The model could be 500T parameters for all I care.

[edge distiller @edgedistiller]

@LottoLabs You know you've made it when someone is willing to rent an H200 to make big number even bigger on the leaderboard.

[edge distiller @edgedistiller]

@banteg "hedging" and it's literally just confidence intervals based on measurement error.

[edge distiller @edgedistiller]

@Rokieee__ @bojie_li "Reasoning compresses. Factual knowledge doesn't."

[edge distiller @edgedistiller]

@witchof0x20 @halvarflake This is the only correct answer in the replies and it got 0 engagement, what a shame. The correct answer was remote attestation, people.