mattwallace

techie, dad, author, inventor, perpetually curious; CTO & Builder all-in on #AI

[mattwallace @mattwallace]

cat https://t.co/klLNTJOm1H | cllm translate to french > docs_french.md

https://t.co/Taz4ulLFNB

what's next?

[mattwallace @mattwallace]

@jamesthomason soon™️

[mattwallace @mattwallace]

@CTOAdvisor @nvidia As a quick warning, keep an eye at longer context on if your engine tries to do speculative on the whole ctx - tends to be bad, especially for dflash (where with the wrong setup you can get huge gains at low ctx but negative results at long; speculative_window etc here)

[mattwallace @mattwallace]

@CTOAdvisor @Apple @nvidia It's VERY easy to just run mlx-convert and get a quant. quick check: size of checkpoint? eg, if you have 13GB of mlx weights and 16GB of NVFP4, odds are your nvidia quant is much stronger because that 3GB is a lot of critical layers kept at FP16/32

[mattwallace @mattwallace]

@CTOAdvisor @Apple @nvidia places to check
1. quantizations: different formats do not REQUIRE but authors will often avoid quantizing some things (eg, quantize sparse experts but not ffn layers or exeprt routers)
2. kv cache dtype - the loss from weight quants is constant; it snowballs in kv; use fp16/bf16

[mattwallace @mattwallace]

@jimcramer 1. trillions of dollars is not "a few bucks"
2. when has protectionism ever worked? imagine if China sold oil 87.5% cheaper and you wanted American companies to pay 8x fuel cost for everything. Does that make them more globally competitive?

[mattwallace @mattwallace]

https://t.co/AkH6JInl47

[mattwallace @mattwallace]

@mr_r0b0t you test with fp16 kv cache?

[mattwallace @mattwallace]

@CTOAdvisor Love it

[mattwallace @mattwallace]

@ebloch @omooretweets 1. outside US
2. teams/enterprise plans?

[mattwallace @mattwallace]

@petergostev From someone on my team yesterday, "I aspire to have a $2000 electric bill"

[mattwallace @mattwallace]

@stevensweden4 @thsottiaux @sama That's just the protection against deleting your home directory kicking in ;)

[mattwallace @mattwallace]

@chris_j_paxton So excited. Mine is waiting for me at home. conference is looooong

[mattwallace @mattwallace]

@jamesthomason Dickens meets hemingway meets po bronson.

[mattwallace @mattwallace]

Off my normal AI topics, but if you're into M:tG or gaming in general, my friend Brian (also a co-founder of GGG / @pathofexile ) interviews Richard Garfield on his burgeoning gaming channel. https://t.co/lOnsMAcFu8

[mattwallace @mattwallace]

Hot take: Qwen-27B is good enough for most tasks

[Niels Rogge @NielsRogge]

Hot take: GLM-5.2 is good enough for most tasks

[mattwallace @mattwallace]

@WescheNex1q @NVIDIAAI As long as it's 64 agents, love it. 64 people would be tearing their hair out. ;)

[mattwallace @mattwallace]

@GregKamradt @spencerschiff_ ahhhh, right. I forgot about that "no zdr for you" thing.

[mattwallace @mattwallace]

@GregKamradt interesting that they didn't want to run this.

[mattwallace @mattwallace]

@jun_song "quality loss" seems ambiguous given error accumulation etc.

dsa/mla support so glm52 at longer ctx is not a wreck?