LLMWildling

I try to make LLM's with two gpus

Do you want me to make you a llm model? let me know

[LLMWildling @LLMWildling]

Laptop died 🤷‍♂️

[LLMWildling @LLMWildling]

@SakanaAILabs Hiring in the Bay Area?

[LLMWildling @LLMWildling]

almost at 125B... MXAR way better than Adam 8 bit. Way faster. Much Speed. Learning go BRRRRRRRR

[LLMWildling @LLMWildling]

@sashadem wahhh lol I have this at home for a year, im printing models. what is this LOL

[LLMWildling @LLMWildling]

@__alpoge__ I did... and ive had this at home for about a year now...

Much faster and much more efficient than y'all

[LLMWildling @LLMWildling]

@sama I know the person who worked on this, you are very lucky to have them

[LLMWildling @LLMWildling]

Gemma 4 250B tomorrow? Started training

[LLMWildling @LLMWildling]

@googlegemma Gonna drop Gemma 4 250B this weekend.

https://t.co/EVj8sRVntA

[LLMWildling @LLMWildling]

@Mayhem4Markets @TheAhmadOsman @nvidia @NVIDIAAI If I had 1 more or 2 more. I could really crush it.

[LLMWildling @LLMWildling]

@Mayhem4Markets @TheAhmadOsman @nvidia @NVIDIAAI If needed I can put my auto researcher on it but im using up all my compute on my new 30x kv cache and im training Gemma 4 250B on my gpus.

[LLMWildling @LLMWildling]

@Mayhem4Markets @TheAhmadOsman @nvidia @NVIDIAAI I should not get distracted but should I grow nemotron 500B to 600B to get their attention on my two gpus? would be faster if they gave us kernels

[LLMWildling @LLMWildling]

@thsottiaux @ah20im wait really? does this count towards our current codex usage?

[LLMWildling @LLMWildling]

@JustinLin610 Why Gemma 4 12b when you can have Gemma 120b https://t.co/EVj8sRVntA

[LLMWildling @LLMWildling]

@demishassabis This one is more fun https://t.co/EVj8sRVntA

[LLMWildling @LLMWildling]

@AlexiGlad Better than mine?

[LLMWildling @LLMWildling]

@TheAhmadOsman It’s already here for some of us I dropped a model without Adam 8bit optimizer yesterday

[LLMWildling @LLMWildling]

@willdepue https://t.co/M3dZpqX6Kz 🎉

[LLMWildling @LLMWildling]

@willdepue its not bad, but there are much faster ways to do that

Mine is a different approach but much faster result. Im in my last round of new optimization testing, about to replace Adam optimizer for a 3x speed
https://t.co/3200r2RGHQ

[LLMWildling @LLMWildling]

Hi all today I release https://t.co/M3dZpqX6Kz

It was made with a method I call MXAR.

This is a 13x speed up in pre/post training. No LORA, just regular full weights modification.

Also update, current KV cache replacement sits at 72X compression. Soon.