@jauneclaude I don't think we currently have the know-how to train MoE routers to be sparse like that. What you suggest would require insane levels of parsity.
But expect more models to use n-grams, which can be easily offloaded to the SSD.
@kusurichan_ DeepSeek V4.1 Flash, GLM 5.3 Flash, Inkling 2 Small and the upcoming Ernie 5.5 Flash Preview are good options to try. See which one fits you best.
@buildwithhassan Possibly https://t.co/ZNha1AkxIz? They take good care to test models out first with their synbad framework. You can use my referral code: https://t.co/jzPWyaRgft
@francoisfleuret Based on what I could infer from the Jalapeno chip report OpenAI uses. ChatGPT 6 likely uses DeepSeek V2 inspired latent attention, likely mixed with SWA and looped transformers. Otherwise, I expect regular GQA + SWA in other frontier models.
@someRandomDev5@SipeedLab@minviable_org I am not so sure about that. As there are plenty of MTP/speculative drafting methods. No MTP comparison is still apples to apples.
@charles_maddock C++ codebase replaced by gluing together few Python libraries. Same performance in real world scenarios, except startup is 0.1 seconds slower.
To celebrate @XSEEDGames PC release of #TrailsOfColdSteel2 I will be doing another Trails giveaway, 1 lucky person gets the chance to win it on #ValentinesDay from yours truly by retweeting this post. Sharing the love of Trails is the best love to share afterall