Top Tweets for #mixtureofexperts
Kicking off the session was Kim et al.'s "MoEvil: Poisoning Expert to Compromise the Safety of Mixture-of-Experts LLMs" showing how one poisoned expert can subvert MoE safety. (https://t.co/TWHp3rDnmk) 2/6
#AISecurity #MixtureOfExperts

Meta's Llama 4 Scout & Maverick introduce powerful MoE multimodal AI for developers, enabling richer, more efficient applications.
#llama4 #multimodalai #mixtureofexperts #openweightmodels
The #AIscaling crisis hits hard: 770B-parameter models need 462GB RAM at Q4. Your 128GB workstation? Obsolete. Is #MixtureOfExperts the escape hatch or just delayed collapse?
https://t.co/3sLaj2Ri4Q

Thinking Machines Lab Opens Inkling 2 Small's 594B Weights After Red-Teaming
#AISafety #ArtificialIntelligence #MixtureOfExperts #OpenWeightModels #ThinkingMachinesLab
https://t.co/B5zb6f2MNS

π Strata: ~50-60 tok/s generation and 1,000+ tok/s prefill
π llama.cpp (same model): ~15-25 tok/s generation and a peak of ~300 tok/s prefill
That is roughly 2-3x faster generation and over 3x faster prefill.
#LLM #LocalAI #MixtureOfExperts #Qwen #Qwen4 #Inference

Strata runs a 125B mixture-of-experts model on a 12 GB GPU: 10 of 512 experts fire per layer, ~6B params active, so hot experts sit in VRAM and the rest wait in RAM.
Total params = storage. Active params = compute.
https://t.co/Dr2a8jNtnD #LLMs #MixtureOfExperts
π¦ #AlephAlpha released #Kolibri, an open-weight #LLM for German and English under the Apache 2.0 license. A #MixtureOfExperts with 78.1B parameters, of which only 3.46B are active per token. Trained from scratch on ~24T tokens (over a fifth German) using 768 #NVIDIA B200 GPUs in Germany and Finland. Weights on #HuggingFace #SovereignAI #opensource #AI π©πͺ
π§ 50 layers, each with 384 experts plus 1 shared expert; a router sends every token to 6 of them. It computes like a 3.5B model but needs the memory of a 78B one: about 78 GB of weights in FP8
π€ New UniBPE tokenizer (128k vocabulary) built for German compound words: "Bundesverfassungsgericht" is 2 tokens instead of 6 with GPT-5's o200k_base. On the full German Basic Law it needed ~15% fewer tokens than GPT-5's tokenizer, and tied it in English
π 262,144 tokens of native context, validated up to 1,048,576. 40 of 50 layers use 512-token sliding-window attention, every 5th layer attends to everything. RULER at 1M tokens: 63.2 vs 57.5 for #Qwen3.5 35B-A3B
π©πͺ Reasons in German on German prompts, with 4 reasoning effort levels (none, low, medium, high). AIME 2025: 87.5 in German, 96.9 in English. Trained with the Merlin-Arthur protocol to abstain: it says "I don't know" 44% of the time instead of hallucinating, vs 11.1% for Qwen3.5 35B-A3B
β οΈ Weak spots: little closed-book knowledge (14.8% correct on Omniscience), multi-turn tool calling at 39.8 on BFCL, 27.7 on Terminal-Bench 2.1 vs 39.7 for Qwen3.5. The dense Qwen3.8 27B scores higher overall (80.2 EN, 79.9 DE) at ~8x the active parameters
π§ Runs on #vLLM with Aleph Alpha's inference plugin, exposing an OpenAI-compatible API. Needs 2x A100/H100 80 GB or 1x H200/B200/B300. Best fit: on-prem #RAG over long German documents like laws, contracts and manuals
π No GPU at hand? Tesseracted Labs hosts Kolibri-1 free for anyone to try over the next few days, no setup needed: https://t.co/iinV3lMgTj
π https://t.co/ioq0BQjYJ5

https://t.co/DGktNeweLi
Moonshot AI has launched its 2.8-trillion-parameter Kimi K3 model, but a high hallucination rate might tempers its frontier-model pitch.
#AI #MoonshotAI #KimiK3 #AIModels #MultimodalAI #MixtureOfExperts #AIReasoningModels #AIBenchmarks #ChinaAI

GLM-4.7-Flash is a 30B model with 3B active params, so why did it crawl on my mini PC? Not the weights, not the quant. The attention design, and the runtime.
#localllms #mixtureofexperts...Show more

Excited to launch SKT-ST-X-0-3B by SKT AI Labs!
βA powerful 3B Parameter Mixture of Experts (MoE) model optimized for high-performance reasoning with a small footprint.
β π
βπ Try it on Hugging Face: https://t.co/X1Mwa6gKOL
β#AI #OpenSource #LLM #MixtureOfExperts #SKTAILabs

$TAO π¨UNLEASH THE KRAKENπ₯
LFG!!! ππππ
#Bittensor #TAO #ConnitoAI #DeAI #MixtureOfExperts $TAO $SOL $CONNITO
Weβre excited to share the Connito whitepaper V1: a framework for decentralized, composable MoE adaptation.
We trains sparse expert subsets, validates updates through Proof-of-Loss, and turns open-model improvement into a distributed expert-level market.
Read the whitepaper: https://t.co/KKkGVnMIRF
$TAO π¨WHITEPAPER π₯
ConnitoAI SN 102 β¬οΈβ¬οΈ
#Bittensor #TAO #ConnitoAI #DeAI #MixtureOfExperts $TAO $SOL $CONNITO
Weβre excited to share the Connito whitepaper V1: a framework for decentralized, composable MoE adaptation.
We trains sparse expert subsets, validates updates through Proof-of-Loss, and turns open-model improvement into a distributed expert-level market.
Read the whitepaper: https://t.co/KKkGVnMIRF
TrendToKnow AI: A.X K1 Technical Report
π₯ Sung Jun Cheon, Jaekyung Cho, Seongho Choi et al.
#AIResearch #MachineLearning #NLP #MixtureOfExperts
Provided by TrendToKnow AI
π https://t.co/LKJEWWPkfp

AMD has announced that Zyphra has successfully trained ZAYA1, a large-scale Mixture-of-Experts (MoE) foundation model, marking a significant milestone in AI development.
https://t.co/0rGYt25qhw
#AIInnovation #MixtureOfExperts #AMDTechnology
Class 04 Assignment
What is mixture of expert's (MOE)
What is prompting framework and six part of prompting framework
#MixtureOfExperts
#PromptingFramework
#ZeroShotPrompting
#OneShotPrompting
#FewShotPrompting
#SystemPrompting
#RolePrompting
#contextualPrompting
@0xAsharib
Class 04 Assignment
What is mixture of expert's (MOE)
What is prompting framework and six part of prompting framework
#MixtureOfExperts
#PromptingFramework
#ZeroShotPrompting
#OneShotPrompting
#FewShotPrompting
#SystemPrompting
#RolePrompting
#contextualPrompting
@0xAsharib
Class 04 Assignment
What is mixture of expert's (MOE)
What is prompting framework and six part of prompting framework
#MixtureOfExperts
#PromptingFramework
#ZeroShotPrompting
#OneShotPrompting
#FewShotPrompting
#SystemPrompting
#RolePrompting
#contextualPrompting
Class 04 Assignment
What is mixture of expert's (MOE)
What is prompting framework and six part of prompting framework
#MixtureOfExperts
#PromptingFramework
#ZeroShotPrompting
#OneShotPrompting
#FewShotPrompting
#SystemPrompting
#RolePrompting
#contextualPrompting
In short:
Mixture of Experts = intelligent scaling.
Instead of one giant model doing everything, it uses the right experts at the right time, powering the next generation of efficient LLMs like Gemini, GPT-4, and Mistral.
#MixtureOfExperts #MoE #DeepLearning #MachineLearning #AIArchitecture #LLM #ScalableAI #AIInnovation #SmartScaling
Happening Now: A great panel as a pert of the mixer of experts series at @NYSE with @Vultr, CMO, Kevin Cochrane and @DDNintelligence CMO, Jyothi Swaroop, led by @furrier of @theCUBE. Discussion is passionate and informative. Lots of regencies to dot com and what came out of it and now AI and what will come out of it.
My POV, we all know where we are headed, the part we donβt know is βhow longβ it will take to get there and who (which vendors) will survive on the journey. Economics rules at the end of the day.
Mixer of the Experts series of hosted by @NYSE Wired Community, led by @bjbaumann2014.
cc @dvellante @RealStrech @nyike
Last Seen Hashtags on Sotwe
thevidiyaamma
Seen from United Kingdom
SonicMovie4
Seen from United States
artistasgsm
Seen from United States
DigitalGoldReserve
Seen from United States
δΈη¬γγγͺ
Seen from Korea
minichat
Seen from United States
Intrusive
Seen from United States
flowerandartsfestival
Seen from United States
arabcouple
Seen from United States
artwalksjc
Seen from United States
Trends for you
Most Popular Users

Elon Musk 
@elonmusk
240.4M followers

Barack Obama 
@barackobama
118.7M followers

Cristiano Ronaldo 
@cristiano
114.4M followers

Donald J. Trump 
@realdonaldtrump
111.8M followers

Narendra Modi 
@narendramodi
107M followers

Rihanna 
@rihanna
98.7M followers

NASA 
@nasa
92.2M followers

Justin Bieber 
@justinbieber
91.7M followers

KATY PERRY 
@katyperry
90M followers

Taylor Swift 
@taylorswift13
83.9M followers

Lady Gaga 
@ladygaga
75.4M followers

Virat Kohli 
@imvkohli
73.3M followers

Kim Kardashian 
@kimkardashian
70.8M followers

YouTube 
@youtube
68.7M followers

Neymar Jr 
@neymarjr
66.3M followers

Bill Gates 
@billgates
65.2M followers

Selena Gomez 
@selenagomez
63M followers

The Ellen Show
@theellenshow
62.1M followers

CNN 
@cnn
61.7M followers

X 
@x
60.6M followers



















