CrystalAI

[CrystalAI @crystalAIorg]

@fulcrumML What you cannot see, you cannot improve. This is important stuff and there's no better team to do it.

[CrystalAI @crystalAIorg]

https://t.co/0eU4DNrNWb

[CrystalAI @crystalAIorg]

CWICLlama-A248M beats Google's Gemma3-270M in evals, and in practice.

"Thinking different" has enabled us to build models that can compete readily with models from the latest Gemma3 family from Google!
#thinkdifferent #crystalai #cwic #gemma #google

[CrystalAI @crystalAIorg]

Find out how you can do this too in our blog and GitHub release!

[CrystalAI @crystalAIorg]

@sergiocoju Reach out to [email protected]!

[CrystalAI @crystalAIorg]

LLM sparsity promises faster and cheaper inference, but current methods can’t cross 50% sparsity without performance collapse… until now.

Introducing CWIC, a trainable sparsity paradigm that beats SOTA methods, enabling 80% sparsity and 4x+ speedups on CPU. Here's how it works…

[CrystalAI @crystalAIorg]

@_avichawla What's described here is network pruning that's been around since the '90s. If you're interested in this our recent work "compute where it counts" (CWIC) should excite you!
https://t.co/Mm82rKLtWT

[CrystalAI @crystalAIorg]

LLM sparsity promises faster and cheaper inference, but current methods can’t cross 50% sparsity without performance collapse… until now.

Introducing CWIC, a trainable sparsity paradigm that beats SOTA methods, enabling 80% sparsity and 4x+ speedups on CPU. Here's how it works…

[CrystalAI @crystalAIorg]

Huggingface models are now live!
We are working on adding triton kernels for decoding at https://t.co/OhaQ8a83ZH !

[CrystalAI @crystalAIorg]

LLM sparsity promises faster and cheaper inference, but current methods can’t cross 50% sparsity without performance collapse… until now.

Introducing CWIC, a trainable sparsity paradigm that beats SOTA methods, enabling 80% sparsity and 4x+ speedups on CPU. Here's how it works…

[CrystalAI @crystalAIorg]

If you are excited about this work, we’d love to hear from you: [email protected].
Read more on our blog: https://t.co/6fUNQlslWG
We’re open sourcing our method at https://t.co/OhaQ8a83ZH and releasing model checkpoints on HuggingFace at https://t.co/RwX2SSm9el

[CrystalAI @crystalAIorg]

5/5: By letting the model decide where it's FLOPs are most valuable, we see that the model dedicates very little compute to formatting tokens like <|start_header_id|>user<|end_header_id|>, system prompts and repeated/quoted information!