Top Tweets for #LLMCompressor

about 2 months ago

Had an AWESOME conversation with the @AllThingsOpen community about how important Local LLMs are, any what tricks you can do in order to run “large” models on relatively small hardware. We talk about model compression (how the #LLMCompressor can quantize a model from 220 GB to 55 GB with <1% accuracy loss) and inference engines like @vllm_project that help developers run LLM’s fast and efficiently, across all types of hardware. Plus! Gave my 2 cents on the future role of developers in the age of AI, where the human in the loop is more important than ever to generate code (or more importantly, maintain it). ‘Twas a good time, check out the interview if you’ve got a second! 📰 Read the write up here: https://t.co/yG9f9Chg6a 🎥 Check out the interview: https://t.co/9BTCymXkbg

364

Vlad Ruso PhD @vlruso

almost 2 years ago

Neural Magic Releases LLM Compressor: A Novel Library to Compress LLMs for Faster Inference with vLLM https://t.co/d8zuG1CeaG #NeuralMagic #LLMCompressor #AIoptimization #ModelCompression #InferenceAcceleration #ai #news #llm #ml #research #ainews #innovation #artificialintel…

Top Tweets for #LLMCompressor

Last Seen Hashtags on Sotwe

Trends for you

Most Popular Users