Top Tweets for #LMDeploy

9 months ago

🚀 Big news for #lmdeploy v0.10.1! 🥳Our #FP8 high-performance inference is no longer limited to the latest #GPUs. It now supports all #NVIDIA architectures from V100 onwards, bringing major speedups to more users. 🤗https://t.co/bPJfr9rz5p

intern_lm's tweet photo. 🚀 Big news for #lmdeploy v0.10.1!
🥳Our #FP8 high-performance inference is no longer limited to the latest #GPUs. It now supports all #NVIDIA architectures from V100 onwards, bringing major speedups to more users.
🤗https://t.co/bPJfr9rz5p https://t.co/x4S4HbUqZQ

1

26

5

7

2K

almost 2 years ago

🥳#LMDeploy support Llama3.1 and its Tool Calling. An example of calling "Wolfram Alpha" to perform complex mathematical calculations can be found from here. https://t.co/lcisu8fFbb 😉Welcome to star LMDeploy! https://t.co/UOPlSs6HxP #LLaMA3 @AIatMeta @Meta

0

4

2

1

767

about 2 years ago

🥳LMDeploy supports 4-bit quantization and deployment of VL models, including: #llava, #internvl, #internlm-xcomposer2, #qwen-vl, #deepseek-vl, #minigemini, #yi-vl. 😀This article will show you how to deploy a #VL model with #LMDeploy. 👉https://t.co/CdEjebXt2r

0

11

1

2

2K

about 2 years ago

🚀 Big news in #lmdeploy v0.3.0! We've optimized GQA inference, achieving 24+ RPS for #internlm2-7b & 17+ for internlm2-20b—1.8x faster than #vLLM! 🌟 Also, introducing support for #Qwen 1.5-MOE, #DBRX, & #DeepSeek-VL. 🥰Stay tuned for more! https://t.co/bPJfr9rz5p

intern_lm's tweet photo. 🚀 Big news in #lmdeploy v0.3.0! We've optimized GQA inference, achieving 24+ RPS for #internlm2-7b & 17+ for internlm2-20b—1.8x faster than #vLLM!
🌟 Also, introducing support for #Qwen 1.5-MOE, #DBRX, & #DeepSeek-VL.
🥰Stay tuned for more!
https://t.co/bPJfr9rz5p https://t.co/TxsB1mw01p

0

9

2

3

2K

over 2 years ago

🥳Thrilled to announce LMDeploy v0.1.0 is here! The inference now achieves ~900 request/s for llama-7b. 🤗A100 benchmarks link: https://t.co/fB1S4fp6Nm 🤗GitHub link: https://t.co/3fry6QNAIX #LMDeploy

1

7

0

2

802

almost 3 years ago

🥳#LMDeploy v0.0.8 has arrived with two incredible features: ✅ Access to all capabilities of #CodeLlama - including code completion, infilling, instruct / chat, and python specialist ✅ Support for both #Baichuan2-7B-Base and Baichuan2-7B-Chat 👉https://t.co/UOPlSs6HxP

OpenMMLab's tweet photo. 🥳#LMDeploy v0.0.8 has arrived with two incredible features:
✅ Access to all capabilities of #CodeLlama - including code completion, infilling, instruct / chat, and python specialist
✅ Support for both #Baichuan2-7B-Base and Baichuan2-7B-Chat
👉https://t.co/UOPlSs6HxP https://t.co/mTyqyBytM9

0

3

0

918

almost 3 years ago

🥳Say hello to the new face of lmdeploy! We hope you love it as much as we do! #lmdeploy 🤗https://t.co/UOPlSs6HxP

1

3

0

713

almost 3 years ago

🥳#lmdeploy Release v0.0.7! ✅Flash attention 2 is now supported, boosting context decoding speed by approximately 45% ✅Token_id decoding has been optimized for better efficiency ✅The gemm-tunned script has been packed in the PyPI package 🤗https://t.co/y2eB4keqak

0

14

4

2

2K

almost 3 years ago

🥳An OpenAI-like RESTful API has been integrated to #LMDeploy. #LLMInference #RESTful-API #LLM 👉https://t.co/UOPlSs6HxP 🤗A Quick guide can be found from https://t.co/maEbsmndB3

0

15

4

1

1K

almost 3 years ago

🥳Significant update in #LMDeploy. #LLMInference 🤗Building on previous work of 4-bit quantized #LLM model inference, we've now added support for tensor parallelism 🤗Support Qwen-7B and Qwen-Chat-7B with dynamic NTK-RoPE scaling and dynamic logN scaling https://t.co/UOPlSs6HxP

0

12

4

3

2K

almost 3 years ago

🥳I just published Faster and More Efficient 4-bit quantized #LLM Model Inference. #LMDeploy 👉https://t.co/iHoKuWyX6c

almost 3 years ago

🥳#LMDeploy supports 4-bit quantized #LLM model inference, 2.4x faster than FP16 on #NVIDIA Geforce RTX 4090, which is the fastest open-source implementation as far as we know. 🤗Check this guide out for more detailed deployment information: https://t.co/3s08oEtY08

OpenMMLab's tweet photo. 🥳#LMDeploy supports 4-bit quantized #LLM model inference, 2.4x faster than FP16 on #NVIDIA Geforce RTX 4090, which is the fastest open-source implementation as far as we know.
🤗Check this guide out for more detailed deployment information: https://t.co/3s08oEtY08 https://t.co/1stiA1kvFd

1

18

2

5K

0

11

2

1

2K

almost 3 years ago

🥳#LMDeploy supports 4-bit quantized #LLM model inference, 2.4x faster than FP16 on #NVIDIA Geforce RTX 4090, which is the fastest open-source implementation as far as we know. 🤗Check this guide out for more detailed deployment information: https://t.co/3s08oEtY08

1

18

2

5K

almost 3 years ago

🥳Excited to announce that #OpenCompass is compatible with #LMDeploy 🤗Support Qwen-7B and XVERSE-13B ! Go to our leaderboard for more results! 😍(Preview) The feature of comparing models will soon be available on our platform. Stay tuned for more updates! https://t.co/VsoY9s51A3

OpenMMLab's tweet photo. 🥳Excited to announce that #OpenCompass is compatible with #LMDeploy
🤗Support Qwen-7B and XVERSE-13B ! Go to our leaderboard for more results!
😍(Preview) The feature of comparing models will soon be available on our platform. Stay tuned for more updates!
https://t.co/VsoY9s51A3 https://t.co/cl71J5oVbq

0

7

0

1K

almost 3 years ago

🧐Want to quickly deploy the #Llama2 models? 😍This article shows us how to quickly deploy the #Llama-2 models with #LMDeploy. 👉Let's get started! #LLM @MetaAI https://t.co/EbhDAElLYC

0

5

1

879