Top Tweets for #modelServing
Just wrapped up an incredible experience at @databricks DevConnect Bangalore, a day full of deep dives, future-ready ideas, and powerful conversations around data & AI!
#Databricks #DevConnect #ModelServing #CostOptimization #DataEngineering #BangaloreTech #AIInfrastructure
State of the Model Serving Communities - August 2025
https://t.co/RdtuO4ryve
#OpenSource #Kubernetes #AI #Inference #ModelServing #RedHat
🚀Excited to see our collaboration with @lmsysorg bring Multiple Token Prediction (MTP) in SGLang to production!
Proud to support faster, smarter open-source LLM serving. #EigenAl #MTP #SGLang #LLMinfra #ModelServing #DeepSeek #OpenSourceAl #AskChatGPT
🚀 Summer Fest Day 5: Multiple Token Prediction in SGLang by @Eigen_AI_ and SGLang Team
1.6× throughput, same quality — open-source & production-ready!
We’ve integrated MTP into SGLang, unlocking up to 60% higher output throughput for models like DeepSeek V3, with zero quality trade-offs.
Key highlights:
- Plug-and-play MTP for any SGLang-served LLM
- Works with Expert Parallelism, PD disaggregation & CUDA Graph
- Draft-then-verify decoding with full model consistency
- 1.6× boost in small clusters, +14% at scale
- Easy tuning via draft_token_num; monitor acceptance length for max gains
Serving LLMs at scale? Don’t leave performance on the table👇 #SGLang #MTP #LLMInfra #ModelServing #DeepSeek #OpenSourceAI #AIInfrastructure #EigenAI
#genai #modelserving FEDML’s Five-layer Model Serving Platform!
FEDML Nexus AI platform (https://t.co/HWftJA1QPO) provides one of the most advanced model inference services composed of a 5-layer architecture:
Layer 0: Deployment and Inference Endpoint. This layer enables HTTPs API, model customization (train/fine-tuning), scalability, scheduling, ops management, logging, monitoring, security (e.g., trust layer for LLM), compliance (SOC2), and on-prem deployment.
Layer 1: FEDML Launch Scheduler. It collaborates with the L0 MLOps platform to handle deployment workflow on GPU devices for running serving code and configuration.
Layer 2: FEDML Serving Framework. It’s a managed framework for serving scalability and observability. It will load the serving engine and user-level serving code.
Layer 3: Model Definition and Inference APIs. Developers can define the model architecture, the inference engine to run the model, and the related schema of the model inference APIs.
Layer 4: Inference Engine and Hardware. This is the layer many machine learning system researchers and hardware accelerator companies work to optimize the inference latency & throughput.
In our newest technical blog post, we delve into the details of FEDML’ model deployment and serving framework and how developers can start using
it:
https://t.co/lA6VA01q7E
🎄 Happy Holidays! KServe v0.12 release candidate is available! Try it out!
https://t.co/Sx0FrrtIrd
#KServe #kubernetes #MLOps #DevOps #CloudNative #Kubeflow #ModelServing #AI #MachineLearning @KnativeProject @LFAIDataFdn @CloudNativeFdn
Integrated with your #Lakehouse data, #ModelServing offers automatic lineage, governance, and monitoring across, data, features, and model lifecycle.
Learn how to use Model Serving to scale from zero all the way up to your most critical needs ⬇️
https://t.co/60nVH9XBqf
Simplify model deployment, reduce infrastructure overheads AND accelerate time to production? Yes, please 🙌
See how #ModelServing on the #Lakehouse does all this and more 👇
https://t.co/zBZteAdrwH
ICYMI: #ModelServing is now generally available 🎉
Users can now deploy their models alongside existing #data and training infrastructure, simplifying the #ML lifecycle and reducing operational costs.
Check it out👇
https://t.co/k0omxN3N0f
NEW ✨ Databricks’ #ModelServing deploys #machinelearning models as a REST API – allowing you to build hassle-free, real-time ML applications for:
✅ Personalized recommendations
✅ #CX chatbots
✅ Fraud Detection
✅ and so much more!
Learn more ⬇️
https://t.co/aofW3TTwj7
🎉 @databricks' highly available, low latency Model Serving solution is now GA! Deploy your models as APIs to integrate them into your web/mobile apps. Check it out: https://t.co/slhcp35yT5 #mlflow #productionml #ml #modelserving #databricks
An exciting blog post on how to unify #realtime and #batchinference with #BentoML and #ApacheSpark!
This approach improves performance, scalability, and cost savings. Check it out at https://t.co/0w2qNqZw1d #mlops #modelserving #opensource
In case you missed it, we have an #MLOps Live (100% Q&A) session with @chaoyu_ tomorrow!
👉 Topic: solving the #ModelServing component of the MLOps stack
Any questions about solving the #ModelServing component of the #MLOps stack? Next Tuesday, @chaoyu_ will take on the challenge to answer any of your concerns.
Find out how the team @Razorpay uses #ApacheFlink for real-time #modelserving and feature generation at scale!
Read more: https://t.co/CjiVlrIwmb
#usecase #eventstreamprocessing
Find out how the team at Razorpay uses #ApacheFlink for real-time #modelserving and feature generation at scale:
Read more: https://t.co/3qIMrQGfMk
#usecase #eventstreamprocessing
Experimented model serving with @bentomlai - easy, scalable, Prometheus metrics, and health monitoring.
How about the integration with @EvidentlyAI for data and target drift monitoring?
#AI #MachineLearning #MLOps #DeepLearning #DataScience #ModelServing
Find out how the team @Razorpay uses #ApacheFlink for real-time #modelserving and feature generation at scale:
Read more: https://t.co/KpYLOBfifl
#usecase #eventstreamprocessing
Deal of the Day, Nov 24, Kubeflow in Action and select titles are on sale. Check them out: https://t.co/DcpsoLASzY
#Kubeflow #kubernetes #OpenShift #AI_ML #AIPipelines #mlpipeline #modelserving #dataprocessing #java #datastreaming #cloud #GCP #googlecloud #TensorFlow
Deal of the Day, Nov 24, Kubeflow in Action and select titles are on sale. Check them out: https://t.co/DcpsoLASzY
#Kubeflow #kubernetes #OpenShift #AI_ML #AIPipelines #mlpipeline #modelserving #dataprocessing #java #datastreaming #cloud #GCP #googlecloud #TensorFlow
Wondering about how to manage your #data for #AI?🤔
👇 Check our latest white paper on how #Hopsworks #FeatureStore manages data throughout the entire lifecycle.
#modelserving #modeltraining #machinelearning #MLOps https://t.co/jlri7SNO1g
Last Seen Hashtags on Sotwe
nudes
Seen from Vietnam
nolimit()*******************************
Seen from Egypt
samsunsahibe
Seen from Turkey
R6Y6
Seen from Germany
R8GT4
Seen from United States
น่าเย็ด
Seen from Thailand
blacksissy
Seen from Kenya
หาดใหญ่นัดเย็ด
Seen from Thailand
Orgasm
Seen from Pakistan
kayseriaktif
Seen from Turkey
Most Popular Users
Elon Musk 
@elonmusk
240.7M followers
Barack Obama
@barackobama
119.2M followers
Donald J. Trump
@realdonaldtrump
111.7M followers
Cristiano Ronaldo
@cristiano
110.9M followers
Narendra Modi
@narendramodi
107M followers
Rihanna
@rihanna
97.7M followers
NASA
@nasa
92.2M followers
Justin Bieber
@justinbieber
91M followers
KATY PERRY
@katyperry
87.8M followers
Taylor Swift
@taylorswift13
81.7M followers
Lady Gaga
@ladygaga
73.2M followers
Virat Kohli
@imvkohli
70.2M followers
Kim Kardashian
@kimkardashian
69.9M followers
YouTube
@youtube
68.7M followers
Bill Gates
@billgates
64M followers
Neymar Jr
@neymarjr
63M followers
The Ellen Show
@theellenshow
62.4M followers
CNN
@cnn
61.9M followers
Selena Gomez
@selenagomez
60.9M followers
X
@x
60.8M followers