Many enterprise GPUs run a single model during inference — even when it uses only ~30% of memory.
So how much capacity is being left on the table?
In our latest benchmark with @nebiusai, we used NVIDIA Run:ai fractional GPU allocation and NVIDIA NIM to measure real-world impact on throughput, latency, and concurrency.
What we found:
✅ 86% of full GPU capacity using just a 0.5 slice
✅ 3× more users with mixed workloads on shared GPUs
✅ Near-linear scaling down to 0.125 slices
✅ Zero latency cliffs during autoscaling
Stop GPU fragmentation. Start maximizing throughput.
🔗 Read the full deep dive: https://t.co/V51I3mi3tM
You selected the VI Admin to Platform Engineer skills session as the people's choice award winner for modern apps at @VMwareExplore in Barcelona! Add it to your schedule today, and hear @boskey and I talk about skills needed to operate app platforms at scale! #TANB1332BCN
It's up to you! You get to decide which sessions will be presented at #VMwareExplore 2024 Barcelona. So are you ready to elevate your platform engineering skills?Vote for this session from @boskey & @darinzook and we'll see you, in Barcelona.
Today is the last day to vote for the People’s Choice Award sessions at @VMwareExplore Barcelona! @boskey and myself have a session submitted for the modern apps track. We’re talking about the transition from VI Admin to Platform Engineer! Vote here -> https://t.co/EdLHngEehv
Are you going to #VMwareExplore? Sign up for the VMware Tanzu Platform for Kubernetes hands-on lab where you will have the opportunity to explore some of the key parts of managing the operations of a Kubernetes based application runtime. https://t.co/HfHHpSGdgl
Worried about securing #APIs and managing costs? Attend our session on Thursday, Aug 29th at 9AM PT at @VMwareExplore to learn how #Kubernetes and #VMwareTanzu can simplify your platform management. Let's tackle those challenges together! https://t.co/O5QAhVJmyh
The 10 Years of Kubernetes Party was inspiring! 🙌🏻
The best part was when @eric_brewer talked about how Kubernetes has been in the making for more than 30 years!
Plus, it’s always inspiring to watch folks like @kelseyhightower and @solomonstre#Kubernetes#KeberTENes
Another great release of VMware Tanzu Application Service is out the door!
It's great to work with such a talented team delivering tremendous business value to our customers! 😎