Next Thursday, October 3, join us for bi-weekly @vllm_project office hours to learn about the new AI Powered vLLM Semantic Router.
GitHub: https://t.co/tqHesjOXKd
Register for office hours: https://t.co/X8hAHYRBgT
New on the Red Hat Developer blog: vLLM Semantic Router – Improving Efficiency in AI Reasoning
An open source router that sends the right query to the right model, reducing cost and latency while maintaining accuracy.
Read here: https://t.co/egzeam4FHa
Austin friends - we’ve got something special coming up on Wednesday, September 17 at @CapitalFactory!
PyTorch ATX + vLLM are teaming up for a joint meetup. Over 200 folks are already signed up, and there’s still a few spots left. Come hang out, learn, and meet others building with LLMs. Here’s what’s on the agenda:
1⃣ Getting started with vLLM inference - Steve Watt, PyTorch ambassador (@wattsteve)
2⃣ Intermediate vLLM: PagedAttention, quantization, speculative decoding & more - Luka Govedič, vLLM committer (@luka_govedic)
3⃣ Semantic Router: Auto-reasoning router for efficient mixture-of-models inference – Huamin Chen, project creator (@root_fs)
4⃣ Scaling vLLM with Kubernetes + llm-d - Greg Pereira, maintainer
Sign up here: https://t.co/FPNaAoBXII
Please help us share the event 🙏
Next up in emerging tech - @somalley108 shows us how she uses Project Kepler to monitor power consumption in resource-constrained edge devices: https://t.co/Jdf8jTukin
In this episode of Technically Speaking with @kernelcdub, we explore the possibilities of more sustainable #cloudnative workloads with @KeplerProject and its co-founder @root_fs. Watch Now: https://t.co/s6zqYhvQva
What's next in green computing? @piparul and @root_fs explain the ins and outs of Project Kepler, the CNCF sandbox project that is bringing power monitoring to Kubernetes and beyond! https://t.co/Gu0NXEWX0t
The five steps to deploy #cloudnative sustainable foundation #AI models starts with the obvious two: #containers to manage the workloads and #Kubernetes to deploy across a distributed infrastructure. We talked about all the steps with @root_fs of @redhat. https://t.co/YX7KiUFQCh
@johanngyger @CaraDelia@root_fs Absolutely agree, I can say at least the folks talking today genuinely care. On the Microsoft and Azure side, we're doubling down in this space, more right after this and throughout summer!
Get ready to explore sustainable computing with Kepler - the Kubernetes-based Efficient Power Level Exporter! Listen to an insightful conversation with @root_fs on Cloud7's Red Hat series. #sustainablecomputing#Kepler#Kubernetes
https://t.co/8g2nziqErG
Accelerate Sustainable Computing with Community Collaboration - Cara Delia, Principal Community Architect Financial Services and Sustainability, Red Hat & Huamin Chen, Senior Principal Software Engineer, Red Hat at KubeCon + CloudNativeCon Europe 2023 https://t.co/u8Ar7sf6xF
THIS WEDNESDAY is the next meeting.
We will have Kepler (https://t.co/YIBOVmzvI9) presenting for the first half hour followed by KubeCon prep discussions.
At #MobileWorldCongress, Red Hat and IBM Research launched an open source project to capture power usage metrics from #Kubernetes clusters.
https://t.co/328z3vfbo3
Great to get an impromptu briefing from @RedHat on how operators are able to deploy K8S packaged workloads to the far edge using Microshift, Podman or OpenShift.
Will hopefully be getting more details.
#MWC23#redhat