Everything you need to start self-hosting an open LLM.
Run it on your own hardware. No API keys. No per-token bill. Nothing leaves your machine.
The full path with @vllm_project: batch inference in Python, an OpenAI-compatible API server in one command, and quantized models that cut an 8B from ~16GB of weights to a quarter of that while keeping 98-100% accuracy.
Walkthrough by @cedricclyburn.
Have to keep #AI systems up and running? Help’s arrived. And it's not just practical, it's free. Hear from @RedHat authors how to manage #genAI apps in production and operationalize AI workloads at scale: https://t.co/O30LmV9H2V. Then download their ebook: https://t.co/Zeebo3Dfnk.
Stop experimenting and start scaling. 🚀 Move your AI projects from the lab to the ledger with open source infrastructure designed for production.
Read the blog to explore our highlight AI sessions and register for Red Hat Summit today! https://t.co/SajHzGH5Lf
Hosted a hackathon this weekend with 90 engineers building AI systems with @vllm_project and @llm_d.
Speculative decoding architectures, RAG pipelines, multimodal assistants. vLLM contributors in the room helping throughout.
📹 @LegareKerrison with the recap. 🎉
Llama 70B as a cloud endpoint costs exponentially more than Llama 8B.
For teams where a smaller model meets the quality bar, that gap is hard to ignore. And with INT4 quantization: 4x smaller, 2x faster, less than 1% accuracy loss.
The right model isn't always the biggest one.
https://t.co/23IHcSmDkk
Join us at ClawCon Boston next week, Wednesday, April 29th. @CJNuland and Red Hat's OpenClaw maintainer @somalley108 will share how to safely scale OpenClaw in the enterprise with semantic routing and Kubernetes. See you there! Register now: https://t.co/NqCrDGGuIF
Get certified sooner ↗️ We’ve launched the new @RedHat Certified Technologist in @OpenShift Exam (EX180) to provide a quicker path to your first certification. Build your foundation before moving onto advanced topics. Enroll today: https://t.co/2c2WGjmRBF
The #AI revolution is here, transforming everything from national economies to developer experience. Jon Hammant of @AWS shares his excitement about the incredible pace of change & the fundamental benefits AI brings to end customers.
https://t.co/fir2NAEciL #AWSSummit
Check out our course, "@RedHat@OpenShift#Virtualization Administration II: Configuring Production Virtual Machines (DO256)." This training is designed to help you move beyond the basics and prepare VM workloads for production-grade performance. https://t.co/9GVDqq5rh6
Looking to modernize? Your next step should be #RHSummit. The agenda is packed with #virtualization sessions that can help propel your business. But you have to be there. Register for Summit today (https://t.co/HjUqfJNWCi), and start building your agenda (https://t.co/5mof45UMK7).
OpenClaw runs with your user-level permissions by default, meaning it inherits access to your GitHub token, Slack creds, filesystem, local network.
On Red Hat OpenShift AI: container isolation, default deny networking, secrets management, and OpenTelemetry traces.
Here's how:
NEW course for @RedHat OpenStack administrators! Learn how to deploy and manage #RedHat OpenStack Services on @OpenShift and its external data plan nodes in our new OpenStack Administration II course: https://t.co/vwRO3NXm23
Explore our blog post for practical tips and strategies to protect your systems from potential threats. Don't miss out on this valuable resource! https://t.co/9C7WDNrVUe
Self-hosting LLM’s typically includes a set of requirements for our infrastructure, namely:
🏎️ Hardware accelerators
⚙️ System & model configuration
📚Specific libraries or dependencies
That’s why at @VoxxedCERN I was honored to demo https://t.co/upqfkipp9I, a project that uses @Podman_io and other container engines to run and serve AI models as OCI images! Wanna see how it works? Full recording linked ⬇️
🍽️ This OpenShift Virtualization cookbook has the ingredients you need for deploying #virtualization workloads for those who are just starting with @OpenShift. Get started today: https://t.co/nRYtevxrUH
Warsaw @vllm_project meetup recordings are live 🇵🇱
Video 1: vLLM roadmap, JetBrains AI in IDEs, and NVIDIA Flex Tensor
Video 2: vLLM Omni for multimodal output and @_llm_d_ for distributed inference on Kubernetes
5 sessions. All technical. Thread below 👇
🍽️ This OpenShift Virtualization cookbook has the ingredients you need for deploying #virtualization workloads for those who are just starting with @OpenShift. Get started today: https://t.co/nRYtevxrUH
AI is difficult to learn, but not anymore!
Introducing "Machine Learning Systems " PDF.
You will get:
• 2620+ pages
• Save 100+ hours on research
And it's 100% FREE!
To get it, just:
• Like and retweet
• Comment " ML "
• DM me in the message for the link
Turn any FastAPI app into an MCP server!
FastAPI-MCP is a zero-config tool for automatically converting FastAPI endpoints as MCP tools and use them with Clause, Cursor or any MCP client.
100% open-source.