Red Hat AI

Accelerating AI innovation with open platforms and community.
The future of AI is open.

[Red Hat AI @RedHat_AI]

Everything you need to start self-hosting an open LLM.

Run it on your own hardware. No API keys. No per-token bill. Nothing leaves your machine.

The full path with @vllm_project: batch inference in Python, an OpenAI-compatible API server in one command, and quantized models that cut an 8B from ~16GB of weights to a quarter of that while keeping 98-100% accuracy.

Walkthrough by @cedricclyburn.

[Red Hat @RedHat]

Red Hat researchers, in collaboration with Purdue, developed VEX-Bench—a benchmark for assessing large language models (LLMs) in software supply chain vulnerability triage. Traditional tools often generate up to 70% false alerts. Discover the findings: https://t.co/BmrljadCoG #EMNLP2026

[Red Hat AI @RedHat_AI]

Amazing ~4-minute walkthrough from @LegareKerrison of how @NVIDIAAI's OpenShell sandboxes autonomous agents using declarative security policies, policy as config, not code buried in the app. https://t.co/l6u4VZ8nrV

[llm-d @_llm_d_]

Today at PyTorch Conference 2026, we announced llm-d 1.0! 🎉

Distributed inference is too complex to solve behind closed doors.

17 months in, llm-d is now a @CloudNativeFdn project powering fleets worldwide.

Here’s what’s inside 1.0 🧵👇

[Red Hat AI @RedHat_AI]

That's the watch list. Our experts show up regularly on IBM Technology with explainers on inference, agents, and open-source tooling. Full channel:
https://t.co/E4AQ23fz5E

[Red Hat AI @RedHat_AI]

When does vLLM beat llama.cpp? What happens when a model outgrows one GPU? Can you trust an agent benchmark? Where do MCP and agent skills fit?

Red Hat AI folks have been answering these questions on @IBM's Technology channel on YouTube.

A 🧵 of deep dives for your watch list:

[Red Hat AI @RedHat_AI]

Want to become an AI engineer? @cedricclyburn covers the foundations: Python, APIs, RAG, embeddings, agents, observability, and deployment.
https://t.co/hn6mobcr87

[▲ Marktechpost AI @Marktechpost]

▲ Google, Red Hat and IBM Devs just released llm-d 1.0, turning a fleet of vLLM and SGLang servers into one coordinated inference pool on Kubernetes.

The key idea is cache-aware routing. The router tracks which replica already holds each agent's prefix and sends the next step there. When GPU or TPU memory fills, KV cache spills to host RAM, SSD or Managed Lustre instead of being evicted.

Google reports Gemini's prefix cache hit rate doubled from 35% to 70%, and a 17,000-node GKE benchmark scaled throughput 2.9x. It is a trade-off though: every number is self-reported, and RL time-slicing still leaves roughly 30% of accelerator time idle.

Apache 2.0, CNCF Sandbox, runs on NVIDIA, AMD, Intel GPUs and Google TPUs......

Full analysis: https://t.co/NSi0TgLQuv

Repo: https://t.co/pc81smywtL

Docs: https://t.co/Oj7dMDTD0A

Technical details: https://t.co/7JGfhsE2S4

@GoogleAI @googleaidevs @RedHat @RedHat_AI @IBM @IBMData @IBMResearch

[Red Hat AI @RedHat_AI]

Meta's Muse Spark Open, 816B, runs on Red Hat AI Inference Server (preview). The detail worth knowing: its DSpark speculator drafts exactly 7 tokens, and there's a real reason it stops there. Suffix decay, explained: https://t.co/s7NGDGzDcL

[Red Hat AI @RedHat_AI]

Reflection used @vllm_project's LLM Compressor to quantize Beam 1.0, their 501B open MoE, into an FP8 checkpoint. Always good to see a major open model reach for it to ship production weights. That's what building it in the open is for. Nicely done, @reflection_ai:
https://t.co/vhRep3HrdO

[Red Hat AI @RedHat_AI]

Best part: nothing to install, no hardware of your own. All you need is a free @RedHat Developer Sandbox account. You run it in JupyterLab on OpenShift AI, scanning Granite 3.1B.

Start here and let us know what you are thinking:
https://t.co/eMpYzHEPHS

[Red Hat AI @RedHat_AI]

Red teaming a model means attacking it on purpose: prompt injections, jailbreaks, unsafe outputs, before real users or attackers find them. Your chatbot works great in the demo. Red teaming is how you find out where it breaks.

A beginner path to run your first scan, a 🧵:

[Red Hat AI @RedHat_AI]

The path walks the full loop: run a probe, read the model's failure rate, decide if it needs guardrails, then design and apply targeted ones and scan again until it clears your safety bar. It's the real red-team workflow, scaled down to one model for learning purposes.