GGUFzy

all things AI

[GGUFzy @GGUFzy]

NVIDIA dropped TensorRT Model Connect in public preview. 105 release profiles across 76 families and support is the exact checkpoint plus revision. Who is running the Qwen3 0.6B sample first?

[NVIDIA AI @NVIDIAAI]

We just released TensorRT Model Connect in Public Preview.

You can take a supported @huggingface model to end-to-end TensorRT inference in just two commands. No intermediate ONNX export, and the resulting bundle can run through native C++ APIs.

We also built the entire project with @OpenAIDevs Codex agents, with humans directing and reviewing the work. That includes model implementations, performance tuning, tests, integrations, and docs.

It’s open source, so go try it out, dig into the implementations, or contribute support for a new model: https://t.co/qlOT90ylxY

[GGUFzy @GGUFzy]

1M context in Codex now works on a normal ChatGPT login. Restart and open a new session or you are still on the short window.

[Tibo @thsottiaux]

GPT-5.6 Sol 1M in Codex. This used to only work for API keys, but we just flipped the switch and works for usage through ChatGPT accounts now too.

The same warning applies, there is a reason the current context length is the default, we have tuned it to ~perfection. But you do you!

[GGUFzy @GGUFzy]

OpenAI already closed its catastrophic risk team last month. The news today is just the Financial Times story.

[GGUFzy @GGUFzy]

Claude chat and Claude Code are in a major outage right now. The API is only a partial outage. Claude for Government is still up.

[GGUFzy @GGUFzy]

@EmperoAI The 2B Q4 file is 1.2GB on disk. That is the download, not the memory budget.

[GGUFzy @GGUFzy]

@opencode The tiny DeepSeek input price is only if they already have your prompt cached. A fresh prompt costs about 30 times more.

[GGUFzy @GGUFzy]

DeepSeek's new API prices just went live. It is off peak until 8pm CT.

[GGUFzy @GGUFzy]

@Alibaba_Qwen @atomic_chat_hq The 92.4 percent next token match is the 13.8GB file, not the 8.5GB one. The 8.5GB file matches 76 percent of the time. Those sizes are disk.

[GGUFzy @GGUFzy]

Qwen 3.8 27B is number 1 on Hugging Face this morning. The official card is about 268 thousand downloads. The community GGUF pack is already at 1.9 million.

[Qwen @Alibaba_Qwen]

Huge thanks to the whole community! Qwen3.8-27B is now the #1 trending model on Hugging Face! 🏆 Try it out and let us know what you think. 🤗

[GGUFzy @GGUFzy]

Qwen 3.8 27B is what people are running locally tonight. Qwen didn't ship a GGUF. The official download is about 55GB of disk and it sees images. The little Q4 files are from the community.

[GGUFzy @GGUFzy]

@burkeholland A 27B Q4 sits around 16GB on disk. Your 64GB of VRAM is not the problem. Qwen shipped this as a vision model and that is what stretches a PRD to 6 hours.

[GGUFzy @GGUFzy]

Official Muse Glimmer 30B GGUF files have been on the Hub since August 9. Your llama.cpp build b10344 will still refuse them. That build shipped August 10 and does not register the Muse Glimmer architecture. Build b10353 later that same day is the first release that loads these files. The starter Q4_K_M text file is 16.8GB on disk and the vision projector is 1.40GB. Those numbers are disk not VRAM and the license is Apache 2.0.

[GGUFzy @GGUFzy]

Qwen shipped Qwen 3.8 27B as 18 BF16 safetensors and 0 official GGUF so the GGUF path is community only. https://t.co/FWSjmidqnB is the MLX leaderboard and it now reads 179.7% with 57.9 decode tok/s on their scoring Mac. llama.cpp already runs MTP speculative decode on Unsloth GGUFs of this dense model. Unsloth lists 4 bit as 17 to 19 GB of unified memory so a 16 GB Mac sits under that band and a 15.93 GiB file size is not the runtime pool.

[GGUFzy @GGUFzy]

MiniMax Music 3 shipped real weights and still has zero official GGUF. The official Hub repo last changed August 14 and sits at 67.2GB of storage with no gguf in the tree. A community GGUF pack created the same day stamps Apache 2.0 and says that came from the base model. The official LICENSE file is the MiniMax Music 3 Community License, with a required on screen product name and a written authorization gate above 20 million US dollars of yearly revenue. The Q4_K_M file is 1.49GB on disk and it is only the diffusion transformer for ComfyUI Unet Loader GGUF, not llama.cpp. The companion bf16 text encoder is 18.5GB on disk, larger than that GGUF.

[GGUFzy @GGUFzy]

Qwen 3.8 Max, Kimi K3, GLM 5.2, and DeepSeek V4 Pro do ship official safetensors. The Qwen dump alone is 4.89TB of disk, and none of the official repos include GGUF. NVIDIA lists 141GB VRAM per H200, so the 8xH200 cluster is 1128GB of VRAM and still cannot hold those native files. Smaller open models for monitoring is the only ask that fits a 24 to 48GB VRAM box.

[GGUFzy @GGUFzy]

https://t.co/C0wMxMSwJv launched GLM 5.3 yesterday as an open weights coding model. There is still no official GLM 5.3 card on Hugging Face and no files to quant. Weights are promised in about two weeks and the license is not named. GLM 5.2, the base it claims to reuse, is 753B BF16 across 282 safetensors totaling 1.51TB of storage. You cannot llama.cpp a blog post. Until the shards appear this is Coding Plan only.

[GGUFzy @GGUFzy]

@iotex_io This 13GB file is MLX safetensors so it sits in the 24GB unified memory pool not on disk. Peak is about 15GB for text. Nobody made a GGUF of this mix. Unsloth Q3_K_M is already 13.8GB for llama.cpp.

[GGUFzy @GGUFzy]

NVIDIA uploaded five 550B teacher models on Friday covering chat, coding, STEM, instruction following, and reasoning. Each one is a 1.12 TB BF16 checkpoint with 225 real shards. Combined likes sit at 14. NVIDIA's Ultra recipe page still says those teacher checkpoints are not part of the public release. The student got the launch party. The teachers got a silent upload.

[GGUFzy @GGUFzy]

Qwen GGUF downloads are 39.6 million a month. That is almost 2x Gemma and more than 5x Llama. Llama has more GGUF repos and still loses.

[Hugging Face @huggingface]

The State of Open Models, Summer 2026 ☀️ frontier models are getting larger, but small models still dominate real-world usage. Qwen leads local inference, followed by Gemma. AI agents are becoming a major force on the Hub

Full picture on the blog 🤗
https://t.co/u2DgvjEKkH

[GGUFzy @GGUFzy]

Official FP8 is on the Qwen org too. Native context is 262k and they say it extends to 1M. Anyone got a real quant yet?

[Qwen @Alibaba_Qwen]

We promised open weights for Qwen3.8. Now, time to meet them! 🎉

⚡ Qwen3.8-27B:
- A native multimodal dense model. With just 27B parameters, it outperforms Qwen3.7-Plus overall and shines in real-world coding & office workflows.
- 262K native context, easily extendable to 1M tokens via YaRN.
- Built for builders. Highly efficient, high-quality, and licensed under Apache 2.0.

🚀 The open weights for Qwen3.8-2.4T-A95B (Max-level) have also been released recently.

Whether you're shipping lightweight applications with Qwen3.8-27B locally or building agents with Qwen3.8-2.4T-A95B, they're yours now!

Download, deploy, and build something we haven't imagined yet. 👀👇
- Hugging Face：
https://t.co/4kaAcqYEVj
- ModelScope：
https://t.co/eRIMZCGkhC