BCat

Co-Founder of 1CatAI YMZX
Building local AI infra, multi-GPU workstations & weird little AI products.

[BCat @barryxv6]

GPU from 2017 just pushed Qwen3.8-27B to 251 tok/s.

1Cat-vLLM 1.5.0 is out. 🐱

4× Tesla V100:
⚡ Qwen3.8 DFlash2: 206–251 tok/s
⚡ 256K decode: 50.38 tok/s
⚡ Long-prefill Attention: ~60.8 TFLOP/s

V100 isn’t getting newer.
The software is getting better.

Make Volta Fast Again.

https://t.co/WbrDoBGIUF

[BCat @barryxv6]

在早期的验证方案中，我们在纠结用风冷还是水冷给V100散热，于是这样一个半分冷半水冷的机器就诞生了……
（我还给他做了个太极图）

[Hikari∣LocalLLM⚡ @Hikari_07_jp]

Hey brother!!
Apparently, people who use weird custom-built PCs get a lot of engagement if they post photos.
Show us👀

[BCat @barryxv6]

@PSueoka55133 unsloth nvpf4 is fine
https://t.co/yyXZafuJtQ

[BCat @barryxv6]

最好是重置卡啊。。。怎么跟我的周重置时间撞车了

[Tibo @thsottiaux]

Ladies and gentlemen... start... your... ENGINES. We are almost Tuesday and I promised a reset for Tuesday. Among some other things. See you soon.

[BCat @barryxv6]

比起量产的工作站，似乎大家更喜欢颇具“工匠精神”的原型机呢，也多多关注一下量产机型吧！

[BCat @barryxv6]

不得不掏出早期的V100原型机了

[BCat @barryxv6]

每日一猫

[BCat @barryxv6]

@UtaAoya above 4g 是必须要打开的

[BCat @barryxv6]

之前接Grok4.6到codex里用会有工具调用协议兼容不完整的问题，不知道Grok4.7的情况咋样

[BCat @barryxv6]

@unbug 四卡的工作站和显卡坞已经市售上千台了，还没有听说过烧卡的。
目前我们没有单独出售过底板，市面的方案也不是我们设计的，可能是仿品和不良品，认准1Cat logo

[BCat @barryxv6]

https://t.co/HA10WmCFaN

[BCat @barryxv6]

@tenhkspark 实际上跑模型运行总功率只有600W左右，没有想象中那么大；
我们做的1Cat Studio支持功耗调控，功耗甚至可以更低

[BCat @barryxv6]

@PSueoka55133 在造出这样的电热炉后，我相信pcie通道不是问题。（图为16 * V100）

[BCat @barryxv6]

不得不掏出早期的V100原型机了

[Ryu @_ryu15_]

奇妙な自作PCを使ってる人はハードの写真を上げると伸びるらしい

[BCat @barryxv6]

综合表现unsloth nvfp4最佳

prompt： 创建一个HTML，内容是SVG绘制一个鹈鹕骑自行车的2D动画，你不需要任何测试创建一个HTML，内容是SVG绘制一个鹈鹕骑自行车的2D动画，你不需要任何测试

[BCat @barryxv6]

同样的模型Qwen3.8 27B，不同量化输出的结果差别真的很大

视频分别是不同量化水平的效果
UNSLOTH-NVFP4-thinking
FP8 thinking
BF16 thinking

[BCat @barryxv6]

FDE 也要下乡了吗？已经接到了县域和乡域的客户需求…

[BCat @barryxv6]

@Ionkosm Who parks his car on my desk?

[BCat @barryxv6]

Absolutely

[Marco Franzon @mfranz_on]

No local AI model has ever escaped from his sandbox as far as I now

[BCat @barryxv6]

@BarryXv2 yep

[BCat @barryxv6]

🚀 We built 1Cat-vLLM-Gaudi and open-sourced it.

Gaudi2 is Intel’s AI accelerator with 96GB HBM2E per card, 2.45 TB/s memory bandwidth, and native FP8 support — a compelling option for LLM inference if you’re willing to get your hands dirty.

The challenge is software: model support, kernels, compilation, and multi-card execution still need serious work.

That’s where 1Cat-vLLM-Gaudi comes in.

Based on vLLM / vLLM-Gaudi, we’re making Gaudi2 a more practical inference platform.

What we’ve built

🔹 Model support

Qwen3.5 / 3.8
DeepSeek V4 / V4.1 Flash
MiniMax H3 audio/video generation

🔹 Native kernels & performance optimization

100+ DeepSeek-related TPC kernel source files
Weight layout, build, and validation tooling
FlashInfer-Gaudi
Optimizations for GDN, quantization, normalization, and state I/O

🔹 Multi-Gaudi execution

2-card TP2
4-card TP2 × PP2
Regional compilation + Native Replay
Reduced communication stalls, state movement, and per-token submission overhead

On our archived Qwen3.8-27B-FP8 single-Gaudi2 benchmark, compared with the previous vLLM-Gaudi GDN path:

→ GDN Prefill throughput: 2.3–2.8×
→ End-to-end request throughput: 1.27–1.55×

Full baselines and validation details are available in the README.

No Gaudi2? You can still test it on real hardware.

We run our own Gaudi2 server cluster and offer rental access and free trials for developers bringing their own models, workloads, or optimization ideas.

⭐ GitHub:
https://t.co/TtH7CUNJKT

🖥️ Gaudi2 rental / free trial:
https://t.co/iphKuu29nn

Stars, Issues, benchmarks, optimizations, and unusual workloads are all welcome.

Let’s Gaudi it. 🐱

[BCat @barryxv6]

We reuse upstream components wherever appropriate. A high-performance port, however, requires more than reusable kernels — model-specific layouts, state management, graph/replay, and TP/PP execution paths also need optimization. Our goal is to make those pieces reusable for future Gaudi ports as well.