Ton Cao

Views are my own. @cyankiwi_ai is my canvas.

[Ton Cao @_cpatonn]

@0xSero Top notch public transports and walkable cities

[Ton Cao @_cpatonn]

Local AI up

[vittorio @IterIntellectus]

xAI down
claude down
codex down

[Ton Cao @_cpatonn]

@superalesha He only shares from other people work. Might even be a bot in a bot farm that scraps, refines and reposts for revenue sharing.

[Ton Cao @_cpatonn]

xhigh reasoning effort for complicated, and low to medium for simple.

In that way, the model reuses its cache, and has cached input and cached output pricing, which usually are 10% of its full price.

[Ton Cao @_cpatonn]

@Youssofal_ 1 Fable 5.1 high session with pi subagents has just drained 25% my weekly Fable usage, and 100% of my 5-hour usage.

[Ton Cao @_cpatonn]

@superalesha Reduce core clock by 100-200 MHz, increase memory clock by 1000 MHz, 275-285 W. There are some speed reduction around less than 10%, but my living room was not a furnace anymore.

[Ton Cao @_cpatonn]

@superalesha Last year, I used ETH undervolting configs (lower core clock, raise memory clock) for inference, and it worked great. Only memory clock is important for inference, and not core clock.

[Ton Cao @_cpatonn]

@0xSero One of the best community.

[Ton Cao @_cpatonn]

Walking around the house, and feeling the heat from DGX Sparks. Happy to know they are solving my problems.

[Ton Cao @_cpatonn]

@aihacs Yeah, it could be a guy, it could be a robots. Everything requires maintenance.

But the thermal damage from the sun should be the same as any current satellites or spaceships.

[Ton Cao @_cpatonn]

Data centres in space sounds like a good idea.

No evironmental harm. Infinite solar energy. Launching a data centres should not be different from satellites.

The only difficulty is electronics behaving differently in space.

Imagine infinite AI floating in space.

[Ton Cao @_cpatonn]

@cfl0ws It should be something as starlink + vllm

[Ton Cao @_cpatonn]

Imagine using AI for both cyber security and cyber attacks. It will just be like Tron, but less cool.

[Abliteration.ai @abliteration_ai]

Today we're releasing abliterated-model-large-v2.

Based on GLM-5.3, which is #3 on Terminal-Bench 4.0 (behind only Opus 5 and Fable), with 2× the cyber exploitation of 5.2.

We abliterated and hosted it so it does the offensive cyber, red teaming, and agent testing work other models refuse to do.

- US-hosted
- FP8
- 1 million context window
- Zero input/output prompt retention

Live now. 🧵

[Ton Cao @_cpatonn]

@superalesha @elonmusk Yeah, cooling is one of the first problem I also thought of. Perhaps this becomes more viable when photon gpus become the reality.

Data centres in space should be no different than big satellites, which do in fact disrupt ground astronomers.

[Ton Cao @_cpatonn]

@0xSero Guess that’s your life path :)

[Alexey Fateev @superalesha]

@1_John_4_16 Qwen3.8 27B retains quality very well, even on INT4 and AWQ. I use it myself. I recommend AWQ by @cyankiwi_ai - they've never let me down.

https://t.co/iMm62LSBWY

[0xSero @0xSero]

Made by Qwen3.8-27B awq-4bit @ 128 tok/s

- Model: https://t.co/R1iNERgmJM
- Build: 4x RTX 3090 (capped at 175w each)
- Time to completion: 6 minutes

Prompt: "I want you to write an algorithm which creates a Dore + picasso style art piece, it must be procedurally generated"

[Ton Cao @_cpatonn]

@0xSero Anthropic $2 trillion valuation is shivering

[Ton Cao @_cpatonn]

Not my model directly, but I'm still happy to see Alexey produce great work using my model as the base!

cyankiwi/Qwen3.8-Flash-Next-AWQ-INT4 is available at https://t.co/miFfoCz1Qg

[Alexey Fateev @superalesha]

YES, FUCKING YES! I DID IT !!!

In my last post, I heavily trashed Qwen3.8 Flash Next, claiming it couldn't do anything at all. I managed to get awesome speed out of it, but in nearly 3 hours it couldn't even generate a simple Pagoda Garden. I had no clue what was going on.

Turns out the issue was with my custom quant, which I built myself to fit 256k context + vision into 4x3090 with good throughput.

@_cpatonn released a properly calibrated AWQ INT4 of this model, but it was larger than mine and didn't fit into my 96GB as-is. CUDA OOM - sounds familiar to everyone, right?

I decided to tackle it and make something that actually works. A full day of work, interrupted only by 5 hours of sleep.

To fit 262K context, I had to un-fuse the fused layers and quantize all non-sensitive projections to FP8. vLLM allows only 1 quantization scheme per fused layer, which turned into writing a custom loader and 13 launch attempts 😭

Attempt 12 finally loaded, and I was so hyped. But the model's output was pure gibberish, total mush. Turns out TP4 was slicing the fused QKV matrix in the wrong places, for fuck's sake. Attempt 13 fixed the slicing. 262K fit.

The model kept spitting garbage on any complex task. Short answers were fine, a 40K needle test was fine, but ask for a three.js scene - and you get 16,384 tokens of complete shit. I was devastated.

The root cause of all this was just mind-bogglingly fucked up.

48 tiny tensors, shared_expert_gate.weight, ~5 KB each. My build script quantized them to FP8, but the runtime read them as BF16 without applying the scale 🤡

The gate expects values around 0.077, but was receiving 448 🤡🤡 Eat up, my dear friend. And this gate determines how much the shared expert contributes to every single token.

I restored all 48 from the original checkpoint - and the pagoda scene, which I had spent hours grinding on and ran dozens of times - just passed twice in a row.

Wanna know how mindblown I was? Exactly.

Then the model swallowed 258,008 real prompt tokens with an image and nailed the needle test. Decode at 63-68 tok/s at 220 W per GPU - identical to what the broken build pushed.

Video is the proof. One prompt, two sets of weights: reference BF16 vs. my Franken-quant running on 4 gaming GPUs from 2020. One-shot, zero fixes. Mine finished first, and the scene came out tighter. It's just 1 prompt, so I won't call it a benchmark.

24 hours of my life for 240 KB of broken tensors 🤡🤡🤡

[Ton Cao @_cpatonn]

@gabu3d_pl Thanks a lot :) Can't really achieve without the community, including you, sharing my quants and @0xSero promoting my Twitter profile

[Ton Cao @_cpatonn]

I am happy with what I am doing.

I make open-source models and don't collect a dime. I don't smoke or drink, even socially.

In the past, I was materialistic, made a lot of money and partied a lot.

I was not happy as as I am. Doing good work is the step up.