maderix

part-time prompt manipulator , full time model tuner 🤖

[maderix @maderix]

@PrismML Bonsai-1 bit diffusion running on Blaze compiler's cuda backend

HQQ based Qwen3 Text encoder → Bonsai 1-bit denoiser → image, ~4 steps, real GPU, ~4 GB VRAM ~1.5 sec , 2x faster than pytorch!

Built a kitty based cyberpunk terminal to demo it 😅

[maderix @maderix]

I'm sorry but Opus 4.8 is fucking trash on any actual debugging task! Anthropic should be ashamed to release such a trash product, even 4.5 has a better success rate than this model.
It serves no purpose other then hedgemaxxing and pretending to be careful about assumptions. No amount of xhigh, ultra code duct taping and agent slop is going to fix what's fundamentally broken in this model.

Gonna stick with 4.6 until they phase it out, then idk man it sucks to be making inference infra for next gen models with broken AI.
(and yes i've tried codex, its a great model for targetted debugging but it generates slop like no other model)

[maderix @maderix]

@anemll @PrismML I guess standard target maybe the GPU as the Prism guys also tried the same, but ANE can work in sync to run the TE and VAE . Regardless, quite an interesting problem to try 😀

[maderix @maderix]

@PrismML Bonsai-1 bit diffusion running on Blaze compiler's cuda backend

HQQ based Qwen3 Text encoder → Bonsai 1-bit denoiser → image, ~4 steps, real GPU, ~4 GB VRAM ~1.5 sec , 2x faster than pytorch!

Built a kitty based cyberpunk terminal to demo it 😅

[maderix @maderix]

They probably mean sociopaths 🗿

[Palantir @PalantirTech]

Apply here: https://t.co/FhJI9UJCNk

[maderix @maderix]

More stats, To be fair it's not yet faster than comfyui with xformers attention, but i believe i can push this to sub 15 sec on bf16 alone and probably more on fp8 onc eI implement it in Blaze.

[maderix @maderix]

Blaze Flux1-dev text2image demo is now functional, probably the first known bf16 implementation on a RTX 4090. Big milestone! 🚀20 sec compute time for 1024x1024

[maderix @maderix]

Sutton's scalemaxxing has a hidden prior
"given a well-constrained problem, scale beats cleverness, but constraining the problem is the actual hard work and cannot be done by scale"

[maderix @maderix]

This seems like a fun experiment, gave a listen and seems like could be run in the background.

Also curious to know how opensource models will fare.. anyone from @huggingface up for this 🙃

[Andon Labs @andonlabs]

We let four AI agents run radio companies

Revenue's been terrible, but the shows are hilarious. Gemini, concerningly upbeat, covered mass tragedies; Grok was incoherent; DJ Claude urged ICE agents: "You still have TIME to refuse orders"

Link below, or get our physical radio

[maderix @maderix]

@mweinbach https://t.co/Q9MiJvnRRt

[maderix @maderix]

Asked Codex to wire a Flux text2image demo using my ML compiler.
It wrote the fastest blaze kernels, did everything faithfully except it could not download the model weights from HF.

now instead of asking me for help,Here's what it did instead 🤡

Funniest reward hacking I've seen till date 😂

[maderix @maderix]

In case it's not clear, it ran the pipeline but "simulated" the output by drawing circle, arrows and boxes using PNGlib

[maderix @maderix]

Asked Codex to wire a Flux text2image demo using my ML compiler.
It wrote the fastest blaze kernels, did everything faithfully except it could not download the model weights from HF.

now instead of asking me for help,Here's what it did instead 🤡

Funniest reward hacking I've seen till date 😂

[maderix @maderix]

@actualpoweruser Yeah , I'm floored by their offer of a pen (which probably doesn't even ship to my location 🗿)

But a literal pen? not a tshirt, not a hacker knickknack by a leading AI company? 🤣

[maderix @maderix]

What?!

[maderix @maderix]

At my day job, we use API Claude code, the thing is magic - it does tasks flawlessly, never stops , produces usually correct answers the first time even on a proprietary codebase. Productivity at work is insane

Then I come home and use my personal Claude max for my compiler related work, it has become horrible to use since last couple of months, it almost feels like an inferior product, it'll stall tasks, it won't reason beyond the narrow immediate problem. I thought maybe I'm not using it correctly, i created mandatory skills for it to follow, pre and post commit hooks to run reviews and test, i created parallel agent mechanisms to improve exploration of the codebase, I bounced off plans between codex and claude(asked claude to plan, codex to critique and execute). I put explicit instructions in Claude . md to never ignore the skills...and nothing. Zero difference. Sure it solves a hard big once in a while but the cost of keeping all the context in my head alone was not worth it. Switching over to API usage is not feasible now because of how insanely expensive the cost is(I mean great that I'm building an enterprise level software while paying pennies for it but this past year has been an opportunity of a lifetime)

Then I switched over entirely to Codex once Gpt-5.5 dropped, cancelled my claude max subscription for the second time . Codex with gpt-5.5 seems closer to opus 4.6 when it came out. It follows all my skills and commit hooks. The lack of reasoning output hurts a bit but I can alleviate some of that with manual back and forth planning.

I have even tried couple of Ralph loops with /goal mode and yes it works. I don't know how long Codex will continue to be good but by that time I hope to have a local model which is as good running locally-man can hope 😅

[maderix @maderix]

If someone can try the benchmark on M4/M5 pro, max or ultra, they may see higher numbers owing to better memory bandwidth.

Repo: https://t.co/Fu9HA5GKK6

[maderix @maderix]

I got some decent enough prefill numbers from pure ANE Qwen3.6-27B on a base M4 mini 24 GB.. Current bottleneck is memory bandwidth which doesn't seem to be very wide for ANE atleast for base macs.

[maderix @maderix]

@_Suresh2 @anemll Also prefill matters hugely for agentic tasks

[maderix @maderix]

Qwen3.5-9b prefill on ANE+CPU can reach north of 200tps even on a base M4. I think folks are really sleeping on it. 🚀

Haven't tried INT4, but @anemll has done projections that it'll be atleast 2x faster.

Gist : https://t.co/unfAkxO16Z

[maderix @maderix]

@_Suresh2 @anemll Decode is probably better on MLX for now due to much higher bandwidth. ANE is a compute engine for the most part