Fabian

AI & Pizza Enthusiast, Software Engineer

[Fabian @onthexitter69]

@afoxdesign I might, reluctantly, pay $1000. But spending more than that for a 2160x2160 headset when I already own the Pico 4 Ultra, Quest 3, and Apple Vision Pro, is just insanity.

[Fabian @onthexitter69]

@ivanfioravanti @scouzi I hope you have better luck than I did but unfortunately I already tried that shit saying I'd sell the M3 Max when I bought the M3 Ultra.

[Fabian @onthexitter69]

@mweinbach Dual 16-core neural engine, you say?

[Fabian @onthexitter69]

@QwenDevs Is mlx-vlm / mlx-lm getting day-zero support?

[Fabian @onthexitter69]

And yet Python became the dominant interface for some of the most compute-intensive workloads in existence. Why? Because Python doesn’t have to do the compute.

It has cheap bindings to optimized native kernels, and mature conventions for passing large buffers between them with little or no copying.

Interpreter overhead barely matters when each Python call launches milliseconds of C++/CUDA work.

[Fabian @onthexitter69]

@jun_song 30/30/30 then AMD just sitting there with 10.

[Fabian @onthexitter69]

@badtakes420 @Kalshi_Finance The US has no open models that compete with China.

[Fabian @onthexitter69]

@JamesMeadlock @u1tra_instinct Would you be able to run the tuner and repeat? The defaults are definitely complete crap at this point.

[Fabian @onthexitter69]

Need someone with a 256-512GB M3 Ultra to test Deepseek-V4 Flash + ANE offload.

No promises about performance but looks like we should be able to push up-to +10%
https://t.co/Rq4MQqyiUI

[Fabian @onthexitter69]

@josethevrtech I'm prepared to believe it. I just find it surprising Meta had such incompetent engineers that they get presented with a performance issue and their solution was to use a different OS.

[Fabian @onthexitter69]

@discostew6082 That's brilliant - it looks like one of the recent projection offloads caused a performance regression. Would you be able to re-run your tests with the latest commit and profiling enabled, and then share your logs? That will help me to identify where the timing is going wrong.

[Fabian @onthexitter69]

@JamesMeadlock @u1tra_instinct Could you run again with profiling enabled and attach logs? I’m trying out all the projections that yielded positive isolated results.

Thanks for all the help

[Fabian @onthexitter69]

@u1tra_instinct I've pushed some updates and a fix to the issue you had (thanks @JamesMeadlock). Could you give it all a fresh retry? Should offload more projections now for a few % bump in theory.

[Fabian @onthexitter69]

@GeryMKS Under M5 Max, we have not yet been able to achieve a performance boost. M5 Pro we have been able to. M5 Ultra (if it happens) may well be able to.

[Fabian @onthexitter69]

Very marginal improvement found by offloading attention to the ANE. Offloading GDN output had negative results but included in the PR as experimental anyway, I think in some lower-end chips it may yield a positive result.
https://t.co/gVId4K78yB

[Fabian @onthexitter69]

@u1tra_instinct Have you been able to test the CPU offloading?

[Fabian @onthexitter69]

(Not claiming credit for this, I’ve just added a tuner and guarding onto Jundot’s efforts to port the changes to Deepseek. I’m still experimenting with further offloads)

[Fabian @onthexitter69]

@jamesckane @threejs @JureTriglav @CodyBennett How’d you get eye track input

[Fabian @onthexitter69]

@techactually @anthropeusthai That dotted line? That’s lossless. That janky ass line doing Florida to New York via California? That’s EarPods.

[Fabian @onthexitter69]

@Youssofal_ People still use Claude's public models for AI research?