Update, instead of 2027 and 64gb+ , the Deepseek model that sometimes bests GPT 5.6 Sol now runs on 24gb after just one day of work. Even smaller m5 max macbooks should be able to use it.
The local era is already here and I commit to get it working in my iPad in under a week.
Today I implemented Deepseek v4.1 flash q2 to check. For now it is testing at around 10 tok/sec on a m1 max, using only 24gb, which is about the same speed lm studio runs 27b on the same machine (lol). It will stabilize around 10-15 tok/sec here or ~30 tok/sec on a m5 max.
Today we have the IPO of the most important AI app in history.
Yes, I Post Out my own AI app, a multimodal local AI app for the average person which is the fastest in the world in each of its modalities and runs on Macs/iPads/iPhones offline.
After nuking MLX Rafa rebuilt it completely from scratch in swift+metal yelling directly at the bytes and atoms. Because of that we have previously impossible features, like running a 35B model decently for chat on an older iPhone16e with only 4~5 gigs of ram for the app.
If you hate me because of ideology or because I am a bigot just use something else offline but I beg you to be extremely careful with cloud based AI. They are coming not for your money but for the soul of your loved ones with superpersuasion and emotional hooks.
@Rafa_Schwinger@ClementDelangue Guess what Iβm doing with my 10m model, which I am ridiculously overtraining to make it potentially useful despite its small size.
Truffle edited a 5.5-minute nature film synced to C418.
It curated the clips, edited and graded them.
The scale is impressive and the frame is genuinely beautiful.