@Prince_Canuma@GolGappay102@GoogleDeepMind@charmainemahach Looking forward for it.
While it’s nice to tinker around, tune many knobs, test infinite options, etc. it would be nice to have a “recommended starting point as of <month>.<year> for <task> with <standard configuration>”.
I know that this is a moving target.
@ivanfioravanti As it was with Gaming on Linux until rhe SteamDeck arrived: the capability is there, but it lacks bundling. A small, complete setup guide or installer for the 80% use case, that just works is still missing. Currently it is still research, trial & error with a moving target.
@ivanfioravanti That’s a good idea, but DS4 is focused at a very limited audience. Shelling out the equivalent of a very nice used car for a small-usercount-LLM rig is not acceptable for many.
@ivanfioravanti Though this is a bit disappointing the different performance/speed ratios are very valuable for the non-highend users. These are the numbers to weigh our decisions which compromise to make, when running our "pay once" LLM workloads. Thanks!
@ivanfioravanti@bnjmn_marie I've been subscribed to his blog for some time now. I guess I understand only half of his academic-ish articles, but I take that as a "know your gaps" kind of encouragement.
@ivanfioravanti@lafaiel That’s on of the benefits of the limited configuration dimensions in the Apple ecosystem. E.g. my personal MBP M4 Pro is free during work/sleep hours and I’d be happy to give something back to the community - even if this is not much.
@bnjmn_marie@N8Programs Do you have some hints how to become an "educated ML user" that understands these fundamentals? For every technology (e.g. databases, programming languages, frameworks, etc.) you need to to have that to use it reliably. For ML I am stuck with the outdated bits from university.
@ivanfioravanti@Apple Are you aware of any collection of benchmarks for the different quants and formats? I know that @bnjmn_marie did something like that for the GGUF quants of Qwen3.5, but never saw anything for MLX