Beamsters

Beam

[Beamsters @Beamsters1]

@GameSpot This is where we call AI to do its real job.

[Beamsters @Beamsters1]

@GrimDawn Just do not let this be your last expansion.

[Beamsters @Beamsters1]

@antirez The official API from Openrouter (not a free one) mistranslate easy English->Thai word from "pig" to "dog".

[Beamsters @Beamsters1]

Inspired by ds4, I am working on a Rust inference project that will bring DeepSeek4 Flash to M1 Max machine with only 32GB of RAM.

[Beamsters @Beamsters1]

The quality of EXL3 is not MLX q4 or q5, it is approaching q6 - paying only 4.15 bit per weight.

Interesting enough?

[Beamsters @Beamsters1]

Latest Exl3 quants on Apple Silicon. Successfully convert Qwen27B model from BF16 to Exl3 on Mac, without touching cuda, quality is now on par with using cuda conversion.

[Beamsters @Beamsters1]

Now everyone can convert Huggingface's Weight to EXL3 format using Apple Silicon. I landed the beta version comfirmed with MiniCPM5-1B, quality on par or above the Exl3 using CUDA. Check https://t.co/Rn3YqtZuvw for v0.2.0!

[Beamsters @Beamsters1]

I ported EXL3 to run well on Apple Silicon. The best quant has arrived in MacOS! - M5 Max can do 2700t/s, 65+t/s on Qwen3.6-35B-A3B and 600 t/s, 38 t/s on Qwen3.6-27B + greedy dflash (20-25tok/s on temp tuned)

https://t.co/1bgN0gFyip

[Beamsters @Beamsters1]

ppl request that chat should be able to load/unload model what do you think? @jundotkim should I do it? or just a toggle to limit one model at a time?

[Beamsters @Beamsters1]

@jundotkim that or lit parse.

[Beamsters @Beamsters1]

My new chat module landed in oMLX 0.4, support multi-chat, samplings, timeline navigation, branching, model variants, profile, inline perf monitoring and svg rendering right inside web ui.

@jundotkim #omlx #mlx

[Arena.ai @arena]

New open model: MiniMax M3 by @MiniMax_AI is live in the Arena!

Find it across Text, Vision, Document and Code Arena: Frontend. Bring your toughest prompts and vote. Scores incoming soon!

[Beamsters @Beamsters1]

@mallexibra @ivanfioravanti m5 128gb using step3.7's llama.cpp build reach 500-1000 pp/s and 40-60 tg/s from 2k-32k context

[Beamsters @Beamsters1]

@jundotkim I'm done with the chat v2. all yours.

[Beamsters @Beamsters1]

@antirez sry man I even ran ds4-eval full tests for 6 hours to confirm and the results was all green.

[Beamsters @Beamsters1]

@antirez Landed finally!

[Beamsters @Beamsters1]

Here's a quick result. Ran 8 questions from ds4-eval and the model answered them correctly.

[Beamsters @Beamsters1]

@antirez I've just landed PR264, found out wide tiles can improve net prefill by +62% in short burst. Hope this can help!

[Beamsters @Beamsters1]

@antirez I'm cooking something amazing. 1000+ Prefill broken in M5 Max 128GB. Gotta refine the edges in a few days. Running ds4-eval to check 15 questions all passed.

[Beamsters @Beamsters1]

@antirez Thanks, I tried it and you know 8Gb for 500k context is a steal, compare to 96Gb for minimax. Now you have M5 Max, maybe you can play with M5 GPU Neural Accelerators… this can unlock extra horsepower.