Magnum Opus Cyber

Think and act

[Magnum Opus Cyber @RuiHe13]

dpr is a very interesting way to look at what inference is really is, it is a verification problem now, dpr compare to cache now is a way to skip all yhe thinking and just give me the result, we will verify them. because these results are "known", the decode becomes somehow prefill in some sense

[Magnum Opus Cyber @RuiHe13]

Heard of SALT? The State Assured Latent Transition engine.
One engine, current support mainly Gemma 4 26B A4B Q4 MLX model,
deterministic inference,
cross platform bit identical compute on Mac, CUDA, AMD, intel,
same across CPU and GPU,
seeded temperature random play,
KV Cache export and reload across session and platforms,
optimized prefill and decode,
inference on CPU and GPU,
as low as 3gb ram or vram for the full Q4 precision,
supports cloud compute, actually developed AMD support on g4ad.xlarge and intel on c6i.2xlarge,
throughput on c6i about 6tks on low context, and about 10 on the g4ad, 15 on mac m4 pro,
internalized speculative decode with your own kv cache from previous exports.

Inference is compute and your KV is a ig part of your compute artifact, don't throw away the ladder you build after you reach the book you want, save it for use later.

tons more features and engineering design, see
Zenodo preprint: https://t.co/PqFTC2lou1
Repo https://t.co/coksRRXzmu

[Magnum Opus Cyber @RuiHe13]

very experienced question, prefill holds around 10~13 at actual context of arounf 64k and decode drops to about 5 to 7 without the speculative decode kick in, there is part of experiment show if you got a good coverage dpr that provides a good estimation, the decode actually can armotize to around 10 tks. so yeah long conext does hurt speed, but salt is built in a way allow it to grow with your usage.

[Magnum Opus Cyber @RuiHe13]

Boundless Thoughts, Governed Acts. The empowering governance Cyberware that gives agents accountability and free agents from repeatitive labors. Experiences now become workflow, and the value of intelligence now can be measured through parameterization.👇
@NousResearch

[Magnum Opus Cyber @RuiHe13]

#PS4share

[Magnum Opus Cyber @RuiHe13]

#PS4share

[Magnum Opus Cyber @RuiHe13]

#PS4share

[Magnum Opus Cyber @RuiHe13]

#PS4share

[Last Week Tonight @LastWeekTonight]

Here's last night's show, in its entirety.

https://t.co/FcggKLlstb

[Donald J. Trump @realDonaldTrump]

....is NOT their fault in any way, shape, or form. They are working closely with us to get rid of it. WE WILL PREVAIL TOGETHER!

[Donald J. Trump @realDonaldTrump]

It is very important that we totally protect our Asian American community in the United States, and all around the world. They are amazing people, and the spreading of the Virus....

[China Xinhua News @XHNews]

Smart and efficient! Watch how robot "armies" sort out packages in Chinese logistics firms

[Bloomberg @business]

The Trump administration plans to release an interim rule on Wednesday that bans federal agencies from buying certain products from China’s Huawei https://t.co/YmKOfv9NMm

[Star ⭐️ @Marielincoln10]

What a difference a year makes! My little Luna is 1️⃣ today 🎈🎂 #MondayMorning