Bernard Huang

[沐安ねこ @show5560]

虽然知道这是为了給具身智能采集数据赚点外快补贴家用

但夜市看见这个还是太他妈赛博朋克了

[Bernard Huang @BernardH789]

@xukego 孙割的眼光确实不错

[Alan.S @patrickstone222]

高敏感的人如果克服了自我讨伐而全然地转向捕捉外界信息，那一定是无比强大的。

[Bernard Huang @BernardH789]

@realchendahuang 产品经理+架构师

[Bernard Huang @BernardH789]

@xiaojingcanxue @grok 这类内幕交易是怎么被发现的？

[Bernard Huang @BernardH789]

@ErEr43150536 这条路我每天都走

[Bernard Huang @BernardH789]

@Huahuazo 哈哈哈哈哈

[Bernard Huang @BernardH789]

@jlxc2001 卖太便宜了，提价到 1399，品牌格调下点功夫

[Bernard Huang @BernardH789]

@dimentary 跟我想的一样，大脑和小脑或许需要一种特殊的 token 来通讯

[Bernard Huang @BernardH789]

LLM 和 Jev 就像大脑和小脑，我们终于有小脑可以用了

[Bernard Huang @BernardH789]

@Qtumanalysis @beffjezos @extropic 跟我的思路不谋而合，没想到有人做出来了，这绝对是一大步！神经网络的计算应该天然是模糊（Analog ）的

[Bernard Huang @BernardH789]

@Summerkanna 达尔文还真没抄错

[Bernard Huang @BernardH789]

@zengying1107 @xukego 靠，梦幻联动了是吧

[Bernard Huang @BernardH789]

@zhhcsasp 这点困难都不想克服，怎么获得成功？

[Bernard Huang @BernardH789]

@Itsfoss 封闭会渐渐导致 VMware 的衰败，利好 Proxmox、QEMU

[Bernard Huang @BernardH789]

@realchendahuang 疯了心了，要投资额 10%怎么不去抢，租个办公室才多少钱

[cts🌸 @gf_256]

yeah this is agi

transcription + full harmony analysis from a video

[Volatile Markets @volatilemarkts]

Twelve machines. 2.4 terabytes of unified memory. One coordinated inference problem.

Seven DGX Sparks: 896 GB, CUDA, brutal at compute.
Five Mac Studios: 1.5 TB, Metal, brutal at bandwidth and unbeatable per watt.
Plus three Mac minis, a 5080, some V100s.

At 4-bit, that Studio memory holds a 1 to 2 trillion parameter model with room left over for a 900,000-token context. The weights fit. That was never the problem.

The problem is prefill. Before a model with that context says one word, it has to read everything you gave it. On Apple Silicon that is roughly 400 tokens a second. A 900K-token load is over half an hour of silence. Decode is fine, 25 to 30 tokens a second all day, quiet, 300 watts. It’s the first word that costs you.

The Sparks prefill four to five times faster and can’t hold the model. The Studios hold the model and can’t prefill. Everyone with mixed silicon owns both halves of the answer and no way to join them.

So we joined them. NVIDIA prefills, Apple decodes, one request. Two engines that share no cache format, no framework, no vendor. Instead of transferring a cache neither can read, the prefill box computes the decoder’s finished cache using the decoder’s own weights and writes it into the decoder’s prefix store. About 10 KB per token crosses the wire, over plain 10 gigabit Ethernet through the two switches in the first picture. No RDMA, no Thunderbolt.

Measured today on DeepSeek-V4-Flash, 284B, 241,000-token cold load:
Mac Studio alone, 12 minutes to the first word.
Two Sparks feeding it, 3 minutes.
Same prompt again, 19 seconds.
Decode identical. Answers identical.

That ratio is what makes the goal real: prefill 900K on the Sparks for a trillion-parameter model living on five Studios. Tonight we took the prefill window from 262K to 524K. Not finished, and every number gets posted either way.

Why this is a paradigm shift for us: our agent is persistent and has 54MB of memory files..and then we drop a transcript or a codebase on top of it mid-conversation. The wait was the product’s real cost. It isn’t anymore.

All credit to everyone who contributed to these concepts before us. We distill knowledge from all the greats and give credit to all. Standing on the shoulders of GitHub wizards unapologetically without fear of failure or judgement. Local ai must win!

https://t.co/X94wAZsND8

#localai #heterogeneousinference #dgxspark #applesilicon

[Bernard Huang @BernardH789]

@dhh @AIatMeta OMG

[Bernard Huang @BernardH789]

GPT的训练方向上好像有偏差了。对于潜在错误、可能发生的问题过于注意，过于小心翼翼。只要深度使用GPT都应该能发现，比如询问一个功能如何实现，答案最后总会带一个“避免...”，内容是某个极小概率可能会发生的异常。Codex最近经常过多写一些超过必要的防御性代码跟这个也有关系。
#Codex #ChatGPT @thsottiaux