Kvsolipse

等我们的末日来让地壳变厚
acc/acc

[Kvsolipse @Kivisolipse]

@01mp_fc 考虑到 Anthropic 面试常规环节之一是公司价值观对齐（面经：熟读 Dario 文章......），说出这种话也不足为奇，只能说相当 cultish 了

[Ke Li 🍁 @KL_Div]

Was just as excited as everyone else to read this cool new paper, and it felt like a trip down memory lane!

XM seems to be the same as IMLE, and the motivation and insights are also similar. Having worked on IMLE for years, I'll provide some context for my followers 👇.

1/n

[めりー犬 @merrysan1218]

宝宝

[COCO @TWOTIMESABITCH]

i love that our generation managed to kill the stereotype that if you have tattoos, you won’t be able to find a job.

now nobody can find a job, whether you have tattoos or not.

[Kvsolipse @Kivisolipse]

@hsn8086 巴黎绝对没这么夸张，可能郊区的确危险，但不去那几个比较臭名昭著的郊区的话，总体上是相当安全的

[Vinne @00vinne]

It's my Own Invention

[Teortaxes▶️ (DeepSeek 推特🐋铁粉 2023 – ∞) @teortaxesTex]

I’ve been saying for over a year. DeepSeek’s discovery of RL for reasoning in r1 is independent from o1 technology, the only commonality is what OpenAI had disclosed. And this is a case in point. Only OpenAI has truly mastered “reasoning effort”. It’s intrinsic to their method.

[rhy$ 🐸 @furitaos]

this is cool !!!!!!!

[Eric Xu (e/Mettā) @xleaps]

科普一下“多模态”的具体含义。

首先，我们虽然不认为盲人有“智力”上的缺陷，但不会因此让盲人靠听收音机去驾驶汽车。因为有些模态比如视觉，不是其他模态（文字或者语音）能够简单取代的。这里面有一个人择原理：即使最终在 AI 内大家都是 token, 但人类选择了用红灯而非多普勒雷达构建了整个交通系统，也选择了用 2D 屏幕而非字节流表达计算机操作，因此 AI 为了在现实世界和数字世界中完成任务，必须接受人类已经接受的模态和输入。

多模态是个被滥用了的词。模型多模态输入有利于内部更好的表示这个人择的世界，特别是视觉信号，可以帮助模型 grounding ，比如“红灯”它到底是一个什么样的信号， 在语义上毫无问题，但在开车场景里要接近人择的感知和推理，才能更加准确的表征人择世界的模型——这个世界上不存在客观的世界模型，都是人择的。

即是是纯粹追求虚幻的 AGI 目标，就以长程任务为标尺， DeepSeek 要做的长程任务，持续学习，最终也都绕不过多模态，特别是人择世界的视觉信号。一个很明显的例子就是盲人不能够仅靠语言，不依赖计算机屏幕，来操作针对视觉无碍者设计的计算机，很好完成长程计算机操作任务。 AI 也会在这种选择压力下，迫使 tokenize 人择的数字视觉信号。

DeepSeek 不愿意在 AGI 的路上做多模态的输出，和 Anthropic 的方向是一样的。因为多模态输出更多是一个工程问题，牵涉到如何构建扩散或者 flow match 模型，如何让输出的分布不塌陷到平均期望上，如何保障多步输出的一致性——在通往 AGI 路上这些都是垂直的任务，可以被通用的 AI 模型一句话调用，因此可以从通用智能里外化掉。

DeepSeek 放出一个只支持 text 输入的模型不代表 DeepSeek 就死死抱住 text-only 不放。实际上 DeepSeek 的非 API , 非开放权重版本是可以处理视觉信号的，而且许多我知道的企业应用里，都有人在试图给 DeepSeek 嫁接一个 ViT.

[Hari @hrkrshnn]

Incredible.

Math notoriously have long latency from discovery to applications.

Euler’s 1763 theorem became RSA encryption in 1977 and now helps secure the internet.

Riemann’s 1854 geometry became relativity in 1915, then GPS in 1978.

Shannon’s 1948 information theory became the foundation of modern communications and the internet.

There's an opportunity here for people to see what breakthrough solutions unlock the next frontier.

[Matt @matt_oai]

2022 / 2026.

Wild to see how far and fast we've come in less than 4 years

[Simo Ryu @cloneofsimo]

Rare photo of Yann Lecun on his way to find the datapoints this “less-than-a-cat-level intelligence” supposedly plagiarized the 10 solutions from

[Andrej Karpathy @karpathy]

We're starting to leave the territory where you'd test an LLM by e.g. "create an svg of pelican on a bicycle". As one idea to generalize it, I was interested what Opus 5 would do if I gave it the first paragraph of the Lord of the Rings, a 1M token budget (~$10) and asked for three js render of it. Opus went off for ~2 hours and wrote 5500 lines of code that (procedurally) rendered the story. It's kind of janky but fun. But it's a bit mindboggling that the LLM has to place and orchestrate various polygon assets in (x,y,z) coordinates and write code that animates it all, and that it even does anything at all.

I also like this kind of examples because no one in their right mind would ever spend the time to write something this custom but LLMs have all the stamina and patience in the world, so it's an example where we go from "no one would ever do this" to "sure, why not, it's ~free". There might be a lot more. But I'm excited about creating hyper custom worlds that you can imagine dropping players into, e.g. here to participate in the LoTR story as a spectator NPC, or one of the characters, or etc. Something like an ephemeral GTA of X on demand.

Last thought is that the domain of worlds/games exposes a weakness in LLMs: they can't easily audit their work because they aren't able to efficiently and natively perceive videos or play games within them. Here, Opus 5 had to very slowly and painstakingly take screenshots at different points, and it messed up a few times and created a bunch of jank. An example of raw capability (multimodal, gameplay) that I think is still quite lacking.

[Rita Repulsive @myc0latt]

If i ever get back into dating and discover that someone i thought I'd really clicked with was using an ai assistant to manage our relationship the whole time i would kill myself in a way that redefines how the world thinks of suicide

[Leander @leanderriefel]

The best European model btw

[Philip Arathoon @philarathoon]

The Maxwell conjecture is false. Counterexample found by AI (GPT-5.6 Sol), communicated by human mathematicians: https://t.co/nBpSJiOIRO

[jianlin.su @Jianlin_S]

Deconstructing Scaling Laws: The Triad of Optimization, Architecture, and Data
https://t.co/jbqiFXsIJN

[Kaguya’s Top Gal @hayasaka_aryan]

Gay niggas can walk naked in parades and fat feminists can go topless but a straight couple making out is how immoral

[Sebastian Raschka @rasbt]

The Kimi K3 architecture figure for yesterday's big open-weight model release, along with some observations and thoughts.

1. Yes, it looks relatively complicated, but it's essentially a scaled-up production version of their Kimi Linear model they released last year (scaled up from 48B -> 2.8T; K3 is by far the biggest open-weight model right now)

2. The one new component compared to Kimi Linear is the LatentMoE. I omitted it in the figure below since it's already very crowded, but that's essentially the same LatentMoE as in Nemotron 3 Ultra (you can find it in my LLM Architecture Gallery if you are curious). The idea here is to compress (down-project) large linear layers similar to multi-head latent attention.

3. Kimi K3's overall trend (similar to Nemotron 3, DeepSeek V4, and others) is also towards better inference efficiency. That is, there are many components that replace existing components with efficiency-tweaked versions. I.e., MoE -> LatentMoE, regular attention -> multi-head latent attention and Kimi Delta Attention. (I also have short tutorials and write-ups in my gallery if you are curious about additional details).

4. The one component change that is not an efficiency tweak is attention residuals. Like DeepSeek V4 improved the residual path with mHC (manifold-constrained Hyper-Connections), attention residuals are a way to improve the residual path, but it works a bit differently. I.e., mHC made the residual path wider. Attention residuals (also already part of Kimi Linear) connect the residuals across layers; the connection itself uses an attention score for an important/contribution weight. According to the report, it improves the validation loss and downstream performance (a bit) consistently and adds about 4% in training cost and 2% in inference cost.

5. Interestingly, Kimi K3 got rid of all RoPE layers and uses NoPE (No Positional Embeddings) everywhere instead. (Again, this is inherited from Kimi Linear). In other architectures, the recent trend was towards RoPE in local attention layers (like sliding window attention) and NoPE in the global layers. There were a few architectures that only used NoPE everywhere, but this is the first frontier-level one as far as I know.

6. Kimi K3 now also has native multimodal support, which is great!

There are several other interesting training tidbits in the technical report, but that's it from the architecture front so far. A really great release overall.

[Wolves Esports @WolvesEsports]

Thank you for everything noname 🫡🙇‍♂️

#ALGS