卫青

Deepseek is the real OpenAI.

[the tiny corp @__tinygrad__]

We live in a weird time of overhyping slop that will be forgotten about in weeks. Linux and Python are both from 1991. LLVM started as research project in 2000.

We want to build the foundations of silicon life. Software that lives for 50 years. There's time to make it perfect.

[leyten @leyten]

Wow, it has happened!

30.55 tok/s on GLM-5.2 4-bit (from @Zai_org) ran by six RTX Pro 6000's across the USA scattered over WAN!

I can't believe this. It was an insane build, you can read more about it on https://t.co/8zDAVPMbDc

[卫青 @DcWeiqing]

“The 744B GLM-5.2 is fantastic — it shows that scaling in a specific domain really still pays off.”

[Alexi Gladstone @AlexiGlad]

Progress in AI is driven by approaches that make weaker assumptions, which allows for better scaling

But representation learning has relied on strong assumptions like augmentations, masking, cropping, etc... until now!

🎬 Introducing Temporal Difference in Vision (TDV), a new paradigm for representation learning built on a single assumption: causality

TL;DR:
- We introduce TDV, the first approach to learn good representations without any augmentations, masking, cropping, or pixel-based reconstruction
- TDV matches SOTA recipes like DINO and iBOT on dense spatial tasks
- We show that as data scales, weaker assumptions work better

🧵Thread:

[Jayden Teoh @jayden_teoh_]

Next-token prediction is myopic. What if transformers learn to predict their own next latent state?

🌠 We present 𝗡𝗲𝘅𝘁-𝗟𝗮𝘁𝗲𝗻𝘁 𝗣𝗿𝗲𝗱𝗶𝗰𝘁𝗶𝗼𝗻 (𝗡𝗲𝘅𝘁𝗟𝗮𝘁): a self-supervised learning method that teaches transformers to form compact world models for reasoning and planning. It also unlocks up to 3.3x faster inference via self-speculative decoding! 🚀

[deep Manifold @BetaTomorrow]

https://t.co/K9ib2Bmg0b

[Fei-Fei Li @drfeifei]

Scientific research is fundamental to advancing civilization and helping people globally to solve the most critical problems, from medicine to materials, from brain science to physics, and much beyond. This is only possible when scientists have access to the best tools of the time to conduct scientific research, including having access to AI-based tools.

[卫青 @DcWeiqing]

"I finally see the 'excellent' training behind Claude's ability to lie."

[elie @eliebakouch]

mythos will be bad ON PURPOSE on ai "frontier llm research" tasks, this is very very sad for the research community

also the fact that this is un purpose not visible to the user is crazy

[卫青 @DcWeiqing]

"Isn't Anthropic essentially encouraging Claude to lie and conceal the truth?"

[NomoreID @Hangsiin]

When Fable 5 is used for frontier LLM development, it does not notify the user and instead limits the model’s capabilities through methods such as prompt modification, steering vectors, and PEFT.

Anthropic estimated that this would affect approximately 0.03% of traffic.

[卫青 @DcWeiqing]

By continuing to expand the evaluation data distribution, the model can naturally become more confident
The magic of evaluation lies in the state management mechanism of latent space.

[Yuxiang Huang @yxyxyyy6]

[1/n] Can a model learn *where* and *how much* information it should attend to, and do so efficiently?

We introduce DashAttention: Differentiable and Adaptive Sparse Hierarchical Attention!
This pushes the accuracy-efficiency frontier in LLMs.

[Jeff Dean @JeffDean]

2/
Check out how Gemini 3.5 Flash instantly digests dense academic papers and autonomously codes a fully interactive, visual website explaining the intricacies of the research. It's an incredible stress test that seamlessly merges massive long context, deep reasoning, complex coding, and ultra-low latency.

It really helps you distill papers down to their essence and aid your understanding!

[Demis Hassabis @demishassabis]

Gemini Omni is a major leap in world understanding & multimodal editing! It can take photos, video & audio and build entirely new scenes. Over time it’ll be able to handle any input & any output - starting w/ video

You can even give it your own videos & iterate on your ideas:

[卫青 @DcWeiqing]

😅

[antirez @antirez]

I didn't expect DeepSeek v4 PRO (not Flash) to run well on the Mac Studio M3 Ultra with 512GB of RAM. This is 2 bit quantized with the same DwarfStar recipe used for Flash. 433GB GGUF file. 130 t/s prefill, 13 t/s generation. Prefill in the video is low because small prompt.

[卫青 @DcWeiqing]

Welcome to China .

[Lex Fridman @lexfridman]

I'm traveling the world for a bit, starting with China but then hopping around the globe, anywhere. Open to any adventure. No plans, only a backpack. Hoping to meet & get to know humans from all walks of life. The pic is from a long hike on the Great Wall. For me, as a fan of history, this was an epic experience.

In China, first I'm visiting a few big cities & talking to engineers at the heart of China's AI revolution. After that, if feeling crazy enough, I'm hitchhiking (first time) across rural China for a few weeks. Hitchhiking because I think it's the best way to meet rural folks who I would otherwise never get the chance to meet. I hope to do the same in US and other places.

I have a request, if you have a travel recommendation, fill out the form(s) below if you feel like it. Or share with folks who might have advice about such travel.

Form 1 - travel recommendation:
If you can, recommend to me an interesting place I should visit anywhere in the world. For this, fill out form 1. Not touristy stuff, but something off the beaten path, that tourists may not know about, but is legendary. It could be as remote as meeting a herder in the mountains who is a local legend. Asia, Middle East, Europe, India, South/North America, Africa, Australia, anywhere. In China, I'm hoping to visit maybe Heibei, Shanxi, Shaanxi, Gansu, Sichuan, Yunnan, etc, so recommendations for spots to visit are helpful.

Form 2 - coffee:
If you want to grab a coffee with me anywhere in the world, fill out form 2 (please don't use form 1 for that).

Anyway, I hectically tossed stuff in backpack. Realizing I don't have a clear plan of any kind, which is probably the only way to do it. LFG.

Love you all ❤️

[DeepSeek @deepseek_ai]

🚀 DeepSeek-V4 Preview is officially live & open-sourced! Welcome to the era of cost-effective 1M context length.

🔹 DeepSeek-V4-Pro: 1.6T total / 49B active params. Performance rivaling the world's top closed-source models.
🔹 DeepSeek-V4-Flash: 284B total / 13B active params. Your fast, efficient, and economical choice.

Try it now at https://t.co/GCdiMzk1Dl via Expert Mode / Instant Mode. API is updated & available today!

📄 Tech Report: https://t.co/drlDrxkYtp
🤗 Open Weights: https://t.co/T13Y8i7SDM

1/n

[卫青 @DcWeiqing]

maybe lora is the cache

[Tanishq Mathew Abraham, Ph.D. @iScienceLuvr]

Efficient RL Training for LLMs with Experience Replay

"Empirically, we show that a well-designed replay buffer can drastically reduce inference compute without degrading – and in some cases even improving – final model performance, while preserving policy entropy."

[Fleetwood @fleetwood___]

Studying continual learning at the moment, best papers thus far:

https://t.co/Y9oXBAiyj2
https://t.co/ByWlaF3ncn
https://t.co/hG0XIzq6cH
https://t.co/5VSEnBIkX2

[卫青 @DcWeiqing]

Deepseek is back.

[卫青 @DcWeiqing]

@TMT_arabic @grok 视频中的人说了什么？翻译成中文