Sfwon17🛸🐐

Newbie

[dunik @dunik_7]

a Tsinghua University lab just put a project on GitHub that replaces a $400,000 H100 rack with a single 24GB graphics card.

it's called ktransformers, and the trick is almost stupidly simple: the experts you actually use stay on the gpu, the ones you don't sit on the cpu until they're called

/ deepseek-v3 and r1 with 139K context in 24gb of vram
/ up to 28x speedup over the standard setup
/ fine-tune deepseek-v3 across four rtx 4090s instead of a datacenter
/ built by tsinghua university's madsys lab, not a startup with a landing page

apache 2.0, and already past 17,000 stars.

-> https://t.co/DjF8CutVAa

bookmark it.

[+ @playoffdroppa]

Kai Cenat you're deadass a hero, a revolutionary, a leader amongst men, a role model, an embodiment of good.

Thank you bruh!

[Bingbin Liu @BingbinL]

We’re entering a data-scarce regime; but is less data always worse? Our #ICML work shows that small-set training (e.g., with repetition) can be actively leveraged as a useful optimization bias for structured reasoning tasks. Come chat with Jingwen and Ezra in Seoul to know more!

[두플래시 @tournsports1]

결혼 후 첫 작품에서 일본 여배우와 키스 해버린 변요한

[Lucas Beyer (bl16) @giffmana]

When frontier labs suddenly cut costs, or say "we found a way to dramatically cut inference memory!!"

This is what they found. The secret sauce. Vision always wins!

Of course, my very capable ex-colleagues who now work on Gemini and Claude already found out years ago:

[Tanishq Mathew Abraham, Ph.D. @iScienceLuvr]

This is one of the papers I'm quite excited about in the past few weeks. It's a very simple but practical modification to the DINOv3 training framework.

Let me explain how it works.

[Liang Zheng @LiangZheng_06]

based on our results under DiffusionBench, LDM converges much faster than pixel-space diffusion at 100k steps. curious why so many people are switching to pixel space, am i missing anything?

https://t.co/Rhmk7OUZv6

[Vatsal Baherwani @vatsalbaherwani]

Scaling laws predict an LLM's pretraining loss, but not its capabilities. Abilities like in-context learning emerge abruptly and only past a certain scale. Our new paper traces this to one bottleneck: learning which tokens attention should focus on. 🧵https://t.co/ja0wc8aK2e

[Alexander Goslin @xandurglar]

I owe it all to this Reddit post

[Peyman Milanfar @docmilanfar]

Latent-space models are a cage we’ve boxed ourselves into. The reason for using them in the first place was always efficiency, but we lost the plot and forgot that the speed costs us in terms of progress. It’s time to move on to pixel-space models for the next state of the art.

[Dimitris Papailiopoulos @DimitrisPapail]

Speaking of recursive self improvement, @nayoung_nylee recently defended her thesis which among other things showed how transformers can learn progressively harder tasks by generating solutions to problems that sit *right at* the boundary of their capabilities.

https://t.co/UPYEYl7Do7

This paper helped me 1) overcome my obsession with transformers and arithmetic and 2) appreciate the value of environments.

She and @jackcai1206 did this before GRPO btw

[Hiroaki Kubo @K4Hiroaki]

S-VGGT: Structure-Aware Subscene Decomposition
for Scalable 3D Foundation Models

Paper: https://t.co/AL9sbK2OqS
Code: https://t.co/gYQG1KW41C

[1/N]

[Hirokatsu Kataoka | 片岡裕雄 @HirokatuKataoka]

"Can vision transformers learn without natural images?"

has just reached 50 citations! In this paper, we showed that Vision Transformers can be visually pre-trained without using any natural images.

[Prometheus @prometxbt]

Monica Lam (Stanford Professor):

"AI writes shallow reports for one reason, you ask it ONE question. Our method asks dozens, like a journalist, and the same chatbot starts writing articles 25% better organized than top AI."

paper presented at a leading AI research conference, her Stanford lab (OVAL) unveiled STORM - a method already used by 70,000+ people to generate Wikipedia-grade, fully-cited articles on topics it has never seen.

the secret formula: 6–8 expert perspectives + cited expert interviews + ruthless outline + grounded section-by-section writing + blind-spot red team = a report you can actually trust.

watch the full breakdown to copy all 5 prompts into Claude.

save this post so the formula is ready when you need it.

[Amto @XAMTO_AI]

20分钟捅出一篇能投顶刊的论文，这事儿现在真不是吹牛批。

🔗 https://t.co/OdlaYkpSM4

以前做实证有多熬人你们心里都有数：选题、扒文献、洗数据、因果识别、稳健性检验、画表、写正文，一套流程干下来俩仨月就没了，还得防着referee挑刺、防着期刊反AI，博士生直接被卷到抑郁边缘。

现在斯坦福REAP联手copaper把这套东西给开源了，23000+个专业Agent技能，横扫经济、政治、社会、心理等8大社科领域，顶尖研究员的方法论全被打包成能直接调的Skill，从选题到投稿一气呵成全自动。

🔗 https://t.co/Dy1pebpzza

卷王时代，趁手工具不在身边，是真的会被淘汰出局。

[Angelica Parente @draparente]

Earlier this year I was getting frustrated with Claude's charts, fed this book to claude and had it generate a Tufte skill. Instantly got simpler/more beautiful visualizations.

https://t.co/lfXwyQfmQG

[쪼리(JJory) @TBJorry]

그녀의 남자친구는 배우 이도현

어려운 형편 속에서도 발달장애를 가진 동생을
평생 책임지겠다고 다짐하며 살아온 사람이다.

원래 농구선수가 꿈이었으나 아버지의 반대로
좌절한 후, 연기에 매력을 느껴 아르바이트를 하며
연기 학원을 다녔다고 한다.

​'슬기로운 감빵생활'로 데뷔한 이후, '호텔 델루나',
'더 글로리', '파묘' 등 수많은 히트작에 출연하며
연기력을 인정받고 집안의 빚을 모두 청산했다고.

배우 임지연과 공개열애 중인데,
너무나도 보기 좋은 선남선녀 🤎

[Suni @suni_code]

Someone on Github Uploaded Company wise Interview Questions 😭😭

I'm going to cancel my leetcode premium now 😭🙂

[Probability and Statistics @probnstat]

Here are 100 hard ML interview questions you must not miss out:

--- Theory & Fundamentals:
1. Explain bias–variance tradeoff mathematically.
2. When does ERM fail?
3. Derive ridge regression solution.
4. Why does L1 induce sparsity?
5. VC dimension intuition?
6. PAC learning basics?
7. When is MLE inconsistent?
8. Difference: MAP vs MLE?
9. Curse of dimensionality formal meaning?
10. When is KL divergence symmetric?

--- Optimization:
11. Why does SGD generalize better?
12. Saddle point vs local minima?
13. Adam vs SGD tradeoffs?
14. Convergence of GD conditions?
15. Why gradient vanishing occurs?
16. Exploding gradients fixes?
17. Hessian interpretation?
18. Natural gradient intuition?
19. Why sharp minima generalize worse?
20. Line search vs fixed LR?

--- Probabilistic ML:
21. Derive EM algorithm.
22. When does EM fail?
23. Gibbs vs Metropolis?
24. Variational inference idea?
25. ELBO derivation?
26. When is VI biased?
27. Exchangeability meaning?
28. Bayesian nonparametrics idea?
29. Prior sensitivity?
30. Posterior collapse in VAEs?

--- Deep Learning:
31. Why transformers beat RNNs?
32. Attention complexity issue?
33. Positional encoding role?
34. Layer norm vs batch norm?
35. Residual connections math?
36. Why overparameterization works?
37. Lottery ticket hypothesis?
38. Double descent curve?
39. Scaling laws intuition?
40. Why ReLU dominates?

--- CNNs & Vision:
41. Why convolution works?
42. Translation invariance proof?
43. Pooling necessity?
44. Dilated conv use?
45. Feature hierarchy explanation?

--- NLP:
46. Word2Vec objective?
47. Negative sampling role?
48. BERT vs GPT difference?
49. Tokenization impact?
50. Hallucination causes?

--- RL:
51. Bellman equation derivation?
52. Policy vs value iteration?
53. Why Q-learning unstable?
54. Exploration vs exploitation math?
55. Actor-critic intuition?
56. Credit assignment problem?
57. Off-policy vs on-policy?
58. Function approximation issues?
59. Reward shaping risks?
60. Why RL is sample inefficient?

--- Generalization & Theory:
61. Why deep nets generalize?
62. Role of implicit regularization?
63. Compression view of generalization?
64. Stability and generalization link?
65. Information bottleneck theory?

--- Graphs & Advanced Topics:
66. GNN oversmoothing issue?
67. Message passing limits?
68. Spectral vs spatial GNNs?
69. Why graph isomorphism hard?
70. Expressivity of GNNs?

--- Causality:
71. Correlation vs causation formal?
72. Backdoor criterion?
73. Instrumental variables?
74. Counterfactual reasoning?
75. Simpson’s paradox explanation?

--- Practical ML:
76. Handling class imbalance?
77. Data leakage examples?
78. Feature engineering impact?
79. When to use cross-validation?
80. Hyperparameter tuning strategies?

--- Systems & Scaling:
81. Distributed training challenges?
82. Data parallel vs model parallel?
83. Memory bottlenecks?
84. Mixed precision benefits?
85. Latency vs throughput tradeoff?

--- Robustness & Ethics:
86. Adversarial examples why?
87. Robust training methods?
88. Fairness in ML?
89. Bias detection?
90. Privacy in ML (DP)?

--- Cutting Edge:
91. Diffusion models vs GANs?
92. Why diffusion stable?
93. RLHF pipeline?
94. Alignment problem?
95. Multimodal learning challenges?
96. Foundation models risks?
97. In-context learning theory?
98. Test-time compute scaling?
99. Mechanistic interpretability?
100. Future of ML Theory?