Shrey Singhal

Machine Learner

[Shrey Singhal @imShrey1411]

@foundinacasino You looked like proper uncle ji lmfao.

[Max @MaxxingDelusion]

Need a girl who'd rather go hiking than get ready for a club.

[Minh Nhat Nguyen @menhguin]

are we gonna acknowledge that a lot of ML papers are genuinely poorly written, and stretch out a concept for 5-10 pages without adequately explaining the initial idea?

[i²cjak @i2cjak]

DO NOT FALL FOR THE BLUE COLLAR PSYOP

YOU NEED TO GET CLOSER TO THE LEVERS OF POWR

[Shrey Singhal @imShrey1411]

@Sahil100X @jitendarma73662 @Adarsh_VMore @ashutosh_sao @deepanshuliv @Deepjyoti100x @kirat_tw @100xDevs @100xSchool @rohanBuilds @AnkurJangid02 Damn that’s good. Would you open source it?

[kache @yacineMTB]

after really intensely doing nothing but control RL, training hundreds of models a day, i feel like i understand something about NNs that is hard to understand

that something is: deep learning is magic. it just works

[Claude @claudeai]

Starting today, 10,000 scientists across every field, from math to chemistry to physics and more, can get Claude through our new Claude Team plan for scientists. Standard seats are free, and premium seats with 5x usage limits are $15 per month, an 80% discount, for one year.

Claude is becoming increasingly capable of scientific work, with recent progress on problems from advanced physics calculations to protein design. Alongside that progress, we've been investing in the research community: Claude Science launched in June, and our AI for Science program funds high-impact projects with free credits. Today's expansion builds on both.

Principal investigators (or equivalent) at academic and nonprofit research institutions can sign up, then add the researchers in their group. Over the coming months, we plan to extend the program well beyond the initial 10,000 seats.

Learn more: https://t.co/RTG1JxWi4Q

[Lisan al Gaib @scaling01]

I really need you to internalize this:
- the current public frontier is in terms of historical progress 3-6 months behind the private frontier
- most benchmarks are still single-agent and only using a few million tokens, while the latest frontier models are trained for multi-agent operations

OpenAI and Anthropic are both 1.5-2 model iterations ahead, meaning something like GPT-6.1-Astra and Mythos 5.2

they are continuing to race internally

[no.mind @the_no_mind]

Shopify billionaire co-founder & CEO Tobi Lütke — one of the most analytically rigorous founders alive — uses affirmations.

His exact words:

"If you tell yourself or write down something about yourself 100 times, it writes into the neurofrontal cortex at such a deep level that your brain will start reconciling you to that. It just works."

His personal example:

He was terrified of public speaking.

For one week he sat down every day for 10 minutes and wrote: "I love public speaking."

A week later the fear was gone.

"It's not like a placebo. You just actively change your neurofrontal cortex in this moment."

Your brain starts shaping your behavior around the identity you rehearse most.

[neural nets. @cneuralnetwork]

if you are in AI, you should be able to draw and explain the transformer arch from memory

[Shrey Singhal @imShrey1411]

End of an era or start of a new era?

[Polymarket @Polymarket]

JUST IN: Nvidia reportedly agrees to buy Hugging Face for $12,900,000,000.00.

[Shrey Singhal @imShrey1411]

You can’t be behind on a path that nobody else is meant to walk on.

[Shrey Singhal @imShrey1411]

@wojakcodes This is very true. I’ve notified it many times in life

[‎Wojak Codes @wojakcodes]

once you get access to people who are doing something awesome, you’ll realise how ordinary most such people are.

Most of them don’t have any serious talent for what they do. They all got there by learning and making mistakes.

There’s absolutely nothing stopping you from achieving what they’ve achieved, but you choose to accept yourself as too stupid or too lazy or too weak.

[Polymarket @Polymarket]

JUST IN: Apple unveils the M5 Ultra, calling it its “most powerful chip ever,” built for 3D rendering & running frontier AI models.

[aether @bluewdrift]

Your death will come on an ordinary day, in the middle of unfinished plans and the world will continue without you.

So live a little.

[Cameron R. Wolfe, Ph.D. @cwolferesearch]

I just published my complete guide to reinforcement learning for LLMs. It's a single, standalone resource for understanding RL from first principles to frontier research.

Read it here: https://t.co/iUMAWUrkQ1

The topics covered are as follows…

RL Fundamentals:
- A general framework for RL (policy, states, actions, transition functions, environment, rewards, trajectories, etc.)
- Core concepts of RL (returns, discounting, action / state / trajectory probabilities, value / advantage functions).
- Formulating and estimating the learning objective for RL.

Basic of RL for LLMs:
- Token-level (MDP) vs. completion-level (bandit) formulations.
- Outcome versus process rewards.
- Value estimation via value models / critics.
- KL divergence, RL setups (RLHF, RLVR, infra, etc.), importance sampling.

Basic Policy Gradients:
- Deriving the RL objective / vanilla policy gradient (VPG) from first principles.
- Different policy-gradient formulations (trajectory returns, reward-to-go, baselines, value functions, and advantages).
- Why policy gradient estimates have high variance (and how baselines help).
- Implementation of VPG.

REINFORCE and its variants:
- REINFORCE as a Monte Carlo implementation of the VPG.
- Completion-level vs. token-level implementations of REINFORCE.
- Reward baselines and KL-regularized rewards.
- RLOO, REINFORCE++ and other critic-free policy gradient algorithms.

Actor-Critic Methods:
- Training value models / critics alongside the policy to estimate advantages.
- Trust Region Policy Optimization (TRPO) and KL-constrained policy updates.
- How PPO simplifies TRPO with a clipped surrogate objective.
- Full PPO details (clipping, advantage estimations, value loss, KL, policy ratios, multi-epoch optimization, etc.) + implementation.
- Generalized Advantage Estimation (GAE) + implementation.
- The memory and computational overhead of actor-critic algorithms.

GRPO:
- Group Relative Policy Optimization (GRPO) as a simpler alternative to PPO.
- Group-relative advantage estimates + implementation.
- Why removing the critic is helpful.
- Limitations of vanilla GRPO algorithm.
- Dr. GRPO, DAPO, GSPO, and other recent improvements.

Recent research topics:
- Online versus offline RL.
- Rubric-based RL.
- RL scaling laws.
- Agentic RL + world modeling.

This post is a synthesis of my writing / learning on RL for over a year. I hope it’s helpful!

[Shrey Singhal @imShrey1411]

@kickingkeys Wow this is so cool. One of the best personal projects I've ever come across.

[Shrey Singhal @imShrey1411]

@SevenSheeps Wow that’s a great achievement

[Deep @DeepStarts]

Bro rejected the rejection mail😭