Ritesh Goru

research @DevRev, BTech @iitbombay

[Ritesh Goru @RiteshGoru]

Give me an LLM powerful enough and a harness on which to place it, and I shall move the world

[Dimitris Papailiopoulos @DimitrisPapail]

say i have two distributions, p (humans) and q (llms).
I can sample x from either p or q
I can compute what q(x) is, but i can't compute what p(x) is
I want an unbiased estimator of KL between p and q, i.e., - sum_{x any possible text} p(x) log q(x)/p(x)

is "sample x from p, compute log q(x)" an unbiased estimate of it?

the answer is no! but if you optimize for it, which is what we do for LLMs, you get that
min_{q which is your llm} -E_{x~p} log q(x) + Entropy(p) = KL(p||q), but H(p) doesn't matter in your optimization so optimizing for KL is the same as optimizing -E_{x~p} log q(x)

so minimizing -log Prob is equivalent to min KL divergence to the ground truth distributin from where you collect samples. Cool!

[Ritesh Goru @RiteshGoru]

@sama
Any plans on releasing open-weight embedding models from OpenAI?

[Air Katakana @airkatakana]

worst graph crime ive ever seen, what the fuck is this

[Ritesh Goru @RiteshGoru]

So much variance in test time compute!

[Artificial Analysis @ArtificialAnlys]

OpenAI gave us early access to GPT-5: our independent benchmarks verify a new high for AI intelligence. We have tested all four GPT-5 reasoning effort levels, revealing 23x differences in token usage and cost between the ‘high’ and ‘minimal’ options and substantial differences in intelligence

We have run our full suite of eight evaluations independently across all reasoning effort configurations of GPT-5 and are reporting benchmark results for intelligence, token usage, and end-to-end latency.

What @OpenAI released: OpenAI has released a single endpoint for GPT-5, but different reasoning efforts offer vastly different intelligence. GPT-5 with reasoning effort “High” reaches a new intelligence frontier, while “Minimal” is near GPT-4.1 level (but more token efficient).

Takeaways from our independent benchmarks:
⚙️ Reasoning effort configuration: GPT-5 offers four reasoning effort configurations: high, medium, low, and minimal. Reasoning effort options steer the model to “think” more or less hard for each query, driving large differences in intelligence, token usage, speed, and cost.

🧠 Intelligence achieved ranges from frontier to GPT-4.1 level: GPT-5 sets a new standard with a score of 68 on our Artificial Analysis Intelligence Index (MMLU-Pro, GPQA Diamond, Humanity’s Last Exam, LiveCodeBench, SciCode, AIME, IFBench & AA-LCR) at High reasoning effort. Medium (67) is close to o3, Low (64) sits between DeepSeek R1 and o3, and Minimal (44) is close to GPT-4.1. While High sets a new standard, the increase over o3 is not comparable to the jump from GPT-3 to GPT-4 or GPT-4o to o1.

💬 Token usage varies 23x between reasoning efforts: GPT-5 with High reasoning effort used more tokens than o3 (82M vs. 50M) to complete our Index, but still fewer than Gemini 2.5 Pro (98M) and DeepSeek R1 0528 (99M). However, Minimal reasoning effort used only 3.5M tokens which is substantially less than GPT-4.1, making GPT-5 Minimal significantly more token-efficient for similar intelligence.

📖 Long Context Reasoning: We released our own Long Context Reasoning (AA-LCR) benchmark earlier this week to test the reasoning capabilities of models across long sequence lengths (sets of documents ~100k tokens in total). GPT-5 stands out for its performance in AA-LCR, with GPT-5 in both High and Medium reasoning efforts topping the benchmark.

🤖 Agentic Capabilities: OpenAI also commented on improvements across capabilities increasingly important to how AI models are used, including agents (long horizon tool calling). We recently added IFBench to our Intelligence Index to cover instruction following and will be adding further evals to cover agentic tool calling to independently test these capabilities.

📡 Vibe checks: We’re testing the personality of the model through MicroEvals on our website which supports running the same prompt across models and comparing results. It’s free to use, we’ll provide an update with our perspective shortly but feel free to share your own!

See below for further analysis:

[Ritesh Goru @RiteshGoru]

@BetaTomorrow @Yampeleg @BetaTomorrow may I know where is this picture from?

[Ritesh Goru @RiteshGoru]

Definitely one of the most useful features for scientific reading!

Claude chat on top of this is 🤌

[alphaXiv @askalphaxiv]

We used Mistral OCR with Claude 3.7 to create blog-style overviews for arXiv papers

Generate beautiful research blogs with figures, key insights, and clear explanations from the paper with just one click

Understand papers in minutes - not hours

[Ritesh Goru @RiteshGoru]

If re-aligning misaligned models is possible, we should call the process model rehab ♻️

[Owain Evans @OwainEvans_UK]

We also tested if emergent misalignment can be induced selectively via a backdoor.
We find that models finetuned to write insecure code given a trigger become misaligned only when that trigger is present.
So the misalignment is hidden unless you know the backdoor.

[Ritesh Goru @RiteshGoru]

Curious to see how thought tokens appear in a 3B model trained for chess!

[Jiayi Pan @jiayi_pirate]

We reproduced DeepSeek R1-Zero in the CountDown game, and it just works

Through RL, the 3B base LM develops self-verification and search abilities all on its own

You can experience the Ahah moment yourself for < $30
Code: https://t.co/UcGKN2SVGj

Here's what we learned 🧵

[devrev @devrev]

@DevRev secures a $100M #SeriesA round at a $1.1B valuation! Our AI-native AgentOS platform is transforming enterprises with advanced #GenAI capabilities, like 1-click data migration and lightweight agents. Next up: global expansion. Read more in @Business: https://t.co/jkeaIhcyfy

@dheeraj @DevReveler @vkhosla @MayfieldFund

[Ritesh Goru @RiteshGoru]

Definitely don't wanna miss this!

[ASSOCIATION OF BANGALORE AMATEUR ASTRONOMERS(ABAA) @abaaonline]

This year’s Geminids meteor shower will peak on December 14 at 19:00 UT. The meteor shower’s radiant is located near Castor, and meteors will appear to originate from this area. The constellation will be above the horizon after 8 PM IST. 1/3

[Mina Protocol (httpz) 🪶 @MinaProtocol]

🪐✨ InterPlanetary Compute System

Decentralized alternative for ec2 instances and lambdas.

Github &gt; https://t.co/sA5iJvC1jL

@vaithak, @RiteshGoru
https://t.co/fF1QSuI5xj

[comMINAty (httpz) zkok.io @minacryptocom]

Check out 3 brand new WIP #zkApps built at #ETHIndia
InterPlanetary Compute System by @vaithak, @RiteshGoru MeData by @0xmmmehulll, @1015zach, @kushagra___ @gaurav05_gaurav, @KunalKhatri137 &amp; Kleio by @vaibhavgeek, @recurshawn https://t.co/CfkBqz4RGU 🚀🪶

[Cliff Pickover @pickover]

This is a mathematics joke. Source: https://t.co/t4s6HsdXv7

[Ritesh Goru @RiteshGoru]

We won a Filecoin 🏊prize at ETHIndia!

Thanks♥ @Filecoin x @ethglobal

It was awesome! Can't wait for the next one.
👉 https://t.co/SLP0BGTLSA

#BuiltOnFilecoin

[Ritesh Goru @RiteshGoru]

@manoj_333 @Startup_Kar Congrats to the team 🎉

[Pravin Raut @itspravinraut]

I have learned more about education, finance, family, and life in this 1 year of a pandemic compared to the last 4 years of my college degree.

[Ritesh Goru @RiteshGoru]

@chesscom @anishgiri wants to know your location

[Cliff Pickover @pickover]

Fractal bird. A giant flock of birds decides to assemble into one composite bird, as humankind gazes in awe and stupefaction. https://t.co/A3SuMxB9gJ

[Manoj Gopalkrishnan @manoj_333]

Our validation study is on medrxiv at https://t.co/MLc8drk0mt

We show that Tapestry Pooling can give results for 961 samples with only 93 tests in a single round of PCR.