Anil Madamala

I love programming and I'm an amateur photographer.

[ServiceNow AI Research @ServiceNowRSRCH]

🚀 Grok Voice Think Fast 1.0 (@xAI) lands on the Pareto frontier on EVA-Bench — no system in the eval beats it on accuracy without sacrificing experience, or vice versa.

📊 Leaderboard: https://t.co/v4QJZwzUsd

@elonmusk #VoiceAgents #ServiceNowResearch #EVABench #GrokVoice #xAI

[a16z @a16z]

World Labs CEO Dr. Fei-Fei Li explains says "world model" has become an overloaded term & explains what each kind of world model does:

"Right now there are three ways of calling world models when it comes to spatial intelligence."

"One is what I call a renderer, when the model puts beautiful pixels on the screen."

"Another kind of world model is what we call a planner. That is more for machines, more for robots."

"The third kind, which I think is the linchpin of the three, is a simulator."

"A simulator could become a renderer. The simulator could become a planner. But this layer is a huge critical path to unlock spatial intelligence. And that's what World Labs is working on."

@drfeifei at Bloomberg Tech live with @emilychangtv

[ServiceNow AI Research @ServiceNowRSRCH]

⭐ 𝗘𝗩𝗔-𝗕𝗲𝗻𝗰𝗵 𝗗𝗮𝘁𝗮 𝟮.𝟬: 𝟯 𝗗𝗼𝗺𝗮𝗶𝗻𝘀, 𝟭𝟮𝟭 𝗧𝗼𝗼𝗹𝘀, 𝟮𝟭𝟯 𝗦𝗰𝗲𝗻𝗮𝗿𝗶𝗼𝘀

We just published an article detailing the major expansion we have done to the data behind EVA-Bench.

🗂️ Data: https://t.co/AJuz1Vcb4w
📄 Article: https://t.co/CNjzAycneQ

#VoiceAgents #OpenSource #Data #AIResearch #ServiceNowResearch

[Sai Rajeswar @RajeswarSai]

Big moment for the whole 𝗘𝗻𝘁𝗲𝗿𝗽𝗿𝗶𝘀𝗲𝗢𝗽𝘀-𝗚𝘆𝗺 team. 🚀
𝗝𝗲𝗻𝘀𝗲𝗻 announced Apriel on the K25 stage. This year, EnterpriseOps-Gym made it into his 𝗖𝗼𝗺𝗽𝘂𝘁𝗲𝘅 𝗸𝗲𝘆𝗻𝗼𝘁𝗲. 🙂

NVIDIA announced 𝗡𝗲𝗺𝗼𝘁𝗿𝗼𝗻 𝟯 𝗨𝗹𝘁𝗿𝗮, the largest Nemotron 3 model to date, with 550B parameters / 55B active ..and it was incredibly exciting to see results on our benchmark featured in the keynote.

Enterprise agents need benchmarks that go beyond static QA, ones that test:
- long-horizon planning
- reliable tool use
- stateful workflows
- enterprise realism and ability to act without breaking things downstream

That is exactly why we built EnterpriseOps-Gym. Huge congrats to the Nemotron team for building agentic capability at the frontier level of intelligence and cost. More to come. 🔥 @NVIDIAAI @nvidia @ServiceNowRSRCH @ServiceNowNews @sagardavasam @shiva_malay @PShravannayak

https://t.co/PJvBFnYsm9

[ServiceNow AI Research @ServiceNowRSRCH]

(1/6) - Announcement

Introducing EVA: A Framework for Evaluating Voice Agents — measuring Accuracy (EVA-A) AND Experience (EVA-X) for real deployment decisions.

Bot-to-bot simulation. Open-sourced. Architecture-agnostic. Extensible by design. Try EVA and tell us what you find! 👇
🌐 https://t.co/MZgNqELJt3

[Dimitris Papailiopoulos @DimitrisPapail]

https://t.co/plZjz8c3nu

[goosewin @Goosewin]

guys you're never gonna believe this

[Anthropic @AnthropicAI]

Finding 1: The longer models reason, the more incoherent they become. This holds across every task and model we tested—whether we measure reasoning tokens, agent actions, or optimizer steps.

[Seph Mard @SephMard]

The "New" Frontier of AI Evaluation 🚀

I’ve been diving into the latest discussion between @lexfridman, @rasbt, and @natolambert. It's exciting to hear how critical evaluation is to expanded the frontier of AI. For anyone building in the evaluation space, the shift is clear: we are moving from "vibes" to verifiable truth.

My key take aways 👇

🔹 RLVR is the New RLHF: The biggest shift in 2025 is Reinforcement Learning with Verifiable Rewards (RLVR). Instead of subjective human preferences, we are scaling models against binary, verifiable truths—math answers, code execution, and logic chains.

🔹 The Contamination Crisis: Static benchmarks are breaking. With models potentially "training on the test," the gold standard is shifting toward dynamic, private evaluations and data created after training cutoffs to ensure true reasoning, not memorization.

🔹 Eval-as-a-Moat: In a world of massive compute, the highest leverage point for researchers isn't more GPUs—it's building better rubrics. Defining a representative problem set that frontier models fail on is now a "career rocket ship."

🔹 Beyond Code Snippets: The next hurdle for evaluation is moving past standalone functions and into "distributed systems" and complex architectures where ground truth is harder to define.

[Google Research @GoogleResearch]

Introducing ATLAS: New scaling laws for massively multilingual language models. We offer practical, data-driven guidance to balance data mix and model size, helping global developers better serve billions of non-English speakers. Learn more: https://t.co/8FsHLBKsou

[Epoch AI @EpochAIResearch]

Models that are good at math benchmarks tend to be good at coding and reasoning benchmarks too, pointing to a common factor driving AI capabilities.

We find that AI benchmark scores are nearly as correlated across domains (0.68) as within them (0.79).

[Paul Graham @paulg]

It's exhausting to be lied to. I think some of the most accomplished liars even take advantage of this fact. Their lies then serve a double purpose: (a) whatever purpose each specific lie serves and (b) to exhaust their audience and thereby batter them into submission.

[Andrew Gordon Wilson @andrewgwils]

We introduce epiplexity, a new measure of information that provides a foundation for how to select, generate, or transform data for learning systems. We have been working on this for almost 2 years, and I cannot contain my excitement! 1/7

[Paul Graham @paulg]

It's so much more interesting focusing on things that are unfashionable, undervalued, overlooked. And there are so many of them! People are great at overlooking.

[Jasper @zjasper]

GAUSS: General Assessment of Underlying Structured Skills in Mathematics

We’re excited to launch GAUSS, a next-generation math AI benchmark built to overcome the limitations of low skill resolution in today’s benchmarks.

What it does
GAUSS profiles LLMs across 12 cognitive skill dimensions, spanning knowledge, reasoning, learning, and creativity, offering a precise and comprehensive view of models’ mathematical ability.

Why it matters
By exposing strengths and weaknesses at a fine-grained level, GAUSS lays the foundation for advancing math AI from surface-level pattern recognition toward genuine reasoning and understanding.

What we found
Applying GAUSS to GPT-5 Thinking, we learned:

✅ Strong in taxonomy recall, evaluating arguments, plausibility checks, summarizing advanced papers, and posing problems

❌ Weak in theorem application, symbolic computation, problem-solving strategies application, geometric intuition and generalization.

What’s next
We’re building curated problem sets with rubrics via community crowdsourcing, skill charts for LLMs, and an AI auto-grader, foundations for model training toward math superintelligence.

We warmly invite everyone to join the GAUSS community, contribute problems through our portal and help shape the future of Math AI!

This work was led by myself and Jiaxin Zhang (@JiaxinZhang626) at @hyperbolic_labs / @Caltech, together with Qiuyu Ren & Tahsin Saffat at @UCBerkeley, Lily Liu (@eqhylxx) at @UCBerkeley → now @OpenAI, Zitong Yang (@ZitongYang0) at @Stanford, Prof. Banghua Zhu (@BanghuaZ) at @nvidia / @UW, and Prof. Yi Ma (@YiMaTweets) at @UCBerkeley / @HKUniversity.

Links and details below 👇 (1/n)

[Startup Archive @StartupArchive_]

Naval Ravikant on why you don’t need to be secretive with your startup idea

“You can always recognize the first-timers because they’re too secretive. And you can always recognize the experienced ones because they don’t care. Once you’ve done these a few times, you realize how much execution, difficulty, and risk there is, and how hard it is to get people to listen to you and believe you. Eventually you end up shouting your idea from the rooftops, just in the hope that somebody will actually use the product. So you end up with the opposite problem.”

Video source:@gigaom (2010)

[Dylan O'Sullivan @DylanoA4]

Murakami, what a line

[Amanda Askell @AmandaAskell]

I think good intellectual communities need a mix of naive young fast updaters and wise old slow updaters. The fast updaters introduce a lot of ideas but also a lot of trash. The slow updaters act as filters and sanity checks. Both groups correct for the deficiencies in the other.

[Anil Madamala @amadamala]

@cohere It seems to be down right now. I'm seeing 404 page @cohere

[Republicans against Trump @RpsAgainstTrump]

Mark Cuban explains how it was actually Trump who raised gas prices