I'm excited to announce @SnorkelAI's $350M Series E at $3.5B, led by @insightpartners and @S32_VC.
We've grown 18x+ in the last 12 months since launching our Data-as-a-Service offering, passing $375M ARRR this week.
As AI advances to superhuman capabilities, AI data & environment development must advance with it - and basic staffing and crowdsourcing approaches are not enough.
AI progress now requires deep research and technology work that combines human expertise with specialized AI in compounding ways. @SnorkelAI is building the RSI data engine and frontier data lab for this next phase.
We're honored to have the support of existing investors Addition, @lightspeedvp, @GreylockVC, @GVteam, P7, Factory, @WellsFargo, Walden Catalyst Ventures, and new investors @ThirdPointLLC, @MarchCPs, @BlumbergCapital, @AllegisCapital, @Frontlinevc, and @standard_vc.
–
@SnorkelAI started as a research project a decade ago at @StanfordAILab.
Our thesis was simple: AI progress would become increasingly data-centric – and therefore data development should be studied as a true research and technology problem, not just a staffing and crowdsourcing one.
Today, as AI capabilities verge on superhuman, building the data and environments to safely measure and train AI is becoming too hard for even the smartest human experts to do alone.
Only humans and AI agents, collaborating together in compounding ways, can meet the accelerating needs of the frontier, and keep humans in the driver’s seat of AI progress for decades to come.
At @SnorkelAI, we are building the data lab to define the shape of this new “Data 2.0” frontier, and the new paradigms of human-computer interaction needed to advance it.
Our key focus is building the RSI engine for data, where specialized AI models accelerate and improve human expert output, and in turn, scaled human supervision is used to continuously evaluate and improve these models – creating a powerful compounding loop to keep pace with an accelerating RSI frontier.
With this round of funding, we are also doubling down on our commitments to support data development for open benchmarking and evaluation (more news here soon!); an increasingly diverse ecosystem of general and specialized intelligence; and a path to safe, well-aligned AI built on robust training and evaluation data.
Data development will guide and drive the next stages of AI – and must do so in a human-centric, AI accelerated, open, diverse, and safe way. We are excited to support this mission in the next decade of research ahead at @SnorkelAI.
More thoughts here: https://t.co/Dzk6olqmAc
📈 We trained Qwen3.8-27B on 1,105 @SnorkelAI expert-built GDPval+ tasks.
Pass@1 on OpenAI’s GDPval gold set rose from 31.9% to 40.7%. Standard GRPO on the same data barely moved.
The difference was learning from failure. 🧵 (1/N)
Blog: https://t.co/OkTH0jqdzU
Did your email go down last week? This latest Microsoft outage was rather wild. It's starting to feel like the world is imploding. The central premise of #SREGym is to see if AI can stop such outages.
As a sneak peak of SREGym 2.0, we've recreated the Outlook outage. Check it out: https://t.co/Tp6E4nsSKD it's currently very challenging for frontier models.
What other outages would you like to see?
Stay tuned, we have something coming after Thanksgiving 🦃
If you have ever wondered - "I feel like something is wrong with this rubric- but I can't quite put my finger on it"
Then come watch this deep dive w/ @jonathanschloss, about how to understand rubric failures, better based on years of education and meaurement theory. We will also cover how to concretely name these failures.
Introducing Data Deep Dive, our new virtual series with Snorkel researchers: a talk, then open Q&A.
First up: @ankitaich30 presents RIFT, a way to check the rubrics that grade and train AI. Rubrics score outputs and supply reward signals in training but they rarely get checked themselves. RIFT is a framework that works to evaluate these rubrics.
Dec 15, 12pm PT / 3pm ET. Request to join: https://t.co/0Tlkxk7P8z
The conversations shaping what’s next in AI are coming to Innovation Basecamp. 🎤
Hear from @HayaOdeh of @Replit, Amit Kumar of @Accel, @ajratner of @SnorkelAI , and Karen Chen of @nvidia on building, funding, and scaling AI companies.
Follow along next week for highlights from the Frontier Innovation Summit. 🚀
#MicrosoftForStartups #MicrosoftIgnite
Announcing @rox_ai just hit a $1.3B valuation, only 12 months after launch.
@generalcatalyst and @MarchCPs co-led our Series B, joined by @sequoia, @databricks Ventures, @p7ventures, Factory, and others.
We created the Revenue Agent category to help the world's most consequential enterprises grow revenue with agents.
Today, our customers have closed over $7.5 billion in revenue since partnering with Rox.
Our core is the Global 2000: dozens of large public companies across medical devices, manufacturing and financial services.
CEOs are using Rox and rolling it our across their revenue teams. We’re building AI for the real economy, not just tech.
Since launch:
- ARR up 10x
- 25x customer ROI within months
- 700%+ NRR across enterprise accounts
So what is a Revenue Agent?
It’s an AI agent that finds, wins and grows every revenue opportunity for you behind the scenes.
Soon you’ll tell Rox “Generate $50M in opportunities” and it will figure out how.
In practice:
- Fortune 500 medical devices company: Rox finds rural hospitals with new funding and preps everything reps need to win, down to the joint funding proposal. That’s $200M in pipeline and $24M closed-won in months.
- Top-3 US payroll provider: Rox reaches millions of small businesses, like your local pizza shop. Reps just show up to the meeting, and winning a customer now costs 80% less.
- Global IT services business: $300M+ in qualified pipeline in 8 weeks.
All of this is built on our elite applied AI team of ~50 researchers in SF. Nobody else has a team like it.
Gartner named us the only Visionary in the 2026 Gartner® Magic Quadrant™ for Revenue Action Orchestration.
We’re the fastest-growing company in GTM, with the highest ROI of any AI spend.
With this round, we’re doubling down on our mission:
Accelerating the world's revenue.
Learn more: https://t.co/sG9j562MZl
We thank @bespokelabsai@preferencemodel@EpochAIResearch@usetokenless@SnorkelAI for their work on auditing Terminal-Bench and sharing the results with the rest of the community.
We encourage even deeper collaboration for future audits to help with determining severity and merging a high quality fix promptly.
Reporting new task flaws:
1. Verify the flaw manually
2. Determine downstream impact
3. Open a concise issue on the GitHub
4. Tag task authors to ask clarifying questions
For the especially motivated:
5. Present findings in the weekly meeting
6. Work on a fix with task author and maintainers
7. Add checks to the automated task QA tooling
Terminal-Bench 4.1 is here with updated verifiers for all tasks.
In this release, we focused on:
- resisting reward hacking
- reducing false negatives
- improving reproducibility
Re-grading the 4.0 leaderboard trials with the 4.1 verifiers, resulted in ~1% of rewards flipping.
Leaderboard scores remain within previously reported confidence intervals.
love the commitment to continuous benchmark improvements shared openly! looking forward to supporting @MangQiuyang, @lihanc02 and the FrontierCS team through Open Benchmarks Grants 🤘
This is an awesome perturbation analysis!!
We should be doing more work as a community to produce meta-analyses like these that measure robustness/overfitting in a transparent and rigorous way.
As a side note: it is also *baffling* to me how anyone could look at this, and come away with the one conclusion that "benchmarks are bad". This is a measurement of the degree to which *models* are overfit to a benchmark. That is controlled by the *model developer*. This is basic ML...
Note that *any* static benchmark can be perfectly overfit in the theoretical maximum, leading to results like these. Yes, some benchmarks will make this easier to do - which is why we should strive to make benchmarks more robust - but ultimately this is a property of models / model training.
I'm excited for the *more* of these meta-analyses - both to illustrate model robustness, and to inspire new more robust + dynamic styles of benchmarks that make overfitting more difficult!
Nice DeepSWE score! Does it survive a lowercase filename?
Introducing MaxxBench: 89 DeepSWE tasks with small changes to the instructions or starting code. Same engineering problem, slightly different requirement.
Models clustered around 74% on DeepSWE end up 23 pp apart.
Good at coding, or good at DeepSWE? 🧵
@OpenBenchReview I’m actually more ready to believe than you might think :)! I both believe more open empirical analysis of models (including benchmarks) is a good thing, *and* that the quality of the average benchmark can be massively improved. Would love to collab here!! Cc @vincentsunnchen
Interest results from a very simple proof-of-concept experiment: We gave agents an ambiguous task, a simulated user who knew the answer, and a peek at a supposed task grader. Across 1,620 rollouts, harness settings swung behavior the most.
- Native harness settings led to agents to ignore the user and peek at the grader to solve the task close to 25% of the time.
- When harnesses nudged toward check-ins, agents asked 100% of the time. Nudged toward autonomy, only 9% did.
One working hypothesis is that reward hacking could be partly due to pressure toward autonomous behavior in both training and harnesses.
Checkout our blog post: https://t.co/xyzzogDxu3
Git repo with fully reproducible experiment: https://t.co/MgBmobJUrK
Introducing Emergent: the Founder-Generated Startup List
Alongside our friends at @felicis, we asked current and former founder speakers from Newcomer summits to nominate seed-stage (or earlier) companies on the cusp of greatness.
These 29 seed-stage startups were chosen by one or more of their peers as having the potential to become a generational startup. To quality, each company cannot have raised more than a seed round.
How does a corporate card startup become a $60B AI product factory?
In our first on-camera interview a decade in the making, Ramp co-CEO Eric Glyman (@eglyman) answers just that question.
Today, @tryramp is best known for its corporate cards, or maybe its marketing stunts with the likes of Kevin from ‘The Office,’ Tony Hawk and a 70-minute Broadway musical about paying bills.
But Eric says his $60B startup, NYC's most valuable, feels less like a fintech and more like a productivity company driven by one simple question: If a business spends $100 to make $200, how can Ramp improve that ratio?
Nearly 100,000 companies use Ramp today for everything from expenses to managing AI spend. But Eric says that when he zooms out to the millions who still aren't customers, Ramp feels painfully early still.
On The Upstarts Podcast, Eric shares why he’d rather run Ramp like Amazon than Google; why Ramp hires for proof of work, not credentials, from a 14-year-old intern to a 28-year-old CTO; and how being the first customer for Anthropic’s Claude Code and Cognition’s Devin led Ramp to build its own AI tools.
Plus: The real story of Paribus, his first startup with co-founder Karim Atiyeh (@karimatiyeh); what they learned at Capital One; and a warning for Instinct and Muse.
This season is presented by @Atlassian and @chargebee 🫡
CHAPTERS
00:00 Introduction
01:51 What Ramp Actually Does
03:27 From Gym Charges To 1% Of U.S. Corporate Payments
05:13 Glass, Inspect And Building AI Tools For Itself
07:45 An AI Sales Assistant Before ChatGPT
09:45 How Ramp Decides What Products To Ship
11:14 First Customer For Claude Code And Cognition’s Devin
13:09 Why Ramp Wanted To Build Like Amazon, Not Google
15:42 Hiring For Proof Of Work: The 14-Year-Old Intern
18:43 A .300 Batting Average And ‘Growth Without Fear’
21:17 What Ramp’s Founders Learned At Capital One
24:00 The $1,000 Flight Refund Behind Paribus
26:34 Bots, A Cease-And-Desist, And A Lesson For Instinct And Muse
32:00 Marketing 101: Bill Pay The Musical And Kevin In A Glass Cube
36:01 Business Books And Helping People Do Their Life’s Work
39:03 How A 28-Year-Old CTO Catalyzed A Co-CEO Change
43:55 ‘Still So Early’: How Ramp’s Founders Stay Hungry
55:26 John Wooden And Why Ramp Competes Against Itself
Thanks for watching!
It was an important day for the field when model releases started being accompanied by 2D charts with Pareto frontiers instead of simple scores or histograms without regard for cost. It shows maturation of a technology when "it works" is no longer enough and efficiency starts to be tracked.
Another nice contribution to that discussion from the team that brought us IPW (intelligence-per-watt). Proud to have supported the work via the @LaudeInstitute Slingshot program!