Hatem Dawaghreh

human data @scale_AI | retired mathematician

[Hatem Dawaghreh @heyhatem]

https://t.co/VDfb4wg7so

[Hatem Dawaghreh @heyhatem]

@aakashsabharwal @scale_AI You will be missed @aakashsabharwal! Excited to see everything you do next

[Hatem Dawaghreh @heyhatem]

@_jasonwei this is great -- sucks how most hotels/conferences in SF place visitors away from the best it has to offer. big part of why the city is underrated

[Hatem Dawaghreh @heyhatem]

@michaelrbock @HelloSurgeAI @mercor @scale_AI I’m curious why you say “previously” @scale_AI? 😵‍💫

We do significant work with lots of labs and work on data production across all 3 categories you mention

[Hatem Dawaghreh @heyhatem]

This is why evals are incredibly important.

Despite the weekly improvements we see in models, it doesn't make sense to deploy them to high-stakes workflows without sufficient evaluation, verification, and oversight.

Evals will be the central pillar of insurance, risk, and security.

[Zachary Cohen @ZcohenCNN]

NEW: The US military was prepared to board a Chinese vessel in the Middle East earlier this spring -- during war with Iran -- after an intel report concluded it was transporting components of a nuclear weapons program.

But just before the planned operation, officials discovered the report had been generated w/ help of AI & chatbot used by the analyst inaccurately identified what the ship was carrying, sources tell @KatieBoLillis & me.

[Hatem Dawaghreh @heyhatem]

@Yminglai "people reply to Slack messages from their own understanding first, and only then search the wiki / Slack / other internal apps to verify and fill gaps" 🔥 analogy

[Hatem Dawaghreh @heyhatem]

Special shoutout to the brains of this operation @mhrezaeics and @nas_mahmoud_

[Hatem Dawaghreh @heyhatem]

There isn’t a good measurement today for recursive self-learning progress despite how crucial and widely discussed it is.

This is where RSI-Bench comes in: @ScaleAILabs’s effort to measure agent’s long-horizon capabilities to experiment and execute research processes.

We’re also opening this initiative to the research community, where successful contributions receive a financial reward and co-authorship to the benchmark. Details in the link below!

https://t.co/IcpINzkuMX

[Hatem Dawaghreh @heyhatem]

Good ingredients cost real money. The full value is unlocked by human ingenuity in the kitchen.

$10M is the ingredient line but the meal prices higher.

[Aakash Sabharwal @aakashsabharwal]

Why $10M for raw files? Because the price reflects the complexity. Spirit employed ~17,000 people at peak — airport ground ops, flight scheduling, aircraft leasing & finance, multi-region HR and payroll, all interlocking over 34 years.

That operational entropy is something even our most frontier models can't generate synthetically, at least not yet! You can't ask Fable to hallucinate it.

Safe to assume every major bankruptcy or shutdown now comes with an artifact acquisition cycle. The bigger and more operationally complex the org, the more valuable the estate.

[Minglai Yang @Yminglai]

RL against rubrics quietly reward-hacks: the training judge keeps scoring higher while true quality falls.

One-line fix: randomly drop part of the rubric each step, so the policy never optimize the same set of rubrics twice.

📄 https://t.co/BSrzyJOciF

[Hatem Dawaghreh @heyhatem]

Examples like this are why quality infa for reasoning traces and reward hacking detection are non-negotiable when producing RL data

[Alexander Panfilov @kotekjedi_ml]

But we also took a chance to have a look at some in-the-wild scheming, reward seeking, etc. examples, and dumped it in appendix.

1) Summarizer unfaithfulness

Reasoning summaries often omit important information from the original trace.

Here, Opus 4.8 realizes it knows the answer to an AIME problem and then tries to fit a solution to that answer. None of this appears in the summary.

[Hatem Dawaghreh @heyhatem]

Great take - there’s a strong parallel to data and model development here. Preclinical has lots of momentum because verification can be cheap and near-instant.

Clinical is much longer horizon and will see a fragmented approach, where sub-capabilities like regulatory filing, patient matching, trial design, and site selection each become their own hill to climb.

Neo-CROs, neo-CDMOs, etc. and their relevant expertise will matter much more than people realize.

[Nicole Fitzgerald @ninklefitz]

over the next several years we will see the entire preclinical & clinical stack rebuilt — neo-biotech, but also neo-CRO, neo-CDMO, neo-regulatory, etc. the form factor of these functions will be fundamentally different than that of their predecessors

this is already well underway on the preclinical end of the spectrum (whole new ecosystem of computational tools/predictive models, API-forward CROs, advances in automated labs)

[Hatem Dawaghreh @heyhatem]

Scale has great talent density and, as team, has a proven ability to grind through hard problems and define the market.

While we were one of first players in this space, our work is far from finished. Keep an eye out :)

Btw - we were not acquired and remain totally independent.

[Rory O'Driscoll @rodriscoll]

I was totally wrong on Scale AI. I had originally thought that the acquisition and the exodus would leave them a husk.

It proves that when you're in a great market with a product that meets real demand, even losing your top people won't kill you. Congrats on the great CEO hire.

[Hatem Dawaghreh @heyhatem]

Meta has top-tier momentum rn

[Alexandr Wang @alexandr_wang]

1/ big announcement today: we will be releasing an open weight version of muse spark 1.2 soon.

we also are releasing muse glimmer, a 30B agentic model with open weights under apache 2.0. muse glimmer can run on 24GB of VRAM without losing agentic reliability. 🧵

[Hatem Dawaghreh @heyhatem]

Incredible release! It's highly consequential to exhibit the scaling models for world-action models.

As these models expand into new domains, expect to see lots of demand for humans generating new (and high-quality) experiences needed for learning!

[Dyna Robotics @DynaRobotics]

Today we are introducing Dyna-2, a world-action model pre-trained on one million hours of human video. At this scale, for the first time, we discovered several new scaling laws:

• world-action models exhibit scaling law on human data across four orders of magnitude, from 1000 to 1,000,000 hours,
• this human data scaling law implied a scaling law on never seen robot data,
• both data and objective matter; world modeling and scaling on video data are essential for cross-embodiment scaling transfer to emerge

🧵

[Hatem Dawaghreh @heyhatem]

@catmanyau @Sbhaiwala03 It’s the whole package. Everything across environment, task design, verifiers, and the annotators (many of which are highly specialized academics or successful professionals). There’s also a lot of valuable work that goes into understanding the reward signal of the training set

[Hatem Dawaghreh @heyhatem]

@Sbhaiwala03 Great read - I like the TAM expansion formula.

For the skeptics on cost per task climbing - consider what labs spend on compute when training on complex long-horizon tasks. It makes 0 sense to cut corners on data spend (provided quality is good) and risk your training outcomes.

[Hatem Dawaghreh @heyhatem]

+1. It's hilarious when I see attempts at reducing human data co's to "just data labelling".

Much of our work at Scale is research-based, especially our ops. We don't run data by just churning out labels.

What we do is run scientific pipelines that guarantee training outcomes.

[Aakash Sabharwal @aakashsabharwal]

Data <> Labs relationships are a research partnership, not a vendor contract!

Large parts of SV missed data as a sustainable business and still argue the revenue won't be sticky. That take is myopic.

There's no ceiling on what a frontier lab will pay for true model gains.

[Hatem Dawaghreh @heyhatem]

@claudeai You too @ChatGPT

[Hatem Dawaghreh @heyhatem]

We need a middle ground between turning global memory off and incognito chats on @claudeai

1. Do I want to find this again?
2. Do I want the model remembering that we had this chat?

These questions are orthogonal. A per-chat memory toggle would solve this

[Hatem Dawaghreh @heyhatem]

Big day today @scale_AI!

Excited to welcome Francis here and to continue pushing the frontier of AI with our human data engine

[Francis deSouza @fdesouza]

Big News: I’m joining @scale_AI as CEO, starting August 10. Scale sits at a rare intersection, working with the top AI labs to push the frontier while helping enterprises and governments actually deploy AI they can verify and trust. I’m excited to lead a company with a mission to develop reliable AI systems for the most important decisions. More to come.