New Horizons is healthy and cruising through the distant Kuiper Belt after the longest hibernation period of its mission yet. A recent beacon confirmed the spacecraft is operating normally more than 5.7 billion miles from Earth. 🚀🌌 https://t.co/jQhxf1Rcvr
This paper from Harvard and MIT quietly answers the most important AI question nobody benchmarks properly:
Can LLMs actually discover science, or are they just good at talking about it?
The paper is called “Evaluating Large Language Models in Scientific Discovery”, and instead of asking models trivia questions, it tests something much harder:
Can models form hypotheses, design experiments, interpret results, and update beliefs like real scientists?
Here’s what the authors did differently 👇
• They evaluate LLMs across the full discovery loop hypothesis → experiment → observation → revision
• Tasks span biology, chemistry, and physics, not toy puzzles
• Models must work with incomplete data, noisy results, and false leads
• Success is measured by scientific progress, not fluency or confidence
What they found is sobering.
LLMs are decent at suggesting hypotheses, but brittle at everything that follows.
✓ They overfit to surface patterns
✓ They struggle to abandon bad hypotheses even when evidence contradicts them
✓ They confuse correlation for causation
✓ They hallucinate explanations when experiments fail
✓ They optimize for plausibility, not truth
Most striking result:
`High benchmark scores do not correlate with scientific discovery ability.`
Some top models that dominate standard reasoning tests completely fail when forced to run iterative experiments and update theories.
Why this matters:
Real science is not one-shot reasoning.
It’s feedback, failure, revision, and restraint.
LLMs today:
• Talk like scientists
• Write like scientists
• But don’t think like scientists yet
The paper’s core takeaway:
Scientific intelligence is not language intelligence.
It requires memory, hypothesis tracking, causal reasoning, and the ability to say “I was wrong.”
Until models can reliably do that, claims about “AI scientists” are mostly premature.
This paper doesn’t hype AI. It defines the gap we still need to close.
And that’s exactly why it’s important.
With Ralph Pudritz, I've been editing the section of the 2nd edition of the Handbook of Exoplanets focused on "Formation and Evolution of Planets and Planetary Systems". This thread collects together the contributions currently available on astro-ph: (1/5)
So long and thanks for all the fish! I just observed my last star this morning... #GaiaDPAC and @esa Gaia teams will do their best to make some great data releases from all the data I gathered! #GaiaDR4 in 2026, #GaiaDR5 around the end of the decade. Keep posted for our news items today at 10:00 CET.
Great (and scary) visualization of 2024 daily temperatures compared to prior years by the BBC today. Evocative of the iconic Joy Division album cover from 1979: https://t.co/o2zLigFV7h
In response to this week's devastating Eaton Fire, we have established the Caltech and JPL Together Relief Fund to support our affected colleagues. Please consider making a contribution to this special fund. Thank you from all of us at Caltech. https://t.co/z5GxU2w0RF
@GregHerczeg So that we can check if we had the same driver! That’s what made me want to get back my Maryland drivers license, then convert it to a European one to enjoy die Luft!
Hubble took the closest look yet at the first known quasar.
Called 3C 273, this quasar is a brightly glowing galactic center powered by a black hole consuming material.
This helped astronomers open up a new gateway into better understanding quasars: https://t.co/16gBOXtWU5
People usually think replication attempts in science are rare. Journals don't publish replications, so scientists don't do them.
In reality there are countless replication attempts (and failures), it's just PhD students assume they did something wrong
https://t.co/cxGGexVOw2
@EricLagadec@EditionsduSeuil Le livre est tellement génial, que je savais pas que j'étais l'un des premières à avoir la chance de le voir avant sa sortie!