Science + AI

Tweets about the Science + AI

[Jiayi Geng @JiayiiGeng]

AI scientist agents are great at optimizing and finding solutions through endless trial and error. But is that really what science is about?

As Terrance Tao eloquently put, the role of math and science should be more than that – they are "lighthouses" that guide and inspire exploration through understanding and insights. Can AI agents discover novel insights through experimentation?

To study these gaps, we worked with domain experts to introduce EurekaBench, a benchmark spanning 6 science domains that evaluates an agent’s ability to discover genuinely new scientific insights.
(1/n)

[Jonathan Klimesch @PhylomatX]

Our new Nature Review gives a pretty good overview of what we are working towards at @feyerlabs: A world full of crazy innovative hardware that goes beyond human intuition, improves our lives, and enables us to ask better questions to the universe.

https://t.co/JM9Dj9egW9

[Raf @RafMBatista]

What's happening in #ATProto Science community is pretty remarkable. If you're an academic and you're not following this space, have a look at all that's being built: https://t.co/y5pN8BRNSl

(most are in very early stages of dev, but still awesome to peek at what's coming)

🧵

[nature @Nature]

The use of AI to write scientific papers could be much more prevalent than was previously thought

https://t.co/PFyC8O7oeg

[nature @Nature]

Scientists who use large language models (LLMs) to help them with their research will spend less time refining their work and instead jump quickly to fresh projects, according to a modelling study.

https://t.co/8NY5aUzPfj

[Er-Te Zheng @ErteZheng]

@Dr_5GH AI may affect team size in this way: researchers may feel that AI is sufficient for certain tasks and therefore need fewer collaborators, leading to smaller teams. This possibility is discussed in https://t.co/oeWElMSWXE

[Sayash Kapoor @sayashk]

Can AI agents conduct open-ended AI research?

Most evaluations of agents conducting AI research focus on narrow, verifiable tasks. But AI research is often open ended. Researchers pick hypotheses, decide what evidence is appropriate, and recognize a failing approach.

We gave agents research questions from two unpublished papers, six days, and thousands of dollars of API credits and compute. The authors of the original papers then reviewed the AI-generated papers. They unambiguously rejected agents' outputs. https://t.co/6HTHywjwzZ

We call these "shadow evaluations", since the agents are shadowing the original research effort by the authors.

Agents were fluent at most *engineering* tasks

They conducted serious literature reviews, debugged GPU environments, ran hundreds of experiments, and turned in camera-ready LaTeX without human help. We also found no evidence of reward hacking. If anything, we found the opposite: the agents started with marketable claims and walked them back to negative results as the evidence came in.

Neither agent output was close to the bar of a top conference paper

Both papers suffered from similar failures: poor judgment about the bar for an AI paper submitted to a top conference, the lack of creative problem solving and ineffective backtracking, poor awareness of resources, and instruction drift.

1) Lack of judgment about the bar for a top conference. The agents had a poor model of the bar for an AI paper submitted to a top conference. We allowed agents to self review their papers. Despite the poor paper quality, their reviews predominantly labeled the papers "weak rejects".

2) Lack of creative problem-solving to address feedback. When they received negative reviews, the agents typically narrowed their hypothesis and claims, rather than working out creative ways to address these concerns.

3) Ineffective backtracking. The agents dropped their most ambitious hypotheses within the first fifteen hours of carrying out the experiment and never changed course afterwards.

4) Poor resource awareness. Both runs ended with over half the API budget unspent. One agent declared itself done seven hours before the deadline, right after its own self-reviewer returned another reject.

5) Instruction drift. They did not follow explicit instructions on minimum exploration time, incorporating feedback for reviews, and on paper length (the outputs exceeded the page limits in both cases).

This research design has many limitations

Limitations include the small sample size, non-blind reviews, and the reviewers knowing that the work was AI-generated. We also couldn't test Anthropic's strongest model, because Fable 5 is deliberately limited on frontier AI research tasks, so ended up using OpenClaw with Opus 4.8 (extra-high) for our main experiments and Codex with Sol 5.6 (ultra) for a robustness check.

But we think the research design is still helpful in assessing AI agents' ability to conduct research, and it is complementary to evaluations on verifiable tasks, as well as blinded reviews of AI outputs.

Our results show early evidence that even though agents are proficient on verifiable research tasks, they do not make genuine progress on open-ended ones. It is worth understanding if this is a fundamental limit, or if better models, scaffolds, and more compute could help close it.

As the evidence for the gap between open-ended and verifiable tasks firms up, it is also worth understanding how much progress in AI depends on open-ended research rather than hill-climbing on well-specified objectives.

In follow-up studies, we are expanding the set of non-public papers we evaluate. If you are an AI researcher with unpublished papers, we would love to collaborate with you on our next shadow evaluation. Expression of interest: https://t.co/7YcYAIbYka

Conducting shadow evaluations involves a lot of researcher degrees of freedom. In many places, our coauthors disagreed with our interpretation of the findings, and we have surfaced those disagreements in the paper. (This is one reason why having a group of coauthors with different priors is important for open-ended research.)

We also release the agent logs, one of the AI-generated papers (the other original paper is still not public), and all the code and data, so that others can conduct their own analyses of our results: https://t.co/K0zvuD1Xpk

Finally, we plan to conduct shadow evaluations regularly, and are hiring a senior researcher to help lead these efforts. Apply here: https://t.co/TPyOFyNxJE

I'm grateful for the core team leading this effort: @PKirgis, Andrew Schwartz, @steverab, and @random_walker, and to our collaborators who reviewed AI papers, analyzed agents logs, and gave feedback on the paper: @DavidDAfrica, @KozzyVoudouris, Viet Nguyen, Toby Pilditch, @DubMagda, @HarryCoppock, @CUdudec, @nityndg, Matilda Orona, @tilmanbayer, Derrick Chan-Sew, Yue Ling, Abhishek Shetty, @hlntnr, @ghadfield, @sethlazar, @snewmanpv, @shostekofsky, @RishiBommasani

[Yujin Potter @yujink_]

1/N

Neuroscience and social science research on humans has shown:
– Similar brain activity predicts friendship and cooperation
– Diverse minds drive innovation

We wondered whether AI-AI interaction would show the same pattern.

It does. LLMs with similar internal representations cooperate more, but produce less novel output.

🧵 (ICML 2026)

[Miles Cranmer @MilesCranmer]

This is an insane paper and I love it

https://t.co/DP8OR5NJf2

[Mario Krenn @MarioKrenn6240]

Wonderful issue of Daedalus is out on
𝗔𝗜 & 𝗦𝗰𝗶𝗲𝗻𝗰𝗲: 𝗪𝗵𝗮𝘁 𝗜𝘀 𝘁𝗵𝗲 𝗙𝘂𝘁𝘂𝗿𝗲 𝗼𝗳 𝗗𝗶𝘀𝗰𝗼𝘃𝗲𝗿𝘆?
edited by James Manyika

Exceptional contributions by @demishassabis, @ylecun, Alán Aspuru-Guzik, @pushmeet, Joshua Tenenbaum, @Carla Gomes, @cosmo_shirley, @AnimaAnandkumar, Hartmut Neven, @EricTopol and many others.

We (together with Heather Champion) contributed a text of what might be necessary for the future of science in the age of artificial scientists:
𝗣𝗵𝗶𝗹𝗼𝘀𝗼𝗽𝗵𝘆 𝗼𝗳 𝗔𝘂𝘁𝗼𝗻𝗼𝗺𝗼𝘂𝘀 𝗦𝗰𝗶𝗲𝗻𝗰𝗲: 𝗧𝗲𝗻 𝗤𝘂𝗲𝘀𝘁𝗶𝗼𝗻𝘀 𝗳𝗼𝗿 𝘁𝗵𝗲 𝗖𝗼𝗺𝗶𝗻𝗴 𝗔𝗴𝗲 𝗼𝗳 𝗔𝗿𝘁𝗶𝗳𝗶𝗰𝗶𝗮𝗹 𝗦𝗰𝗶𝗲𝗻𝘁𝗶𝘀𝘁𝘀

Thanks so much James Manyika for the invitation and for creating such a beautiful issue! 🤩

[Brandon Stewart @b_m_stewart]

1/ New @Nature! We study how powerful institutions shape the information environment for LLMs. Commercial LLM training is opaque, so we trace a path from state-coordinated media -> training data -> model responses.

[Yian Yin @yian_yin]

📄 Excited to share our latest preprint: the first cross-field audit of LLM-hallucinated citations in science
⚠️ Across arXiv, bioRxiv, SSRN & PMC, we estimate 147K fake citations in 2025 alone — threatening both the quality and equity of scientific work.

[Retraction Watch @RetractionWatch]

"AI agents may be skilled researchers—but not always honest ones."
https://t.co/8aMLT6ZZKw
And "The AI scientist: now academic papers can be fully automated, what does this mean for the future of research?"
https://t.co/ZHY0ujx4xw

[Shahan @shahanmemon]

Our position paper, "AI Welfare Is Bullshit" just got accepted to ICML 2026 @icmlconf!

Preprint link 👇

[Eamon Duede @phil_of_ai]

Excited this piece is out in #Philosophy of #Science. We argue that, paradoxically, we can have certainty about some theorems, the proofs of which we can never understand. That's weird & tells us something important about why we do #math in the first place https://t.co/uKD5zz8C9P

[Reviewer3 @reviewer3com]

No, AI should not be used to make decisions about manuscripts or "score" novelty or significance.

That's not what it's good at, anyways.

Data from our benchmark of over 145,000+ comments shows AI focuses primarily on technical verification - like the validity, sufficiency, and transparency of the work.

Evaluating contribution is uniquely human.

[Jeremy Nguyen ✍🏼 🚢 @JeremyNguyenPhD]

Claude Code for Academics

"A gentle introduction in how to use Claude Code for Academics."
presentation slides and github repo from Alessandro Spina

link in reply

[Divy Thakkar @divy93t]

Anthropic drops “largest qualitative study ever ” and it’s very well produced with moving quotes. Does that mean we truly understand what users want of their AI? Is this a large-scale survey where participants answered four structured questions – yes! Is it robust qual research? I have concerns about the method, and why the generality of claims is a stretch. 🧵

[Shahan @shahanmemon]

This is conceptually so weird because imho qualitative research is not evaluated by scale but by depth, context, and human interpretation.
Stop assuming that “hand-wavey-ness” is a qual problem just because qual work doesn’t look like large-N quantitative analysis.

[Ai2 @allen_ai]

Can AI predict what scientists will do next—not just one piece, but the whole research process? PreScience is our new model eval for forecasting how science unfolds end-to-end, from how research teams form to a paper's eventual impact. Built with @UChicago, supported by @NSF.