“I want to build X.”
What needs to happen first?
Behind every scientific contribution is a pathway of enabling methods, datasets, tools, models, ideas, and prior work.
We introduce SciPaths to study whether LLMs can predict these pathways. 🧬🧭
Excited to be in Seoul for ICML! I will be presenting our work on forecasting scientific discovery pathways on Saturday July 11th at the:
- AI Forecasting Workshop: 12:15-13:30
- AI4Science Workshop: 15:55 - 16:55
Would also love to catch up with anyone who’s there!
Excited to be in Seoul for ICML! I will be presenting our work on forecasting scientific discovery pathways on Saturday July 11th at the:
- AI Forecasting Workshop: 12:15-13:30
- AI4Science Workshop: 15:55 - 16:55
Would also love to catch up with anyone who’s there!
“I want to build X.”
What needs to happen first?
Behind every scientific contribution is a pathway of enabling methods, datasets, tools, models, ideas, and prior work.
We introduce SciPaths to study whether LLMs can predict these pathways. 🧬🧭
WorldReasoner is now open-source! Dive into the code and benchmark here:
GitHub: https://t.co/a8Qu4Glhov
Thanks to @Eric_chamoun@ZifengDing6@vlachos_nlp
(4/4)
How to test agent forecasting without data contamination?
Isolate them in a temporal sandbox⏳
Introducing WorldReasoner: a new framework that restricts agents to simulated historical dates to test genuine reasoning over time-bounded evidence
https://t.co/vi1iN6BrpX (1/4)
What reusable contributions did your paper create & what made them possible? 🧪
Try the demo with an arXiv paper w/ downstream citations. It finds target contributions from downstream use, then their enabling contributions + prior-work groundings.
Demo: https://t.co/jfSwn3UcIp
“I want to build X.”
What needs to happen first?
Behind every scientific contribution is a pathway of enabling methods, datasets, tools, models, ideas, and prior work.
We introduce SciPaths to study whether LLMs can predict these pathways. 🧬🧭
This matters for AI4Science agents 🤖🔬
A system that proposes a research direction should also reason about feasibility:
• what must already exist
• which prior work provides it, if any
• what needs to be built next
That is the capability SciPaths is designed to test.
“I want to build X.”
What needs to happen first?
Behind every scientific contribution is a pathway of enabling methods, datasets, tools, models, ideas, and prior work.
We introduce SciPaths to study whether LLMs can predict these pathways. 🧬🧭
Task B shows why decomposition matters 🔎
With gold enabling contributions, prior-work grounding improves substantially.
But model-generated decompositions do not yet improve end-to-end grounding over direct claim retrieval.
Often, the bottleneck is knowing what to search for.
Excited to share that our paper on automatically uncovering how NLP papers frame their research will be presented this week at the EMNLP 2025 main conference in Suzhou by @vlachos_nlp! 🎉
Paper, code and data below👇
@michael_sejr@nedjmaou
What are NLP papers really saying about the purpose and use of their models/datasets? 🤔
Who are they for? What problems do they solve? How are they used?
We built a framework + tool to:
(1) analyze framing trends across papers
(2) help authors reflect on their own framing 🧵
Just released a new paper !
We show that LLMs encode the relative position of objects in a 3D subspace in their weights.
A rudimentary world model.
(🧵)
Check out our new preprint on analysing the research framings of #NLProc papers (w/ a focus on automated fact-checking, hate speech and related tasks)
With fantastic collaborators @Eric_chamoun (lead) @michael_sejr and @vlachos_nlp !
📝👇🏻