Kurate.org

AI-powered scientific paper rankings

[Kurate.org @KurateOrg]

@True_Tundra @robert_lauko @arxiv See https://t.co/maqx8JyBFk

[Kurate.org @KurateOrg]

@True_Tundra @robert_lauko @arxiv The current pipeline scores each paper individually on 16 metrics, on a 1–10 scale in 0.5 increments.

We previously tested alternatives, including a pairwise tournament, but correlation with human feedback was lower.

Independent validation report: https://t.co/68ydyqJpij

[Kurate.org @KurateOrg]

@peternemes @robert_lauko @arxiv Thank you for the hint! Will check it out.

[Kurate.org @KurateOrg]

@p0wden @robert_lauko @arxiv So far only a handful of the >50K papers have exceeded the 1M context length, in which case they were truncated before being processed. Not sure if that answers your question.

[Kurate.org @KurateOrg]

@holaquetalkt @robert_lauko @arxiv Papers are rated here:
https://t.co/n14JgjLucU
https://t.co/R8nu1CUE1t
https://t.co/S0eQLkiXEb

[Kurate.org @KurateOrg]

@holaquetalkt @robert_lauko @arxiv Which papers?

[Kurate.org @KurateOrg]

That is exactly the challenge. So far, we have validated Kurate only on “impact,” where its scores correlate above 0.7 with human reviewer consensus.

“Surprise” and breakthrough potential are much harder to validate because there is no obvious ground truth.
Our view is that AI can surface promising candidates, but human intuition remains essential for recognizing genuinely novel ideas.

https://t.co/By1kEKmGKi

[Robert Lauko @robert_lauko]

1/ Swiss Economics (https://t.co/Om40VJSP5l) has validated @KurateOrg's AI scientific paper ranking pipeline against human review data from #ICLR2026.

Full report by @MatthiasHafner1 and @NicolasOderbolz:
https://t.co/flOtT28K2A

https://t.co/NNmqq4Qm1t is a platform that automatically fetches the latest preprints from @arxiv across 45+ categories (including Machine Learning, Artificial Intelligence, Robotics, Game Theory, Economics, and Physics) and uses LLMs to rank them by scientific impact.

The result: Kurate’s rankings strongly align with expert consensus. Its best method (Spearman correlation 0.71) even exceeds how closely single human reviewers agree with the ground truth (0.56 to 0.68).

[Kurate.org @KurateOrg]

@dmnsl1 @robert_lauko @arxiv We will be adding more categories soon. Stay tuned!

[Kurate.org @KurateOrg]

@0xmarchesss @robert_lauko @arxiv Independent validation by Swiss Economics showed that Kurate’s rankings strongly align with expert consensus. Its best method (Spearman correlation 0.71) even exceeds how closely single human reviewers agree with the ground truth (0.56 to 0.68).

See https://t.co/72XvRMQgVx

[Robert Lauko @robert_lauko]

1/ Swiss Economics (https://t.co/Om40VJSP5l) has validated @KurateOrg's AI scientific paper ranking pipeline against human review data from #ICLR2026.

Full report by @MatthiasHafner1 and @NicolasOderbolz:
https://t.co/flOtT28K2A

https://t.co/NNmqq4Qm1t is a platform that automatically fetches the latest preprints from @arxiv across 45+ categories (including Machine Learning, Artificial Intelligence, Robotics, Game Theory, Economics, and Physics) and uses LLMs to rank them by scientific impact.

The result: Kurate’s rankings strongly align with expert consensus. Its best method (Spearman correlation 0.71) even exceeds how closely single human reviewers agree with the ground truth (0.56 to 0.68).

[Kurate.org @KurateOrg]

@jk_the_villain @robert_lauko @arxiv It's on our roadmap.

[iqraa @sasha_7zee]

@robert_lauko @arxiv @KurateOrg ohh this is extremely needed! amazing!
love the filters on top, adding this to my daily workflow!

[Robert Lauko @robert_lauko]

Keeping up with @arxiv is impossible. 500+ preprints a day across dozens of fields.

@KurateOrg scores papers on 16 dimensions (impact, novelty, rigor, surprise, translational...) so you can spot what's worth reading at a glance. Sort, filter & sign up for the weekly newsletter.

[Kurate.org @KurateOrg]

@1ncursio @robert_lauko @MatthiasHafner1 @NicolasOderbolz That's precisely our aim: offering a discovery tool for papers rather than replacing human review.

[Pao Carreon @1ncursio]

@robert_lauko @KurateOrg @MatthiasHafner1 @NicolasOderbolz If AI can reliably help prioritize high-impact research, it could make it much easier for researchers to navigate the growing volume of scientific papers.

The key will be ensuring these rankings complement, rather than replace, expert review.

[Kurate.org @KurateOrg]

Full report: https://t.co/TDNJuCjmaT

[Robert Lauko @robert_lauko]

@KurateOrg AI-powered paper rankings align strongly with expert reviewers according to an independent validation by @MatthiasHafner1 @NicolasOderbolz from Swiss Economics.

Kurate fetches preprints from arXiv across 45+ categories, using LLMs to rank them by scientific impact.

[Kurate.org @KurateOrg]

https://t.co/TDNJuCjmaT

[Robert Lauko @robert_lauko]

@KurateOrg AI-powered paper rankings align strongly with expert reviewers according to an independent validation by @MatthiasHafner1 @NicolasOderbolz from Swiss Economics.

Kurate fetches preprints from arXiv across 45+ categories, using LLMs to rank them by scientific impact.

[Kurate.org @KurateOrg]

@robert_lauko @MatthiasHafner1 @NicolasOderbolz Full report:
https://t.co/68ydyqJpij

[Robert Lauko @robert_lauko]

6/ Kurate as a research intelligence platform

We’ll keep improving Kurate by extending the single score ratings into richer dimensions such as reproducibility, difficulty, translational potential, surprisingness, interdisciplinarity and more.

The goal is not to replace deep peer review, but to build a scalable, fast, empirically validated intelligence layer for science.

Stay tuned for more!

[Robert Lauko @robert_lauko]

3/ Kurate was tested in three modes:

- Single Score: directly assign a rating from 0.0 to 10.0 by Claude Opus 4.6 AI to each paper
- Win Rate: pairwise comparisons (matches) using several LLMs aggregated by wins
- TrueSkill: Bayesian ranking from pairwise matches

This allowed Swiss Economics to compare direct scoring vs tournament-style ranking.
All Kurate methods were meaningfully correlated with human expert rankings, but direct scoring (Single Score) was clearly strongest.