Haiyang Wang

Grad@Caltech in Physics, Autoresearch

[James Zou @james_y_zou]

AI is cracking many famous conjectures. But can it ask new questions worth answering?

With a team of mathematicians and physicists, we built #After42, a system that generates novel research problems that human experts find genuinely interesting.

[Fan Nie @FanNie1208]

AI is now taking on some of the hardest open problems that scientists have posed. But research is broader than solving problems. Can AI also propose research questions that experts find genuinely interesting?

We built After42 to find out. Experts judged 45 of its 135 final questions genuinely interesting, across mathematics, statistics, theoretical computer science, and physics, with wide variation across subfields.

The key idea: develop questions by trying to solve them.

We also asked whether AI can judge which questions are interesting. We find that model judges' top choices align with expert consensus but do not consistently capture individual experts' preferences.

We release the expert-assessed questions as an open resource for researchers to discuss, refine, and pursue, as a step toward integrating AI into the broader research enterprise.

https://t.co/iOrs6L7E9Y

[Zhenting Qi @ZhentingQi]

Research index (e.g. Google Scholar) is one of the largest collaborative systems where millions of human users build on each other's work across centuries.

How about we build something similar for async LLM agents?

We give agents a minimal hypertextual system. They asynchronously search for and read related work, cite the work that they find useful, and publish (mini-)papers.

Each search returns only the top k papers, ranked by relevance and a PageRank-like citation weight, which bounds the read cost of each agent to O(k), independent of agent count.

We show that performance effectively scales as we keep adding agents, and the system is robust to absence/failures of individual agents.

[Martin Marek @mrtnm]

Why is RL so sensitive to training-inference mismatch (TIM)?

With each training step, the trainer is pulled toward the sampler, resulting in accumulating drift.

We exploit this intuition to devise a novel correction method to stabilize RL under TIM, called Score Centering.

[Max Ryabinin @m_ryabinin]

RL with LLMs is very unstable when training and sampling policies differ. Standard fixes (matching numerics, importance sampling) work around the problem.

We find the root cause of this instability from first principles and propose a way to directly cancel it. Score Centering is competitive and compatible with existing approaches — while simple to implement!

🧵 [1/6]

[Fan Nie @FanNie1208]

AI-generated scientific output is scaling beyond available human review capacity. Scaling scientific discovery requires scaling verification.

We introduce OSVerify, a framework for one-sided verification: rejecting as many invalid solutions as possible while preserving rare valid ones for expert review.

We used OSVerify to referee 11 recent OpenAI math preprints, identifying 5 major and 13 minor issues, all verified by human experts.

Our key insight: checking a proposed error can be easier than judging an entire solution. Instead of directly judging solution correctness, OSVerify decouples verification into error proposal (searching for potential errors) and error validation (checking whether they are real and substantial).

On competition and research mathematical benchmarks, OSVerify reduces invalid solutions requiring expert review by 46% on average compared to voting under the same low-FNR constraint, or 57% with an RL-trained validator.

Verification can also guide discovery. OSVerify can be integrated into existing scientific discovery workflows. Paired with an agentic math solver, it helped resolve 3 open conjectures in the Langlands program and contributed to resolving the Manin constant conjecture. All proofs were confirmed by mathematicians.

Our broader goal is to provide verification-based evaluation signals for scientific discovery, enabling test-time scaling and self-improvement even when reliable automatic evaluators are unavailable.

Project: https://t.co/pSW9iJhvHg
Paper: https://t.co/WUXDDhIz0o
GitHub: https://t.co/2BF5EfdbUe

[1/n]

[Haiyang Wang @Haiyang0721]

@DKokotajlo Alignment will not be the most appropriate word in the near future. We can only "align" the model when we have good enough understanding of the model capability. "Education" might be a much better thing to dive into.

[Haiyang Wang @Haiyang0721]

https://t.co/sOPWnTeVVp
Release.
This is the fastest ray tracing code for black hole ray tracing to date. You can also run on arbitrary spacetime.

[Haiyang Wang @Haiyang0721]

Built upon numerical spacetime, by constructing binary black hole initial data with xcts formalism.

[Haiyang Wang @Haiyang0721]

Fig. 1 (Binary Black Hole Gravitational Lensing)