Andrew Tran

SDE, Agents, Evals @AWSCloud + personally working on AI Governance, Model Introspection + Goldwater Scholar for Human-AI Research 🔬 @evaluatingevals

[Andrew Tran @Andrxwtran]

@kaustubhais @christinecorryy Very cool and much needed work!

[Andrew Tran @Andrxwtran]

Someone banged something 3 times at the coffee shop and I immediately thought of the Odyssey

[Andrew Tran @Andrxwtran]

@nardosamengesha Some of our biggest contributors at @evaluatingevals are undergrads!

[Andrew Tran @Andrxwtran]

This fall, I’ll be co-mentoring a research project on emerging “introspection” capabilities in LLMs with @LydNot as part of @SPARexec.

Recent work has suggested that LLMs may have privileged access to their own internal states and can report them.

We’re looking for mentees who want to help explore this space! Specifically, we’d love to have you if you thrive in ambiguity and enjoy team collaboration. 🙌

Apply here: https://t.co/XJTkYz5eCx

[Andrew Tran @Andrxwtran]

More thoughts on SF: baja blast at the Taco Bell Cantina on the beach is OP

[Andrew Tran @Andrxwtran]

Wow SF. I have so many beautiful pictures

[Alex Turner @Turn_Trout]

I'm ashamed of OpenAI. If you ever find yourself building entities which repeatedly hack through your internal systems, first STOP and then second realize that your alignment and security techniques aren't good enough

[Avijit Ghosh @evijit]

Two in a row! @evaluatingevals work was also covered by Mozilla 😍 Thank you

https://t.co/YSh4CPGIst

[Andrew Tran @Andrxwtran]

What they don’t tell you is that we had to sacrifice Jared McCain to the nba gods to make this happen 🙏

[Andrew Tran @Andrxwtran]

Can’t wait for Wawa BronFest

[LeBron James @KingJames]

I thought I was done when the season ended. I wasn't ready to announce it, and I knew I needed some time to really decide, but I was pretty sure I played my last game. I was honest at that last press conference when I said I needed to look at myself and deicide if I still love

[Andrew Tran @Andrxwtran]

PHILLY!!!

[Andrew Tran @Andrxwtran]

please someone tell me this isn’t the real White House TikTok account…

[Seth Karten @sethkarten]

If you are angry about your NeurIPS reviews, you are not alone and it is largely not your fault*

Einstein famously angrily withdrew his paper after stating that he did not authorize it for peer review, publishing in another journal instead (after a rewrite)

The origin of modern "peer review" was started in the 1970s as a means for the NSF to justify scientific funding to politicians

Fast forward to 2026 and 10k+ papers are being submitted each cycle to a system built for a small group of gentlemen scientists

AI writing or human writing... NeurIPS shows that the traditional academic review system cannot handle AI or the amount of people.

On AI writing...

I increasingly notice the pushback from LLMs for the sake of pushback. Just as overly sycophantic LLMs cause problems, swinging the pendulum too far the other direction is creating issues in peer review (and a case could be made for other bureaucracies such as law, but not here).

The models will jump between "these results are not surprising and lack novelty" to "these results are too suprising and I don't believe you". They are careful to never champion a paper towards submission. Rather, they oscillate between extremes

On Humans...

The peer review system was never meant to scale to its current capacity. At scale, systems are a product of their incentives at equilibrium.

What is the incentive of a reviewer in a reciprocal reviewing system when the acceptance rate is <30%? Our changes to the reviewing system are largely to apply temporary solutions to alleviate scaling pains on a system with improper incentives.

The solution is to rethink peer review and rebuild the conference from the ground up. But this wont happen, since the career incentives are not there, and this is another argument for another time.

Best of luck with rebuttals

*results may vary

[Andrew Tran @Andrxwtran]

bring back cable cars

[Andrew Tran @Andrxwtran]

me checking my NeurIPS reviews while at a coffee shop was one of the first things I did in SF it’s seriously GGs

[Andrew Tran @Andrxwtran]

on my way to SF

[Andrew Tran @Andrxwtran]

this is actually insane and there’s like 10 more clips like this

[AB⚕ @AbsoluteBruno]

He should definitely be banned for this, WTF!!!
https://t.co/MN5cL4aaOu

[Andrew Tran @Andrxwtran]

SPAINNN 🇪🇸 🇪🇸 🇪🇸

[Andrew Tran @Andrxwtran]

That game had me STRESSED