Andrew Liu

Biosecurity at @OpenAI | prev. @SecureBio, @Harvard PhD

[Andrew Liu @andrew2liu]

4. This builds upon OpenAI's HealthBench, a set of 5,000 health questions and rubrics written by 262 doctors. Some questions were based on real Google search queries and some were physician-written or -guided. Thanks to OpenAI for making this benchmark open source!

[Andrew Liu @andrew2liu]

As a side project, I asked GPT, Claude and Gemini to answer 2,800 user health questions and graded them with real doctor-written rubrics: https://t.co/IeAU1e6Rsg. Here's what I found:

1. GPT-5.6 Sol and Claude Opus 5 did best, getting 52-58% of information correct.

[Andrew Liu @andrew2liu]

3. The website allows you to search your specific issue to compare model answers on it. Here's me searching for "shoulder" (inspired by my volleyball shoulder injuries); Opus 5 is slightly more trustworthy than GPT-5.6 Sol according to the physician rubrics.

[Andrew Liu @andrew2liu]

Thanks to @_El_Marshall_ @seth_donoughe @JasperGeh
and Ben Mueller!

[Andrew Liu @andrew2liu]

Happy to share @SecureBio review of @AnthropicAI 's unredacted chemical and biological risk report! https://t.co/EngNFzsoQ4

[SecureBio @SecureBio]

@AnthropicAI collaborated with us on a review of the unredacted chemical and biological (CB) risk sections of their February 2026 Risk Report. After months of followup and hundreds of pages reviewed, we share our external review below.

[Andrew Liu @andrew2liu]

Anthropic shared their unredacted report and 110 pages of material. Over two months, they responded to 70+ follow-ups on model capabilities and safeguards. We also ran tests of the refusal classifiers. We used these to assess the chem and bio risk of Opus 4.6 and similar models.

[Andrew Liu @andrew2liu]

@seth_donoughe @_harm0n

[Andrew Liu @andrew2liu]

Happy to share that ABC-Bench was accepted to ICML!

When we benchmark LLM agents on biosecurity tasks, *all agents beat the median human expert.* In wet lab, o4-mini wrote scripts that assembled DNA on an OT robot.

Blog: https://t.co/pX5STRAg0u
Paper: https://t.co/U7arUcqhkU

[SecureBio @SecureBio]

We’ve seen LLMs beat expert virologists in multiple-choice tests.

The question now is: can they use that knowledge in practice?

Our new assessment, ABC-Bench, tests LLMs on real lab work and biological problem-solving.

[Andrew Liu @andrew2liu]

Thanks to co-authors Samira Nedungadi, Bryce Cai, Alex Kleinman, Harmon Bhasin, and @seth_donoughe !

[Michael Lai @Mtclai]

I've lived in SF for 8 years, and I've always wondered —

Why are there more dogs (200K) than kids (115K)!?

I love dogs (👇🏻 is my family dog Luna), but my mission is unlocking child potential.

🧵 with 6 hypotheses on why there are so few kids in SF: 1/35

[Andrew Yang🧢⬆️🇺🇸 @AndrewYang]

Great news #yanggang - Evelyn and I are moving to Georgia to help @ossoff and @ReverendWarnock win! This is our only chance to clear Mitch out of the way and help Joe and Kamala get things done in the next 4 years. More details to come but let’s go!!! 😀🇺🇸🚀

[Andrew Liu @andrew2liu]

Great writeup by my PhD classmate Doug!

[Douglas Yao @DouglasYaoDY]

We previously showed that only 11% of disease heritability is mediated by gene expression from eQTL studies. Curious about how we actually get this number? I've written up a (mostly) non-technical blog post that explains the method: https://t.co/WmKoEQWwz5

[Andrew Liu @andrew2liu]

I've already mailed in my ballot, but if I didn't have a mail-in ballot, I would vote early on say Monday, Aug 24, to avoid the crowds and COVID19. My voting location is open for 8 hours that day, which should be enough to spread out the crowds and socially distance.

[Andrew Liu @andrew2liu]

tl;dr: If you're in MA, I urge you to consider voting for Ed Markey (endorsed by Warren, AOC, etc.) in the Dem Senate primary. 5k votes could change the result! Mail your ballot now or vote Aug 22-28 at your location: https://t.co/W9E18Rznrg.

Why Markey?

[Andrew Liu @andrew2liu]

If you've already requested a mail-in ballot, mail it out now! Otherwise you can vote early in person Aug 22-28 or on Election Day on Sep 1. Find your voting location here: https://t.co/W9E18Rznrg.