Sam Cox

MTS @FutureHouseSF
PhD student @UofR

[Sam Rodriques @SGRodriques]

Today, we’re announcing the first major discovery made by our AI Scientist with the lab in the loop: a promising new treatment for dry AMD, a major cause of blindness.

Our agents generated the hypotheses, designed the experiments, analyzed the data, iterated, even made figures for the paper. The resulting manuscript is a first-of-a-kind in the natural sciences, in which everything that needed to be done to write the paper was done by AI agents, apart from actually conducting the physical experiments in the lab and writing the final manuscript. We are also introducing Robin, the first multi-agent system that fully automates the in-silico components of scientific discovery, which made this discovery. This is the first time that we are aware of that hypothesis generation, experimentation, and data analysis have been joined up in closed loop, and is the beginning of a massive acceleration in the pace of scientific discovery that will be driven by these agents. We will be open-sourcing the code and data next week.

Robin is a multi-agent system that uses Crow, Falcon, and Finch, the agents on our platform, to generate novel hypotheses, plan experiments, and analyze data. We asked Robin to find a new treatment for dry age-related macular degeneration. Robin considered the disease mechanisms associated with dry AMD, proposed a specific experimental assay that could be used to evaluate hypotheses in the wet lab, and proposed specific molecules we could test in that assay. We tested the molecules and gave it the resulting data, which it analyzed before proposing more experiments. In the end, it identified Ripasudil, a Rho Kinase inhibitor (ROCK inhibitor) that is approved in Japan for several other diseases, which seems very promising as potential treatment for dry AMD. It also identified specific molecular mechanisms that might underlie the effects of Ripasudil in RPE cells, from an RNA sequencing experiment it proposed. To be clear, no one has proposed using ROCK inhibitors to treat dry AMD in the literature before, as far as we can find, and I think it would have been very difficult for us to come up with this hypothesis without the agents. We have also run the proposed treatment by several experts in AMD, who confirm that it is interesting and novel. Moreover, this project was fast: with Robin in hand, the entire project took about 10 weeks, which is way shorter than it would have taken if we had been doing all of the in-silico components ourselves.

Important caveats: We are real biologists at FutureHouse, so I want to be clear that although the discovery here is exciting, we are not claiming that we have cured dry AMD. Fully validating this hypothesis as a treatment for dry AMD will take human trials, which will take much longer. Also, this discovery is cool, but it is not yet a "move 37"-style discovery. At the current rate of progress, I'm sure we will get to that level soon.

Congratulations to the team. Congratulations in particular to Robin, which generated the hypotheses, proposed the experiments, analyzed the data and generated the figures. And major congratulations also to the human team, which built Robin: @MichaelaThinks, @agreeb66, @benjamin0chang, @ludomitch, Mo Razzak, Kiki Szostkiewicz, and Angela Yiu.

[FutureHouse @FutureHouseSF]

Today, we're announcing BixBench, built in collaboration with @SciMac - a benchmark for AI agents tackling real bioinformatics tasks. We've created 53 scenarios with 296 questions that test how AI handles computational biology challenges. BixBench includes evaluation metrics and an open-source environment for LLMs to execute these tasks. #AIinScience

[FutureHouse @FutureHouseSF]

Today, FutureHouse is launching the FutureHouse Platform, bringing the first-ever superintelligent scientific AI agents to scientists everywhere via a web interface and API. The Platform is launching with four agents, each with their own specialization:

[Sam Rodriques @SGRodriques]

Today, we are launching the first publicly available AI Scientist, via the FutureHouse Platform.

Our AI Scientist agents can perform a wide variety of scientific tasks better than humans. By chaining them together, we've already started to discover new biology really fast. With the platform, we are bringing these capabilities to the wider community. Watch our long-form video, in the comments below, to learn more about how the platform works and how you can use it to make new discoveries, and go to our website or see the comments below to access the platform.

We are releasing three superhuman AI Scientist agents today, each with their own specialization:

A general-purpose agent (Crow);

An agent to automate literature reviews (Falcon); and

An agent to answer the question “Has anyone done X before” (Owl).

We are also releasing an experimental agent, Phoenix, that has access to a wide variety of tools for planning experiments in chemistry. More on that below.

The three literature search agents (Crow, Falcon, and Owl) have benchmarked superhuman performance. They also have access to a large corpus of full scientific texts, which means that you can ask them more detailed questions about experimental protocols and study limitations that general-purpose web search agents, which usually only have access to abstracts, might miss. Our agents also use a variety of factors to distinguish source quality, so that they don’t end up relying on low-quality papers or pop-science sources. Finally, and critically, we have an API, which is intended to allow researchers to integrate our agents into their workflows.

Phoenix is an experimental project we put together recently just to demonstrate what can happen if you give the agents access to lots of scientific tools. It is not better than humans at planning experiments yet, and it makes a lot more mistakes than Crow, Falcon, or Owl. We want to see all the ways you can break it!

The agents we are releasing today cannot yet do all (or even most!) aspects of scientific research autonomously. However, as we show in the video, you can already use them to generate and evaluate new hypotheses and plan new experiments way faster than before. Internally, we also have dedicated agents for data analysis, hypothesis generation, protein engineering, and more, and we plan to launch these on the platform in the coming months as well. Within a year or two, it is easy to imagine that the vast majority of desk work that scientists do today will be accelerated with the help of AI agents like the ones we are releasing today.

The platform is currently free-to-use. Over time, depending on how people use it, we may implement pricing plans. If you want higher rate limits, especially for research projects, get in touch. @m_skarlinski, @andrewwhite01, @_tnadolski, Remo Storni, @semajazarb, @ludomitch, @MichaelaThinks, as well as @jasonjoyride and his team for making such fantastic videos of us!

[FutureHouse @FutureHouseSF]

We are launching our FutureHouse Platform today! Our platform gives researchers public access to FutureHouse AI Scientist Agents for the first time. Check it out, at our website, and https://t.co/BlavgMdyrc.

[FutureHouse @FutureHouseSF]

We are launching a closed beta today for our data analysis agent. We are looking for extremely talented bioinformaticians and computational biologists to help us test it. Sign up here: https://t.co/DAuOOJYHGk

[Sam Rodriques @SGRodriques]

The next frontier for AI Agents in Science will be data analysis. Today, we're releasing BixBench, the most sophisticated benchmark yet for data analysis in biology. Agents that can do these tasks will be powerful tools for discovery. So far, they're not even close.

[Andrew White 🐦‍⬛ @andrewwhite01]

Molecular dynamics requires a lot of expert knowledge to set-up and analyze simulations. We set out to automate it with LLM agents: MDCrow!

[Andrew White 🐦‍⬛ @andrewwhite01]

Finishing 2024 with one more research result! We’ve trained small language agents to do hard sci tasks: engineering proteins, manipulating DNA, and working with sci literature in a new library called Aviary. We beat humans and frontier LLMs on these tasks!

[Sam Cox @SamCox822]

@andrewwhite01 we really need less data

[Marta Skreta @martoskreto]

(🧵1/3) presenting 2 posters at #ICML2024 @AI_for_Science in‼️5 mins‼️on

(1) improving protein function representations
(2) sample-efficient molecule generation

more info in comments below -- come by and say hi! 🤗

[Kevin M Jablonka @kmjablonka]

If you want to extract data using LLMs, you should check out our review

📝 arXiv https://t.co/y0gwlf6SPg
💻 hands-on online book https://t.co/V5148DFlOI

[FutureHouse @FutureHouseSF]

This is probably one of the best opportunities for a junior bio researcher right now who wants to learn how the wet lab and AI will interact in the future of AI-powered science. If that sounds like you, get in touch!

[FutureHouse @FutureHouseSF]

LAB-Bench, our new benchmark for language models and agents on scientific research tasks in biology, is available today on HF here: https://t.co/7fFO49gnZc

[Andrew White 🐦‍⬛ @andrewwhite01]

We released today ~2,500 hard bio evals that test lab protocol design, scientific literature RAG, sequence design, figure understanding, table understanding, and more. We benchmarked against >10 independent PhD-level experts. Frontier LLMs do not exceed humans...yet 1/2

[FutureHouse @FutureHouseSF]

ChemCrow was one of the first serious demonstrations of using AI to automate science. There will be many more to come. Major congratulations to the team: @SamCox822 @CarloBalda97 @OSchilter @CarloBalda97 @andrewwhite01 @pschwllr

[Andrew White 🐦‍⬛ @andrewwhite01]

ChemCrow is out today in @NatMachIntell! ChemCrow is an agent that uses chem tools and a cloud-based robotic lab for open-ended chem tasks. It’s been a journey to get to publication and I’d like to share some history about it. It started back in 2022. 1/8

[Sam Cox @SamCox822]

sign up link -> https://t.co/OWUIwtGv0W

[Sam Cox @SamCox822]

Live in 15 minutes! Catch this talk from Dr. Adrienne Hoarfrost on AI + Microbial Dark Matter by signing up below

[FutureHouse @FutureHouseSF]

Sign up now! Less than an hour until we're live

[Sam Cox @SamCox822]

@jakublala the chances of me calling to schedule anything is about 0%