Shawn Higdon

[Kenny Workman @kenbwork]

AI Agents will be core infrastructure for genomic surveillance.

We introduce BioSecBench-Surveillance, a verifiable benchmark for testing whether AI agents can make the analytical decisions required in these workflows.

The benchmark contains 100 evaluations spanning seven task categories, six sample types, and both short- and long-read sequencing. Agents receive realistic sequencing data and sparse surveillance context, then must choose the right tools, references, thresholds, and analysis paths.

As sequencing volumes increase, genomic surveillance is increasingly limited by analysis. Public health workflows depend on bespoke and sometimes tacit choices with sequence references, databases, filters, normalization, and thresholds.

AI agents are promising because they can inspect files, run tools, and iterate through workflows autonomously. But surveillance is a challenging problem. Agents must chain complex scientific and analysis decisions correctly from messy biological context.

We evaluated sixteen model-harness configurations across roughly 4,800 runs. Pass rates ranged from about 14% to 50%, with most frontier configurations clustered between 38% and 50%. Refusals varied sharply by harness and provider, from zero to nearly one-third of tasks.

Performance varied more by task type and sequencing technology than by sample type or assay. Most task categories landed between 35% and 50%, but anomaly detection fell to 20%, with genetic-engineering characterization next at 35%.

Long-read datasets were also harder, scoring 26% versus 41% for short-read datasets. Sample type, nucleic-acid target, and assay type moved performance much less: clinical and isolate samples were handled best, wastewater was somewhat worse, DNA and RNA differed only modestly, and shotgun and targeted assays were nearly identical.

Agents usually found reasonable tools, but struggled with scientific judgment. The failures came from choices around how to invoke those tools in context, eg. selecting the wrong reference, threshold, normalization method, or final interpretation of biological signal.

The hardest tasks were open-ended judgement calls where the agent had to decide what mattered without being told what target to look for. Anomaly detection requires deciding whether a weak signal was real or background. Genetic-engineering characterization required deciding whether a sequence pattern reflected deliberate construction rather than native or homologous biology.

We are building toward a future where agents analyze surveillance data as it arrives, fast enough to shape an outbreak response while it still matters. Today’s agents might not be reliable enough to do so, but by measuring their capabilities, we get closer to this future.

[Oliur @UltraLinx]

Can you read 900 words per minute? Try it.

[Taylor Reiter @ReiterTaylor]

Introducing 🎉sourmashconsumr 🎉, an R package that reads, parses, plots, and analyzes the outputs of the sourmash Python package.
Pub: https://t.co/xuE9I4uPND
GitHub repo: https://t.co/baYEEKWqYW
Documentation site: https://t.co/Y0mZnWngBV
🧵 [1/8]

[Taylor Reiter @ReiterTaylor]

sourmashconsumr does lots of things, but here's a sampling of the visualizations you can make.
[2/8]

[Shawn Higdon @Smiles_Higdon]

@ReiterTaylor Sweet, I’m looking forward to checking this out!

[Mwaura Kahiga Livingstone Nganga @Nganga_Mwaura_N]

Excited for the future of synthetic apomixis. #plantbiotech #agtech
https://t.co/TmFgsKD6Xn

[Steven Salzberg 💙💛 @StevenSalzberg1]

Metagenomics folks: an explainer of how the new release of KrakenUniq can run on your laptop, even with a 400GB database, thanks to very elegant engineering by Christopher Pockrandt. See our @biorxivpreprint for more 1/9 https://t.co/hW3JNntsXx

[WeimerMicroLab @BartWeimersLab]

Very happy to share a collaborative effort with @TheHessLab with the lead author being #ClaireShaw that details host cues that alter pathogen virulence. Check it out!

https://t.co/5MVbA9tD6a

[𝕐 @nomad421]

Recently the paper describing Cuttlefish 2 (led by @scarecrow00007 and in collaboration with @marekkoki and @sdeorowicz) was published in @GenomeBiology (https://t.co/RFvxS60LbU). We're quite excited about Cuttlefish 2 and what it enables. (short 🧵) 1/

[Chenxin Li, PhD (@chenxinli2.bsky.social) @ChenxinLi2]

Can I get a few more followers to get to 2000? Thanks.

[Taylor Reiter @ReiterTaylor]

🎉 New tutorial 🎉 Ever wanted to go from raw metagenome reads to a handy taxonomic table like the ones in phyloseq or speedyseq? Sourmash gather & sourmash tax get you there quickly and accurately 🏃‍♀️ 💯 https://t.co/2ySJP72it9

[Matthew Clark @Matthew_BaTiSiO]

Flagellar motor of Salmonella bacteria, in complex with the the MotAB stators (Pink). MotA is a pentameric H+ channel that couples the electrochemical gradient into rotational energy. MotB dimers anchor the stator into the peptidoglycan layer. #Blender3d #b3d #sciart

[Javier Santoyo @jsantoyo]

CAMI II: identifying best practices and issues for metagenomics software. #Metagenomics #BestPractices #Bioinformatics
https://t.co/kwza1MyjO1 @naturemethods

[Dr. Silas Kieser @SilasKieser]

Now we can google protein function https://t.co/HY5cSyRc9T

[bioRxiv Plant Bio @biorxiv_plants]

A novel cis-element enabled bacterial uptake by plant cells https://t.co/5J35c7wdFG #biorxiv_plants

[Tania Pozzo @pozzo_tania]

Building a scientific movement in developing countries #iGEMBolivia decided to learn #synbio to solve its local problems by participating in #iGEM2021. Congratulations @igemBolivia!!🙌🤩💪🏼🧬🇧🇴

[Anthony Pompliano 🌪 @APompliano]

Inflation or recession.

Pick your poison.

[Hadeer Elhabashy @HadeerElhabashy]

I made this figure out of my fascination with protein architectures. I call it "Molecular Machinery of Life." Thought to share it with you to enjoy your eyes. Can you detect, by heart, the Nobel Prize-Winning structures? @AcademicChatter #AcademicTwitter

[Shawn Higdon @Smiles_Higdon]

@ejtibbett Congrats Dr!