David Shih

Computational Biologist

[David Shih @djhshih]

@smead2 @sokrypton This depends on the prompt that is used. Reviewers can use AI to generate ignorant and spiteful reviews that are detailed and specific. In the old days, at least the bad reviewers had to put effort into writing detailed comments.

[David Shih @djhshih]

@bots_and_bits @sokrypton I've had a paid reviewer use AI to review my grant. This paid reviewer gave nonsensical comments that are highly technical and specific, and the nonsensical comments were the same as what Claude Haiku gave.

[Sergey Ovchinnikov @sokrypton]

Got reviews back for a manuscript today, completely AI generated. 🫤

How should I respond?

[David Shih @djhshih]

@CSProfKGD I automate grading now using AUROC. I also use criteria related to commit history and code novelty, but students just do many "refactor" commits. At least I don't have to tell them their AI-generated code is bad, and I just focus on ramping up the problem difficulty.

[David Shih @djhshih]

@CSProfKGD Higher weighting for exams. It is now difficult to assess problem solving skills using assignments. On basic problems, everyone uses AI to get them right. On hard problems, no one gets them right using AI, and they haven't learned to solve problems by thinking deeply.

[David Shih @djhshih]

Update: Sigh. I ran my model through Claude and it gave the same nonsensical comments on the math. I never consented to having my confidential grant application uploaded to a website that is known for training on copyrighted material during review. This is academic misconduct!

[David Shih @djhshih]

How do you deal with grant reviewers who mistaken sound math as "fatal flaws"?

I just received the reviewers' comments back for my grant. First reviewer gave perfect scores. Second reviewer criticized my model for two "fatal flaws" and accuses me of being "confused." 1/

[David Shih @djhshih]

I presume that the reviewer panel was too busy to look at the technical details and assumed that the paid reviewer wouldn't make such elementary mistakes and wrongly accuse the author of being "confused."

Any advice on how to deal with this situation?

[David Shih @djhshih]

I guess I should be grateful that the reviewer provided two specific comments, which revealed that he/she doesn't understand basic statistical modelling.

5/

[David Shih @djhshih]

@lpachter This seems vaguely similar to log TPM (sample-centered). I imagine it would discard any cell-specific overall transcript abundance information, which contains both biological signal and technical noise.

[Jorge Bravo Abad @bravo_abad]

No scaling laws for single-cell foundation models: when bigger atlases stop teaching the model anything

In language and vision, the recipe has been simple: more data, bigger models, better performance. Single-cell biology borrowed that playbook. Foundation models for transcriptomics jumped from 1 million cells to atlases of over 100 million, on the assumption that scale would unlock the same gains. Alan DenAdel and coauthors put that assumption to the test, and the result is sobering.

Working from a 22.2-million-cell corpus, they pretrained 400 models across five architectures (from PCA and a variational autoencoder up to the Geneformer transformer) and ran 6,400 evaluation experiments. They varied not just dataset size (1% to 75%) but also diversity, using cell-type re-weighting and geometric sketching to deliberately enrich rare cell types and transcriptional states.

The finding: performance saturates almost immediately. On cell-type classification, batch integration, and perturbation prediction, most models hit their ceiling at roughly 1% of the corpus, about 200,000 cells. Beyond that, adding millions more cells changed essentially nothing. More diversity didn't help. Even spiking in genome-scale Perturb-seq data, to give the models perturbed phenotypes rather than just healthy ones, failed to move the needle. Larger models did score better overall, but they too plateaued early on data.

Two points stood out. Simple baselines (PCA, logistic regression) often matched or beat the transformers. And the strongest model, SCimilarity, won not because of size but because its contrastive training objective is aligned with the downstream task. For single-cell data, what you train on and how you frame the objective matters far more than how much you collect.

This reframes a quiet but expensive habit. In drug discovery, biotech, and any pipeline leaning on cell atlases, the instinct to keep scaling pretraining corpora may be burning compute for no return. The real leverage sits elsewhere: curating high-quality, task-relevant data and matching the training objective to the actual question you're trying to answer.

Paper: DenAdel et al., journal license | https://t.co/X7GxoxF5U5

[Sholto David @addictedtoigno1]

I am sending an open letter to Thermo Fisher. Their response to my response to their manipulated western blot is bullying and petty. Yes, this western blot really is manipulated, it is unfair on me to say otherwise. I don't make those accusations lightly. #ThermoFishy

[Rahul Satija @satijalab]

Our 10th Single Cell Genomics Day is next Friday (6/12)!

Thanks to amazing speakers Aviv Regev @xinjin @anshulkundaje @junyue_cao and many more! Talks are live-streamed on YouTube and are free (no registration required) at https://t.co/G5Pq3EwyHF

[Sholto David @addictedtoigno1]

Earlier this week I posted an example of a fake western blot provided by ThermoFisher to demonstrate the validity of a p53 antibody. I considered it an amusing curiosity. In fact ThermoFisher has systematically manipulated antibody validation data. Short Thread... 🧵

[Rafael Irizarry @rafalab]

A flaw in Person-Δ may be overstating progress in single-cell perturbation prediction models. Pearson warned about this in the 19th century: reusing the same controls induces spurious correlation. Split the controls, and much of the claimed prediction power fades. Link below 👇

[Sholto David @addictedtoigno1]

Surprised to discover that Thermo Fisher appears to show a fake western blot for the validation of one of their p53 antibodies. I've added a diagram to show the very similar bands. This does not appear to be one of the "published figures", but their own internal data.

[David Shih @djhshih]

@JFPuget Thanks for sharing this. It's very helpful!