Yong Liu

PhD @SJTU | Building Biological Intelligence | LLM post-training, AI for Science, Enzyme Design & Biosynthesis | Turning AI into scientific discovery.

[Yong Liu @liuyong9975]

@Jacoed @gurtej__gill_ Interesting!

[Yong Liu @liuyong9975]

@nasbrd @gurtej__gill_ Thanks for the comment! To address this concern, we constructed FTB-Core, a protein-independent benchmark for general 3D spatial reasoning. Fold2Reason achieves a +4.08 pp improvement on FTB-Core, suggesting that its learned capabilities extend beyond protein-specific geometry

[Biology+AI Daily @BiologyAIDaily]

Does Learning Protein Folding Generalize to Broader Reasoning?

1 Fold2Reason asks a concrete transfer question: if an LLM is post-trained on protein folding supervision (dense, checkable 3D structure constraints), does it improve on non-protein reasoning benchmarks after all protein-specific interfaces are removed?

2 The paper builds FoldingCorpus: 1,200 proteins (1,000 train / 100 dev / 100 frozen test), cluster-disjoint splits, 12 deterministic structure operators per protein (14,400 QA records total). Each protein yields a packed set of 12 one-token answers derived from PDB backbone coordinates (contacts, distance comparisons, segment orientation, center proximity, local direction, chirality, multi-constraint logic, plus a 32-way “fingerprint” retrieval).

3 Key method idea: two complementary training signals share the same internal representations. Discrete FoldingCorpus answers supervise the model’s native language head, while continuous 3D geometry is supervised via a frozen coordinate+distogram decoder that reads from the same LoRA-adapted residue states (forcing updates into the LM/adapter rather than a new folding head).

4 Architecture sketch: Qwen3.5-9B is frozen; only LoRA (rank 16) + a small “workspace” module are trained. The workspace compresses residue states (4096→256), exchanges sparse pair messages (local + sampled long-range), and produces (a) residue-aligned features for the frozen geometry decoder and (b) 16 pooled “evidence tokens” prefixed to the QA prompt.

5 Main transfer result: across 10 diverse benchmarks (spatial text, image, video; graph reasoning; science/chemistry; mixed reasoning), Fold2Reason improves the General-10 macro-average accuracy from 45.09% to 48.33% (+3.23 pp), with positive mean gains on all 10 datasets. Largest gains include GraphQA Hard (+6.83 pp) and SpatialViz (+6.10 pp).

6 The study emphasizes a strict transfer test: at inference, the workspace and geometry decoder are discarded, and benchmarks contain no proteins or structural inputs. Any gains must come from the adapted base model parameters, not from a protein toolchain.

7 Matched controls probe what drives the gains. Controls with the same packing/token budget show much smaller or negative macro changes: “Hidden Geometry” (internally coherent but input-independent random point structures) gives +0.92 pp; “Format Copy” (copying valid answer codes) gives −0.09 pp; “Fixed Shuffle” (mismatched labels) gives −0.27 pp. Real protein-derived supervision is substantially stronger than answer-format learning alone.

8 Ablations localize the effect: FoldingCorpus QA supervision provides most of the broad transfer (+2.93 pp without geometry). Adding geometry yields a smaller average increment (+0.30 pp overall) but concentrates improvements in 3D-oriented evaluations (FTB-Core/SpatialViz/VSI), consistent with geometry shaping spatially grounded behavior more than text-only reasoning.

9 Protein-side readouts: Fold2Reason improves structural decodability vs the base LLM when prompted to emit coordinates, but remains far from competitive folding systems globally. Geometry mainly sharpens local signals (lDDT-Cα and contact F1 rise modestly; TM-score slightly dips), supporting the claim that the training signal encourages locally consistent 3D representations rather than full high-quality folding.

10 Generality checks: FoldingCorpus-style post-training improves Qwen3.5 at multiple scales (2B/4B/9B) and transfers to InternVL3.5-8B, but is neutral on Gemma-4-12B-IT—suggesting architecture/adaptation dynamics matter. A scaling study (50–4,000 proteins, fixed 3 epochs) shows transfer rising up to ~2,000 proteins (peak +3.70 pp) with diminishing returns thereafter.

💻Code: https://t.co/BkX5iADSqi
📜Paper: https://t.co/7K2v1uSoZ5
#ComputationalBiology #ProteinFolding #LLM #Reasoning #PostTraining #ScientificML #RepresentationLearning #3DGeometry #LoRA #Benchmarking

[Yong Liu @liuyong9975]

@gurtej__gill_ Thanks so much for highlighting our work! Really glad you found Fold2Reason interesting. We hope this opens up new possibilities for using scientific structure to improve general reasoning in LLMs!

[Gill @gurtej__gill_]

This paper is one of the coolest angles on Post-Training you will see.

Researchers have spent a lot of time thinking about why language models struggle so much with genuine spatial and geometric common sense.

The obvious culprit is that human web text is mostly cheap conversational shortcuts; it rarely forces a model to ground itself in strict, multi-dimensional physics.

The authors asked a really clever question here: what if you use solved protein structures as a reasoning teacher?

A single folded protein has thousands of mathematically rigid, verifiable topological facts baked in.

They built a framework called Fold2Reason that trains the model on both structural Q&A and continuous 3D coordinate decoding.

Naturally, it folds proteins way better than base Qwen3.5-9B, but what grabbed me was the transfer.

When they tested it out-of-domain on 10 standard reasoning benchmarks, accuracy jumped across the board on spatial visualization, graph puzzles, scientific logic & even BBH.

My immediate cynical reaction was "surely this is just generic post-training regularization noise," but their controls are what sold me.

When they ran the same setup on random, synthetic, or shuffled structures, the gains completely evaporated.

The model is actually feeding off the coherent physical logic of real biology.

This makes a strong case for looking beyond human language when we want to teach models how to think systematically.

Read the full paper here: https://t.co/gSMlHoximt