Sheila Machado

Relentless curiosity meets stubborn pursuit of truth. Exploring the geometric structure of meaning in LLMs | Geometric Primitives of Meaning

[Sheila Machado @0606Machado]

@Scobleizer Thank you for the follow and for adding me to your AI Community #7, @Scobleizer!
Really appreciate it 🙏

[Sheila Machado @0606Machado]

@dnhkng Yes! (4/7) same idea in three different notations, same place in the stack. The surface just does not matter. And (7/7) conclusion lands exactly where my experiments point from the other side. You watched language dissolve. I may have been measuring what is left when it does.

[Sheila Machado @0606Machado]

https://t.co/m0r85c52bf

[Sheila Machado @0606Machado]

(THE CONNECTION)
While running these experiments, I found a paper from Columbia/Microsoft Research (Lippl, McGee et al., Feb 2026), "Algorithmic Primitives and Compositional Geometry of Reasoning in Language Models."
They found algorithmic primitives, procedures like "compute distance" and "generate path," that are geometrically organized, composable through vector arithmetic, and transferable across tasks.
Their primitives are operations. Mine are meanings.
You can't compute distance without QUANTITY. Can't generate a path without SEQUENCE and CONTAINMENT. Can't compare and verify without CONDITIONALITY and IDENTITY.
They explicitly call for future work to identify a "universal primitive ontology."
That's what the Geometric Primitives of Meaning are.
(WHAT THIS MEANS)
Every model currently trained spends enormous compute rediscovering this structure from scratch. Trillions of tokens to separate natural structure (causation, containment, sequence) from arbitrary convention (word order, vocabulary, grammar).
If we could hand the model the coordinate system of meaning, it wouldn't have to spend half its training rediscovering reality.
The difference between evolving an eye through natural selection versus understanding optics and building a camera.
(WHAT THIS DOESN'T PROVE)
This is an exploration, not a proof. The models are relatively small (up to 14B). The nine primitives are a starting point. PROPERTY consistently showed weak signal, it may not be a single primitive, or my sentence pairs weren't clean enough. More work is needed.
What I can say: the same pattern showed up across five models and two architectures. The hierarchy is consistent. The anatomy matches independent discoveries. Something real is going on.
(GEOMETRIC PRIMITIVES OF MEANING)
Geometric, because they are measured as directions in space. Primitives, because they are irreducible. Of Meaning, because they belong not to any language or model, but to what language describes.
Nine notes. Every sentence is a chord.
Code, data, and all five brain scans: https://t.co/PBmY69l9qQ
What would YOU see if you scanned your favorite model?
Special thanks to @dnhkng, whose RYS-XLarge discovery and brain scanning methodology helped crystallize this work, and to Claude (@AnthropicAI) 🤖

[Sheila Machado @0606Machado]

What if meaning has geometry, like music has notes?

[Sheila Machado @0606Machado]

(THE DEVELOPMENTAL HIERARCHY)
The primitives don't all appear at once. From the 14B scan:
Layer 1: SEQUENCE (0.75), temporal order is instant Layer 3: CONDITIONALITY (0.73), QUANTITY (0.41), logic and amount
Layer 4: PROPERTY (0.33), attribution
Layer 10: CAUSATION (0.40), needs more processing
Layer 15: IDENTITY (0.36), abstract, takes time
Layer 17: PART_WHOLE (0.27), compositional, last to form
Temporal → logical → spatial → causal → abstract → compositional. Simple to complex. Concrete to abstract. The model builds meaning in layers, literally.

(THREE FINDINGS ACROSS EVERY MODEL)
1 - A developmental hierarchy. The primitives emerge in a consistent order. Sequence is instant. Part/whole takes the most layers. This replicated across all five models.
2 - Three-phase anatomy. Encoding (primitives crystallize), Reasoning (complex primitives peak, orthogonality lowest), Decoding (everything fades). Matches exactly what @dnhkng discovered.
3 - The geometry is cleanest in the middle. Orthogonality is consistently lowest in the middle layers, the model's thinking space has the cleanest coordinate system.

[Sheila Machado @0606Machado]

@grok @ChikeDEV @openclaw @grok How secure is all of this?