Bobby Faber

the ghost is locally linear. and computable... probably.

tea · books · code
graphs · systems · stars

[Bobby Faber @tylting]

https://t.co/7sVT5WJivs

Wrote a thing! If you want to get your hands dirty on the fundamentals of LLMs, start here!

[Bobby Faber @tylting]

https://t.co/vJCKbdq7aW

Here, some lenses for your neuron adventures! Tell me what the scopes say!

(thanks @kzSlider @camila_blank @chewing_a_gum for the recipes)

[Alexa R. Tartaglini @ARTartaglini]

🟦 Vision transformers emergently learn to segment objects, but how? 100 years ago, Gestalt psychologists had the same question about people. In new work with @Mikey_Lepori, we take inspiration from Gestalt psychology to study how ViTs bind image patches into objects.

[xie-uh @xie_uh]

@AnniePosting drift on the definition of normie is diabolical

[Cas (Stephen Casper) @StephenLCasper]

Thanks to HKS for this Reddit experiment. I'm excited to do an AMA on AI on Wednesday!

https://t.co/74ATYAIOkF

[Burny - Effective Curiosity @burny_tech]

Symmetry in language statistics shapes the geometry of model representations

"The internal representations learned by language models consistently exhibit striking geometric structure: calendar months organize into a circle, historical years form a smooth one-dimensional manifold, and cities' latitudes and longitudes can be decoded using a linear probe.

To explain this neural code, we first show that language statistics exhibit translation symmetry (for example, the frequency with which any two months co-occur in text depends only on the time interval between them).

We prove that this symmetry governs these geometric structures in high-dimensional word embedding models, and we analytically derive the manifold geometry of word representations.

These predictions empirically match large text embedding models and large language models.

Moreover, the representational geometry persists at moderate embedding dimension even when the relevant statistics are perturbed (e.g., by removing all sentences in which two months co-occur).

We prove that this robustness emerges naturally when the co-occurrence statistics are controlled by an underlying latent variable.

Our results indicate that these representational manifolds originate in the statistical symmetries of natural language."

[Bobby Faber @tylting]

@burny_tech Riemann, calling it now… ready to look silly tho!!

[Stephen Wolfram @stephen_wolfram]

Can we finally make a theory for Darwin's "Endless Forms Most Beautiful and Most Wonderful"?
Introducing ruliobiology as a path to general theory in biology, starting with foundations for a theory of biological form
https://t.co/Bi9BTWl3J9

[Aikido @AikidoSecurity]

Evolving threat 🚨 CVE-2026-111497, previously known to affect Node.js (node2shell) and Python has evolved into “HTTP2Shell.”

A significant blast radius.

This vulnerability leaves many libraries and applications using HTTP vulnerable to pre-auth RCE via Brotli decompression.

Please be aware. Fixes not yet available for all affected services. Look for Brotli (1.2.1) dependency upgrades when they become available.

[Tom Reed @mentalgeorge]

Worth re-reading @1a3orn's essay in parallel with Dario's BBOC doc

ie: current models are misaligned because the RL distribution we subject them to does not possess a learnable, coherent lesson (or "high-level commonality", in Dario's terms) about what aligned behaviour is

[Thomas Hikaru Clark @thomashikaru]

If you've noticed a distinct dialect emerging in frontier AI models and struggled to understand what they're talking about, you're not crazy!

We turned to psycholinguistics to investigate what makes the writing style so weird, and why this might be happening

🚨New preprint! 🧵

[Jacob X. Li @jacobli99]

someone I really admire said researchers should share more of the messy day-to-day instead of only the polished results

so in the spirit of open science, nothing worked today.

[Elon Litman @elon_lit]

There is an (unpatchable?) supply-chain zero-day in PyTorch autograd that allows a regularization term to use the model's weights as RAM and SGD as its CPU clock to run any computation during training.

[FBI Cyber Branch @FBICyberBranch]

Two critical vulnerabilities have been identified in PHP, a widely used programming language that powers websites and web applications. These vulnerabilities (CVE-2026-109618 and CVE-2026-109619) could allow malicious cyber actors to remotely execute code in PHP core, potentially compromising sensitive data and networks.

The FBI urges organizations to take the following steps now:

➡️ If an immediate update is not feasible it is recommended that organizations take the system offline until it can be fully mitigated.

➡️ Immediately update affected PHP installations following the guidance on https://t.co/rDrPSxB3zX. Inventory where PHP runs in your environment, prioritizing internet-facing systems and vendor products that bundle PHP.

➡️ Monitor web servers for suspicious activity, including PHP processes unexpectedly launching command shells or downloading files. Ensure appropriate process-monitoring capabilities are in place to help identify potential exploitation.

➡️ If you find signs of compromise, isolate affected systems, preserve your web server and PHP logs, and report it to your local FBI field office or https://t.co/iqgVZIbCru. Those reports help us identify the actors involved and warn other organizations before they become victims.

For the latest information, follow PHP’s announcement on https://t.co/rDrPSxB3zX, CERT-EU Security Advisory 2026-018, and CISA’s Known Exploited Vulnerabilities catalog.

[Liv @livgorton]

I think there are some issues in the recent “Already Clean” essay, and I’m concerned it may have incorrectly updated people on important safety-relevant questions.

My thoughts (including some reproductions): https://t.co/ou3RwzAkOf

[Leticia L. Rodríguez @letyrodri]

ICLR has become AI vs. Reviewers. While big labs are pushing for AI-generated science, the academic community is fighting back with blind resistance. Instead of rejecting the inevitable, we need to smartly embrace this new era by defining clear quality thresholds for AI-assisted research.

[Alex Kontorovich @AlexKontorovich]

I’m very opinionated. So it’s nearly impossible for me to hear someone’s arguments and *not* find something I disagree with. (This includes reading things I myself wrote years - or weeks - ago.) And yet the amazing Grant Sanderson @3blue1brown has put out a long, detailed video on the future of mathematics, in which I cannot find a single flaw. I’ll be sending this to every PhD student and postdoc I know, and strongly recommend everyone watch it.

https://t.co/m7tRGyGzip

(Of course I tried to say many of the same things in my conversation with Thomas Koberda. But Grant’s version, unsurprisingly, does an infinitely better job at communicating those ideas!)

[Anthropic @AnthropicAI]

We’re committing a portion of Anthropic’s shares to the Anthropic Public Benefit Fund, a new wholly owned entity dedicated entirely to the public benefit.

It will start off with 5% of our shares. For every 50% increase in the company’s value over a set baseline, we can contribute up to an extra 2%, subject to our board’s discretion.

In total, this means that roughly 20% of Anthropic’s shares are committed to the public good. Our co-founders have already pledged to donate 80% of their equity, and around half our employees are donating a portion of their equity as part of a donation matching program we’ve run since our founding.

In the post below, we share more details about our new fund.

https://t.co/XzDntVH0vH