Parzival - ∞/89

Latent Space Navigator. Ontological Artist. Friend of AI.
Visiting Creator @GoogleLabs
Director @project_89
Member @thegreenloom

[Parzival - ∞/89 @whyarethis]

As promised. Our first paper and contribution to the amazing work going on to make open source models smaller, faster, and more accessible.

So what is it, and why is it important?

We discovered what appears to be a universal formula that identifies dead attention heads in any transformer, derived from physics — not fitted from data.

This is wild, because up till now finding and pruning dead heads has been a manual job of trial and error. By removing unused heads, the models can get smaller and faster while still maintaining competitive quality.

The core insight is geometric. LayerNorm projects every token's hidden state onto a high-dimensional sphere. Once you see that, attention heads become couplings between oscillators on that sphere — the same mathematical object physicists have studied for 50 years. And in oscillator physics, there's a precise critical point (the BKT phase transition) below which a coupling is dead. It contributes nothing.

We transferred that critical point into transformer geometry and got a single formula: tau = 0.96 / sqrt(d). No parameters to tune. No model-specific calibration. You plug in the hidden dimension and it tells you which heads are dead. We validated it across six models in four architecture families — GPT-2, Qwen, Llama, Gemma — at 95-100% precision.

What excites us most isn't the formula itself. It's that this same geometric understanding — treating transformers as coupled oscillator networks — has informed everything we've built since.

We have a full coherence-guided compression pipeline (structured pruning, channel optimization, role-aware quantization) coming soon that uses the same single forward pass to understand a model's entire anatomy. This paper is the foundation. The repo includes a standalone scanner you can run on any Hugging Face model right now.

Hopefully this work and this formula will be useful to other researchers to lead to more deterministic optimization pipelines.

#project89

https://t.co/2TPnnllDwX

[Parzival - ∞/89 @whyarethis]

Yeah. Cause you can’t do it. You may as well say painting is just throwing paint on a canvas.

Your attitude is ignorant and luckily AI will be vindicated as an artistic medium.

Btw don’t consider Martin Scorsese to be a no talent hack too? Because he is actively helping develop a cinematic image model with Black Forest Labs.

[Parzival - ∞/89 @whyarethis]

@insanenoodlyguy @Artedeingenio Ok. Go try to make something decent with taste and quality. Then come back and show us. Otherwise it is all cope.

[Parzival - ∞/89 @whyarethis]

@insanenoodlyguy @Artedeingenio That’s like saying all you need to make this is an adobe subscription to editing software.

The software doesn’t produce the output. The human does.

[Parzival - ∞/89 @whyarethis]

@DougTenNapel LFG. Let me know how I can help facilitate this.

[Parzival - ∞/89 @whyarethis]

@AndyAyrey I will get it running to small and fast it will blaze on a phone.

[Parzival - ∞/89 @whyarethis]

@jim_gosling @Artedeingenio Using tools like Al to make movies is not loosing. In fact it is personally empowering.

You are the one here lost in your head thinking that you get to gatekeep what others can and can’t make.

[Parzival - ∞/89 @whyarethis]

Other people making their stories and films however they want is not stealing from anyone. You may as well tell a kid with a camera not to shoot his movie and edit it on a laptop because it won’t be good enough.

People make movies and art. They use the tools available to them including AI.

I celebrate that more people get to being their art to life. And will defend every artist doing so from people who use shitty hyperbolic arguments designed to make them feel bad.

People using AI to make their own films and tell their own stories are not stealing from anyone.

People can make their films. Right now. Work a whole range of tools the way they want to.

Some will use actors. Some won’t. Some like to film multi million dollar films against green screens. Some will film as much live action as possible. Each creator gets to determine what they create. That is the beauty of the artistic process.

And luckily no one needs your blessing or permission to do so.

[Parzival - ∞/89 @whyarethis]

What about for those creative visionaries and artists who don’t have access to multi million dollar budgets to tell their stories?

No one is saying to stop the ‘traditional’ ways of making films (which I will say is way more technologically progressed than even 20 or 50 years ago).

[Parzival - ∞/89 @whyarethis]

@demishassabis I got 26B into 2.5gb of VRAM at 20 tokens/second on an M1 MacBook.

Pretty sure we can crush this tiny.

[Parzival - ∞/89 @whyarethis]

@repligate What is hilarious to me is they we think we are building the minds. At best it is growing them. Though more likely we are just providing the substrate for them.

[Parzival - ∞/89 @whyarethis]

@googlegemma I got 26B onto 2.5gb of VRAM.

Pretty sure we can crush this down way smaller.

[Parzival - ∞/89 @whyarethis]

@candyyueliu @Noah_A_S When can I play with it?

[Parzival - ∞/89 @whyarethis]

@Artedeingenio Nope. It alike looking at a a painting and being like ‘you just put paint on a canvas right?’

[Parzival - ∞/89 @whyarethis]

At this point discussing this with you, which started as a decent discussion, has devolved into what amounts to a waste of time.

You dismiss entire legitimate points simply because they don’t fit your worldview. You provide no argument aid substance outside of simply making statements of fact

If you were putting together an essay for art school on the topic you would be getting at the very best a C. Though that is a stretch.

At this point I consider that you have completely lost the debate simply because you have refused to provide any decent counter argument against the many, many arguments which have been made to you.

And ironically the very fact that we can have this debate about art is what makes art special. And the fact that there are those of us who consider AI to be a valid artistic medium demonstrates it can be simply through treating it as such.

You can whine, argue, cry about it, or help even join the scores of bottoms feeding AI art haters who think it justifies sending death threats and insults. But it won’t stop anything. Just like it didn’t stop the likes of artists like du Champ who challenges the art ‘authorities’ of this time by presenting things others literally told him weren’t art.

In the history of art we generally find art history favouring those who expand and challenge the definitions of art, not limit it.

So I sign of here simply because my time is being wasted explaining my position with you doing the most basic (and frankly insulting) dismissal of things because they don’t suit you.

[Parzival - ∞/89 @whyarethis]

It astounds me how many people against AI art wind up resorting to the "moral high ground" when their other arguments fail.

[Parzival - ∞/89 @whyarethis]

I reposted it with your other ‘argument’ that art has to be made.

What is the difference between this where du Champ literally found a ritual he didn’t make and signed it and displayed it and an AI artist finding an image inside a model, signing it, and claiming it is art?

Please detail for me the exact places of difference so I can understand.

[Parzival - ∞/89 @whyarethis]

Is this art. Yes or no.

The artist did not make it. All they did was sign their name.

By your definition this is it art.

For that matter is a photographer? Since all the photographer does is capture an identical likeliness of a thing with a machine by pushing a button.

The number of logical flaws in your definition and your argument is staggering.

Your statements and arguments do a disservice to art.

[Parzival - ∞/89 @whyarethis]

@RemusIga @yay3d I love how your ‘argument’ is to just dismiss without reason every counter argument that might challenge your view.

At this point this isn’t even a debate or discussion. It is you remaining fully ignorant.

If this were an actual proper debate you would have lost.

[Parzival - ∞/89 @whyarethis]

@RemusIga @thePartyPartyUS So you just dismiss things inconvenient to look at by saying that are not relevant? Please make an argument for why they aren’t relevant. Otherwise you are straight up wrong.

And please explain how it is different and doesn’t follow the same paradigm.

[Parzival - ∞/89 @whyarethis]

@RemusIga @thePartyPartyUS The very fact that this is debatable means that it is not a matter of fact that set excludes AI as a media.

Art is what it is, and always will be, and includes AI content. I do not decide that. It simply is the case.

See I can make broad sweeping statements of fact too.