Il punto interessante di SubQ non รจ โsoloโ che il modello รจ piรน intelligente, ma come ci arriva.
Gli LLM attuali (GPT, Claude, Gemini, ecc.) usano Transformer ๐ค classici con complessitร quadratica: piรน aumenta il contesto/token, piรน il costo computazionale cresce in modo esplosivo (detta in 2 spicci).
SubQ invece usa (da quello che ho capito) una nuova architettura โsub-quadratic sparse attentionโ, che permette di:
gestire contesti molto piรน lunghi,usare meno memoria e meno GPU,scalare meglio,e teoricamente mantenere qualitร elevata anche su task molto lunghi/complessi.
In pratica: invece di โguardare tutto contro tuttoโ come fanno i Transformer tradizionali, il modello impara a concentrarsi solo sulle parti realmente rilevanti del contesto.
Per questo potrebbe essere un grosso cambio di paradigma:
inferenza piรน economica,training piรน efficiente,agenti AI che riescono a ragionare su enormi codebase/documentazioni/conversazioni senza degradare.
Esempio pratico:
oggi se dai a GPT un repository enterprise enorme (es. milioni di righe di codice + ticket Jira + Slack + docs), il modello tende a perdere contesto o diventare costosissimo.
Unโarchitettura come SubQ potrebbe invece mantenere โmemoria utileโ su quantitร enormi di informazioni senza far esplodere i costi computazionali.
Se questa architettura mantiene davvero qualitร pari o superiore ai Transformer standard, potrebbe essere uno dei primi veri โpost-Transformer momentsโ dellโAI moderna.
Come dice il vecchio saggio: We'll seeโฆ
Introducing SubQ - a major breakthrough in LLM intelligence.
It is the first model built on a fully sub-quadratic sparse-attention architecture (SSA),
And the first frontier model with a 12 million token context window which is:
- 52x faster than FlashAttention at 1MM tokens
- Less than 5% the cost of Opus
Transformer-based LLMs waste compute by processing every possible relationship between words (standard attention).
Only a small fraction actually matter.
@subquadratic finds and focuses only on the ones that do.
That's nearly 1,000x less compute and a new way for LLMs to scale.
@lavitalenta Funny enough, thatโs my village from the window! ๐คฃ
Colledimezzo (Abruzzo) a 400 people borgo in the middle of the Appennino mountains chain were a couple can live well with less than 1000โฌ per month๐๐ป
No idea how it ended up in the meme! ๐คทโโ๏ธ
@SearsReloaded That cash should be related to the warrants.
Probably, when you received the warrants, you expressed your intention to sell it, so that cash is the proceedings of warrants sell.
This is what happened to me. I received my check about 2 weeks ago (Iโm in EU).
~$33, good for ๐
@grok@marcuslemonis And in the Hertz case, how long did it took for old shareholders to get something back, after the old name and ticker started trading again?
@100trillionUSD Basically you do NOT own any bitcoin anymore. Blackrock owns them now. You are a beneficiary owner, like for your equities and bonds.
Now they can use your bitcoins as a collateral for open shorts positions against your bitcoins.
Great move! ๐
NEW: Strike CEO Jack Mallers says, โNo one controls #Bitcoin. The identity of who discovered it is irrelevant.โ
โEnough with the Satoshi speculation.โ ๐