New Podcast Alert.
Today on #GAEATalks, @alex_whedon and Graeme Scott go deep on the macroeconomics of AI, why an efficiency-focused architecture is not a nice-to-have but an inevitability, and the two fundamental flaws of Transformer based models that are quietly capping what AI can do.
Click here for the full episode.
https://t.co/1lUlQxaNCM
"The more data you send to the model the less efficient it gets.
This is a major problem that leads to deficiencies in the model's ability to reason over large amounts of data, it limits the amount of training that we can do on a fixed amount of compute, and it creates skyrocketing costs which we're seeing very much in the market today.
As someone who has been working in the space for 10 years, this was a very important problem to solve.”
@alex_whedon & @Justin_SubQ from @subquadratic had a great conversation with @sophiadew on @MTSlive.
Check out this clip from their interview:
https://t.co/NCaVUnqIal
Had a great chat with @fredericl at @thenewstack.
We went deep on why architecture matters, and what is next for @subquadratic.
Go check it out. Link in comments.
We had a great conversation with @sophiadew x on @MTSlive.
We talked about why scaling AI isn't just about bigger models, but better architecture and what we're building next at @subquadratic.
Check it out.
@AppenResearch independently evaluated @subquadratic's SubQ 1.1 Small Preview on NIAH (1M–12M tokens) and LiveCodeBench (1,055 problems).
Results: 100% NIAH retrieval at 1M–2M, 98% at 12M, 89.7% pass@4 on LiveCodeBench.
Public brief: https://t.co/73PEMIKPLQ
Here is the technical report on SubQ 1.1 Small.
https://t.co/bu8AEc4lsk
This is the second iteration on our Subquadratic Sparse Attention (SSA) model, and the first to be deployed with design partners in the coming weeks.
The results are compelling and verified by @AppenResearch.
- Near-perfect long-context retrieval up to 12M tokens on the needle-in-a-haystack test, with up to nearly 1,000x attention compute reduction.
- A balance of long-context optimization and general reasoning ability, with strong performance retained across knowledge, coding, and non-coding enterprise agent benchmarks.
- At 1M tokens, SubQ 1.1 Small requires 64.5x less compute than dense attention and runs 56x faster than FlashAttention-2.
These results highlight a significant scaling advantage thanks to the efficiency gains from the SSA architecture.
We included some details and learnings from the development process which may be helpful to the community.
Comment with questions, I’ll try to respond!
We've partnered with Appen to evaluate the benchmarks we published last week.
Results are in and we've actually improved across the board.
Link below to the full report.
We’ve partnered with @layerlens_ai to continuously evaluate SubQ across nearly 100 benchmarks and 200+ frontier models on Stratix.
The goal is to continuously improve performance while reinforcing a shared commitment to transparency, auditability, and responsible model assessment.
Results and future evaluations will be published publicly.
Link below to learn more.
Latest @refreshmiami News: @subquadratic raised $29M on the idea that it has cracked AI’s biggest math problem. Now comes the hard part.
https://t.co/kxTMYBgvAg
#miamitech