James will be joining for a part of this, and is a brilliant man who has done so much for this cause.
@JamesTitcombe@matthewsyed
Links to the challenge, the fundraising page, and the @The_Gazette article.
https://t.co/Tz7bRHpI52
https://t.co/Rk0e1jFrTa
We also show ThriftAttention significantly outperforms inference-time sparsity methods at equivalent FLOPs budgets. This is because ThriftAttention degrades much more smoothly than hard sparsity methods.
Negative log likelihood analysis shows that FP4 attention increasingly degrades at longer contexts. Conversely, ThriftAttention's output quality is maintained relative to FP16, making ThriftAttention increasingly valuable at longer contexts.
Across long-context benchmarks, by computing just 5% of the attention computation in FP16, ThriftAttention recovers 94% of the performance gap between FP4 and FP16!
The method is simple. 1) Use a heuristic to identify important key-value blocks for a given query. 2) Compute important blocks in FP16, the rest in FP4.
FP4 Attention is fast, but at long context quality collapses.
Introducing ThriftAttention: keep important attention blocks in FP16, compute the rest in FP4.
Result: FP16 output quality at sub-byte inference latency.
https://t.co/zjkmvsVzrD
https://t.co/f8VYg7AjMK
Slabhead has never let his country down, easy to imagine a scenario where we are goal down and we may score it due to his massive set piece dominance, biggest shocker for me after the season he had.
The beginning of the end for Tuchel, scoring an own goal with the fans before the tournament even starts.
6 - Liverpool have now scored six 90th-minute winners against Everton in the Premier League - the most one side has against a single opponent in the competition's history. Bragging.
Steven Gerrard on Galatasaray:
"They were an embarrassment tonight from start to finish. It was pathetic and then you see them rolling around. They had a couple of genuine injuries. Liverpool made Galatasaray look bang average. I thought they were pathetic." (TNT)
RIP the most underrated British winger of all time John Robertson. Imagine a mixture of Kvara, Grealish and Pires and you’ll get a feel for his style as a player.
No wonder he was voted as Nottingham Forest’s favourite player of all time.
#NFFC
On behalf of the European football community, we are saddened by the passing of former Scotland and Nottingham Forest winger John Robertson.
Robertson scored the only goal in the 1980 European Cup final, having assisted the winner in the 1979 final.
Our thoughts are with his family, friends and former teammates.