flainez

I code Apps and stuff

[Yann LeCun @ylecun]

Worth repeating:
Do not confuse retrieval with reasoning.
Do not confuse rote learning with understanding.
Do not confuse accumulated knowledge with intelligence.

[Peter Doherty @petedoherty]

This week in 2007, @babyshamblesuk released Shotters Nation x

[Ilya Sutskever @ilyasut]

I am starting a new company:

[Claire Penis @ZeroSuitCamus]

Imagine going to the club in 1970 to hang with your friends and a band you’ve never heard of called Kraftwerk is playing and they ruin your night by inventing techno music

[Rohan Paul @rohanpaul_ai]

This is really a 'WOW' paper. 🤯

Claims that MatMul operations can be completely eliminated from LLMs while maintaining strong performance at billion-parameter scales and by utilizing an optimized kernel during inference, their model’s memory consumption can be reduced by more than 10× compared to unoptimized models. 🤯

'Scalable MatMul-free Language Modeling'

Concludes that it is possible to create the first scalable MatMul-free LLM that achieves performance on par with state-of-the-art Transformers at billion-parameter scales.

📌 The proposed MatMul-free LLM replaces MatMul operations in dense layers with ternary accumulations using weights constrained to {-1, 0, +1}. This reduces computational cost and memory utilization while preserving network expressiveness.

📌 To remove MatMul from self-attention, the Gated Recurrent Unit (GRU) is optimized to rely solely on element-wise products, creating the MatMul-free Linear GRU (MLGRU) token mixer. The MLGRU simplifies the GRU by removing hidden-state related weights, enabling parallel computation, and replacing remaining weights with ternary matrices.

📌 For MatMul-free channel mixing, the Gated Linear Unit (GLU) is adapted to use BitLinear layers with ternary weights, eliminating expensive MatMuls while maintaining effectiveness in mixing information across channels.

📌 The paper introduces a hardware-efficient fused BitLinear layer that optimizes RMSNorm and BitLinear operations. By fusing these operations and utilizing shared memory, training speed improves by 25.6% and memory consumption reduces by 61% over an unoptimized baseline.

📌 Experimental results show that the MatMul-free LLM achieves competitive performance compared to Transformer++ baselines on downstream tasks, with the performance gap narrowing as model size increases. The scaling law projections suggest MatMul-free LLM can outperform Transformer++ in efficiency and potentially in loss when scaled up.

📌 A custom FPGA accelerator is built to exploit the lightweight operations of the MatMul-free LLM. The accelerator processes billion-parameter scale models at 13W beyond human-readable throughput, demonstrating the potential for brain-like efficiency in future lightweight LLMs.

[Swizec Teller @Swizec]

Don’t use AI to write your tests

[HIDEO_KOJIMA @HIDEO_KOJIMA_EN]

Good morning.

[HIDEO_KOJIMA @HIDEO_KOJIMA_EN]

Witnessed “Furiosa: A Mad Max Saga”!
This movie, which easily surpasses "MAD" and even past "FURY", is at its "MAX" (masterpiece)! Ever since I saw the first film when I was 16 years old, George Miller has saved me, encouraged me, and changed my way of life countless times. He is my God, and the SAGA that he tells is my Bible.

[HIDEO_KOJIMA @HIDEO_KOJIMA_EN]

May the 4th be with you.

[Ben Stiller @BenStiller]

I never met a kinder, more empathetic comedy genius. He was so funny. And deep. As a stand up he was really iconic in the 70s. Cool, funny, self deprecating and hip. He was a friend to my parents and the whole Stiller family. As a kid i remember seeing him at the Improv and how nice he was to me and my sister. Over the years he would always reach out with support and love or a kind word - sometimes out of the blue. It always felt special to hear from him. I feel very lucky to have known him over all these years. I’m sad I won’t see him again.

We’ll miss him very, very much. A true comedy great. Sending love to all of Richard’s family and friends.

[John Carmack @ID_AA_Carmack]

“Coding” was never the source of value, and people shouldn’t get overly attached to it. Problem solving is the core skill. The discipline and precision demanded by traditional programming will remain valuable transferable attributes, but they won’t be a barrier to entry.

Many times over the years I have thought about a great programmer I knew that loved assembly language to the point of not wanting to move to C. I have to fight some similar feelings of my own around using existing massive codebases and inefficient languages, but I push through.

I had somewhat resigned myself to the fact that I might be missing out on the “final abstraction”, where you realize that managing people is more powerful than any personal tool. I just don’t like it, and I can live with the limitations that puts on me.

I suspect that I will enjoy managing AIs more, even if they wind up being better programmers than I am.

[Chema Pamundi @chemapamundirl]

"Dune: parte 2" mejora a la primera en todo, tanto a nivel visual como de guion, liberada ya de la presentación de personajes y entornos que convertía a la anterior entrega en un prólogo de luxe algo árido. Carisma, profundidad emocional, valentía estilística y epicidad rotunda.

[The Hives @TheHives]

The Hives, next question.

[The Hives @TheHives]

If you start The Hives soon to be award winning new album ‘The Death of Randy Fitzsimmons’ at 11:53:51pm tonight you’ll enter the New Year hearing the words ‘Countdown to Shutdown’.

Start 2024 the correct way.

[HIDEO_KOJIMA @HIDEO_KOJIMA_EN]

A24 and DEATH STRANDING will be made into a live-action movie🎬📷🫶

“A24 was born into this world about 10 years ago, their presence is singular within the industry, they are like no other. The films they are delivering to the world are high in quality and very innovative. I have been attracted to their creations and they have even inspired my own work. Their innovative approach to storytelling aligns with what Kojima Productions has been doing for the last 8 years. Now, we are making a Death Stranding movie together. There are a lot of “game adaptation films" out there but what we are creating is not just a direct translation of the game. The intention is that our audience will not only be fans of the games, but our film will be for anyone who loves cinema. We are creating a Death Stranding universe that has never been seen before, achievable only through the medium of film, it will be born.” - Hideo Kojima

[KOJIMA PRODUCTIONS (Eng) @KojiPro2015_EN]

KOJIMA PRODUCTIONS and A24 will team up to adapt DEATH STRANDING into a live-action feature film!🤝

Click here for more details👉
https://t.co/VbQi66xaW4

#KojimaProductions #A24 @A24

[A24 @A24]

It’s official: we’ve partnered with Kojima Productions on the live-action feature film adaptation of Hideo Kojima’s acclaimed video game, DEATH STRANDING.