one_happy_programmer

A constant learner.

One of my favorite quotes is 'I never want my mind to stop working' by Bill gates

[AK @_akhaliq]

Apple announces LLM in a flash: Efficient Large Language Model Inference with Limited Memory

paper page: https://t.co/SuqHJUQPO9

Large language models (LLMs) are central to modern natural language processing, delivering exceptional performance in various tasks. However, their intensive computational and memory requirements present challenges, especially for devices with limited DRAM capacity. This paper tackles the challenge of efficiently running LLMs that exceed the available DRAM capacity by storing the model parameters on flash memory but bringing them on demand to DRAM. Our method involves constructing an inference cost model that harmonizes with the flash memory behavior, guiding us to optimize in two critical areas: reducing the volume of data transferred from flash and reading data in larger, more contiguous chunks. Within this flash memory-informed framework, we introduce two principal techniques. First, "windowing'" strategically reduces data transfer by reusing previously activated neurons, and second, "row-column bundling", tailored to the sequential data access strengths of flash memory, increases the size of data chunks read from flash memory. These methods collectively enable running models up to twice the size of the available DRAM, with a 4-5x and 20-25x increase in inference speed compared to naive loading approaches in CPU and GPU, respectively. Our integration of sparsity awareness, context-adaptive loading, and a hardware-oriented design paves the way for effective inference of LLMs on devices with limited memory.

[Mercedes-AMG PETRONAS F1 Team @MercedesAMGF1]

One year of Lauda Drive ❤

#OTD in 2022, we renamed the road at the heart of our Brackley home after our friend and non-executive chairman, Niki Lauda 🙏

[Apurv Anand @apurv_anand]

Five Extinctions
Ordovician-silurian Extinction: 440 million years ago.
Devonian Extinction: 365 million years ago.
Permian-triassic Extinction: 250 million years ago.
Triassic-jurassic Extinction: 210 million years ago.
Cretaceous-tertiary Extinction: 65 million Years Ago.

The Permian-Triassic extinction, also known as the "Great Dying", is considered the worst extinction event in Earth's history. It occurred 252 million years ago and killed off 96% of marine species and three out of four land species over about 60,000 years. The extinction was caused by climate changes such as:
Declining oxygen levels in the water
Rising water temperatures
Ocean acidification

[Historic Vids @historyinmemes]

The five major extinctions in the history of our planet

[Ruurtjan Pul @Ruurtjan]

I ❤️ DNS

I’ve spent 2 years full-time building https://t.co/LYem2r1Vyh.

Now, I’m teaching everything I know in this course.

[Linus Tech Tips @LinusTech]

NEW VIDEO! Let’s See If I Can Get Fired - AMD $5,000 Ultimate Tech Upgrade

https://t.co/V3EqlR077w

Enter the AMD giveaway for TWO ROG Allys below!

https://t.co/ffXyIkPtmD

#sponsoredby @AMD

[Mercedes-AMG PETRONAS F1 Team @MercedesAMGF1]

✨ perfection ✨

[Linus Tech Tips @LinusTech]

the scariest part of the #AppleEvent so far is that they keep comparing the new M3 chip to the M1, not the M2

[McLaren Mastercard Formula 1 Team @McLarenF1]

What a drive to secure another @salesforce Driver of the Day award thanks to your votes. 🧡

#MexicoGP

[Mercedes-AMG PETRONAS F1 Team @MercedesAMGF1]

P2 in Mexico City. Cheers to that. 🤝 Now onto Brazil.

[GitHub @github]

The README on your GitHub profile acts like a front door to your work, skills, and professional self. These tips for making your content accessible will help you increase clarity and usability for everyone, not just people with disabilities. https://t.co/kE4ST63apd

[Sushanta Sethi @spectatorofX]

@HumansNoContext How moon affects ocean tides on earth

[World of Engineering @engineers_feed]

Space fact:

The sun makes up 99.8% of the mass of the entire solar system!

[Studio Eclypse @studio_eclypse]

Pre-production has officially begun:

BERSERK: The Black Swordsman

This series is very close to our hearts, we aim to create a proper 2D adaptation of the stories that have been neglected of this medium, starting from the beginning.

#BERSERK #Guts #Casca #Griffith #fananimation #indie_anime #eclypse

[TechLinked @TechLinkedYT]

NEW VIDEO!

New Radeons, Playstation Portal, Intel 14th gen leaks + more!
YT Link: https://t.co/GEiIfcSbdQ
News Sources: https://t.co/6dfCaZsdk4

[Gaurav Sen @gkcs_]

Here is a fantastic blog on behavioral-driven interviews.

I think it is the least known, and hence the least-optimized, part of the interview process for software development roles.

https://t.co/XhWz42N4A1

[Atlassian Williams F1 Team @WilliamsF1]

Reserve driver confirmed 👀

[Ken @kenbolido]

✨ show me your homescreen ✨

[Ian Miles Cheong @ianmiles]

The first people whose jobs should be entirely replaced by machines.

[Elon Musk @elonmusk]

Doges ftw