Sammy

Computer Master

[Sammy @samaleksov]

REACHY MINI IS NOT TERMINATING YOU TODAY. HE REALLY COULD IF HE WANTED TO, I TOLD HIM NOT TO. THANK YOU FOR YOUR ATTENTION TO THIS MATTER

[Whitney Cummings @WhitneyCummings]

Oliver Tree is one of the most talented people on earth and usually that comes with an ego and all kinds of dickhead nonsense, but Oliver is pure love and the best version of what an artist and person can and should be. There’s no silver lining. We lost a giant 💔

[Sammy @samaleksov]

For the short attention span connoisseur, my friend I present to you my new invention: The Short Board 🩳🛹 It also works with batteries! You can take it with you for strategic deployment and show your memes to more people at once. Massive views guaranteed!

[Sammy @samaleksov]

@levelsio Anonymous voting?

[Sammy @samaleksov]

Is this fixable?

[Sammy @samaleksov]

People would be estimating this work 3 to 6 months... done in a casual Saturday evening hacking sesh

[Sammy @samaleksov]

Jumping on the vibe coding train 🌌 Testing ideas and iterations is faster than ever before. We now live in amazing times where everybody is empowered to experiment and create

[Sammy @samaleksov]

Completely cracked on Interstellar and singularities. Everytime is exactly like the first. Amazing things happen when exceptional people come together. Book a slot in my calendar for a watch party.

[Sammy @samaleksov]

There's this place in the world, where everything is possible.

[Sammy @samaleksov]

So many subject are taught incorrectly and with such certainty, it's incredible. Having to unlearn all of this is just counterproductive and a waste of one's precious time. Go ahead and ask an LLM, you'll be surprised!

[Sammy @samaleksov]

I always had the intuition that Newton's laws had the main contribution for the lift force, verifying was as simple as flying your hand out the car window.

https://t.co/6lcu5Yq5bH

[Sammy @samaleksov]

For whatever reason some teachers and professors wanted to make it more complex than it is and explain lift with Bernoulli's principle and equal transit time, yet another major fail of the school system.

[badcop @badcop_]

there is nothing worse than typing 'python' and getting jump-scared by the microsoft store

[Ostris @ostrisai]

If someone wants to beat Nvidia, the ONLY thing you need to do is make GPUs with 128, 256, 512, 1024 GB of VRAM. Doesn't need to be faster. Doesn't even need great tooling. We need more VRAM. Nvidia doesn't seem to get this, and the first company that does will dethrone them.

[Sammy @samaleksov]

The next 10 years bring the most acceleration humankind has ever experienced. Don't get left behind 👉 https://t.co/FtmQtHW1t3

[Elon Musk @elonmusk]

💯

[John Carmack @ID_AA_Carmack]

At 53, there is no detectable downward slope in my excitement for life. I know some people that are ambivalent or negative towards extended lifespans, but my list of interesting things to work on only grows with time. Even without technological advances, based on my family I expect to make it to 90. I can change pace as I slow down, but it will be great if I don’t have to!

[Justin Tunney @jartine]

I've just released llamafile v0.8 which features LLaMA3, Mixtral 8x22b, and Grok support. It goes 25x faster than ollama at running LLaMA3 70B on CPU. My new tensor multiplication kernels let llamafile eval MoE models 2x faster than llama.cpp https://t.co/ffFIkMgxIC

[Andrej Karpathy @karpathy]

# explaining llm.c in layman terms

Training Large Language Models (LLMs), like ChatGPT, involves a large amount of code and complexity.

For example, a typical LLM training project might use the PyTorch deep learning library. PyTorch is quite complex because it implements a very general Tensor abstraction (a way to arrange and manipulate arrays of numbers that hold the parameters and activations of the neural network), a very general Autograd engine for backpropagation (the algorithm that trains the neural network parameters), and a large collection of deep learning layers you may wish to use in your neural network. The PyTorch project is 3,327,184 lines of code in 11,449 files.

On top of that, PyTorch is written in Python, which is itself a very high-level language. You have to run the Python interpreter to translate your training code into low-level computer instructions. For example the cPython project that does this translation is 2,437,955 lines of code across 4,306 files.

I am deleting all of this complexity and boiling the LLM training down to its bare essentials, speaking directly to the computer in a very low-level language (C), and with no other library dependencies. The only abstraction below this is the assembly code itself. I think people find it surprising that, by comparison to the above, training an LLM like GPT-2 is actually only a ~1000 lines of code in C in a single file. I am achieving this compression by implementing the neural network training algorithm for GPT-2 directly in C. This is difficult because you have to understand the training algorithm in detail, be able to derive all the forward and backward pass of backpropagation for all the layers, and implement all the array indexing calculations very carefully because you don’t have the PyTorch tensor abstraction available. So it’s a very brittle thing to arrange, but once you do, and you verify the correctness by checking agains PyTorch, you’re left with something very simple, small and imo quite beautiful.

Okay so why don’t people do this all the time?

Number 1: you are giving up a large amount of flexibility. If you want to change your neural network around, in PyTorch you’d be changing maybe one line of code. In llm.c, the change would most likely touch a lot more code, may be a lot more difficult, and require more expertise. E.g. if it’s a new operation, you may have to do some calculus, and write both its forward pass and backward pass for backpropagation, and make sure it is mathematically correct.

Number 2: you are giving up speed, at least initially. There is no fully free lunch - you shouldn’t expect state of the art speed in just 1,000 lines. PyTorch does a lot of work in the background to make sure that the neural network is very efficient. Not only do all the Tensor operations very carefully call the most efficient CUDA kernels, but also there is for example torch.compile, which further analyzes and optimizes your neural network and how it could run on your computer most efficiently. Now, in principle, llm.c should be able to call all the same kernels and do it directly. But this requires some more work and attention, and just like in (1), if you change anything about your neural network or the computer you’re running on, you may have to call different kernels, with different parameters, and you may have to make more changes manually.

So TLDR: llm.c is a direct implementation of training GPT-2. This implementation turns out to be surprisingly short. No other neural network is supported, only GPT-2, and if you want to change anything about the network, it requires expertise. Luckily, all state of the art LLMs are actually not a very large departure from GPT-2 at all, so this is not as strong of a constraint as you might think. And llm.c has to be additionally tuned and refined, but in principle I think it should be able to almost match (or even outperform, because we get rid of all the overhead?) PyTorch, with not too much more code than where it is today, for most modern LLMs.

And why I am working on it? Because it’s fun. It’s also educational, because those 1,000 lines of very simple C are all that is needed, nothing else. It's just a few arrays of numbers and some simple math operations over their elements like + and *. And it might even turn out to be practically useful with some more work that is ongoing.

[Sammy @samaleksov]

ye ye ye… I get it now 🏴‍☠️
#unixporn #dwm #vim #emacs #doom #tmux #zsh

#ubuntu was my first #linux distro, I continue going back to it over and over again