Qurbani App

World's #1 Network Marketplace for the Livestock Community! Powered by AI.

[Ihtesham Ali @ihteshamali]

A mathematician who shared an office with Claude Shannon at Bell Labs gave one lecture in 1986 that explains why some people win Nobel Prizes and other equally smart people spend their whole lives doing forgettable work.

His name was Richard Hamming. He won the Turing Award. He invented error-correcting codes that made modern computing possible. And he spent 30 years at Bell Labs sitting in a cafeteria at lunch watching which scientists became legendary and which ones faded into nothing.

In March 1986, he walked into a Bellcore auditorium in front of 200 researchers and told them exactly what he had seen.

Here's the framework that has been quoted by every serious scientist for the last 40 years.

His opening line landed like a punch. He said most scientists he worked with at Bell Labs were just as smart as the Nobel Prize winners. Just as hardworking. Just as credentialed. And yet at the end of a 40-year career, one group had changed entire fields and the other group was forgotten by the time they retired.

He wanted to know what the difference actually was. And he said it wasn't luck. It wasn't IQ. It was a specific set of habits that almost nobody is willing to follow.

The first habit was the one that hurts the most to hear. He said most scientists deliberately avoid the most important problem in their field because the odds of failure are too high. They pick a safe adjacent problem, solve it cleanly, publish it, and move on. And because they never swing at the hard problem, they never hit it. He said if you do not work on an important problem, it is unlikely you will do important work. That is not a motivational line. That is a logical one.

The second habit was about doors. Literal doors. He noticed that the scientists at Bell Labs who kept their office doors closed got more done in the short term because they had no interruptions. But the scientists who kept their doors open got more done over a career. The open-door scientists were interrupted constantly. They also absorbed every new idea passing through the hallway. Ten years in, they were working on problems the closed-door scientists did not even know existed.

The third habit was inversion. When Bell Labs refused to give him the team of programmers he wanted, Hamming sat with the rejection for weeks. Then he flipped the question. Instead of asking for programmers to write the programs, he asked why machines could not write the programs themselves. That single inversion pushed him into the frontier of computer science. He said the pattern repeats everywhere. What looks like a defect, if you flip it correctly, becomes the exact thing that pushes you ahead of everyone else.

The fourth habit was the one that hit me the hardest. He said knowledge and productivity compound like interest. Someone who works 10 percent harder than you does not produce 10 percent more over a career. They produce twice as much. The gap doesn't add. It multiplies. And it compounds silently for years before anyone notices.

He finished the lecture with a line I have never been able to shake.

He said Pasteur's famous quote is right. Luck favors the prepared mind. But he meant it literally. You don't hope for luck. You engineer the conditions where luck can land on you. Open doors. Important problems. Inverted questions. Compounded hours. Those are not traits. Those are choices you make every single day.

The transcript has been sitting on the University of Virginia's computer science website for almost 30 years. The video is free on YouTube. Stripe Press reprinted the full lectures as a book in 2020 and Bret Victor wrote the foreword.

Hamming died in 1998. He gave his final lecture a few weeks before. He was 82.

The lecture that explains why some careers become legendary and others disappear is still free. Most people who could benefit from it will never open it.

[برنامج قدراتك - الأرشيف @DGAQtech_AR]

ضمن مسارات #قدراتك للبرامج الدولية، (13) قيادي من مختلف الجهات الحكومية يخطون بثقة في قيادة التحول الرقمي الحكومي لدى جهاتهم؛ عبر برنامج "قيادة التحول الرقمي الحكومي من خلال إعادة هندسة الإجراءات والذكاء الاصطناعي" بالتعاون مع @ISSNUS المنعقد في سنغافورة 🇸🇬.

[Rohan Paul @rohanpaul_ai]

MIT's "Mathematics for Computer Science".

A 1048 page available for Free.

Focuses on explaining the use of mathematical models and methods to analyze problems in computer science.

[Zulfiqar Ahmed 🤔 @ZulfiqarAhmed69]

Watching a great interview by Shahzad Saleem (CEO Nishat) where he highlights structural problems in our economy and in our system and offers insights and simple solutions.

It is a long podcast but worth watching/listening to.

Here is a clip where he talks about this ‘elite capture’, a term we see and hear too often in Pakistan.

[Yasser @yasser_elsaid_]

I rewatch this once a month for motivation

[Vinod Khosla @vkhosla]

Eric Johnson, one of my partners who was previously CTO of GitLab, wrote a fantastic guide on how to hire a CTO. Read here: https://t.co/psUAmhzj1o

[Nick Abraham @NickAbraham12]

Sam Altman, Founder of OpenAI, on cold email:

[Qurbani App @QurbaniApp]

@ZulfiqarAhmed69 God bless and hope you recover very soon!

[Qurbani App @QurbaniApp]

@RMHildebrandt Send

[Z Fellows @zfellows]

How to do Great Work by @paulg

[Andrej Karpathy @karpathy]

📽️ New 4 hour (lol) video lecture on YouTube:
"Let’s reproduce GPT-2 (124M)"
https://t.co/QTUdu8b0qh

The video ended up so long because it is... comprehensive: we start with empty file and end up with a GPT-2 (124M) model:
- first we build the GPT-2 network
- then we optimize it to train very fast
- then we set up the training run optimization and hyperparameters by referencing GPT-2 and GPT-3 papers
- then we bring up model evaluation, and
- then cross our fingers and go to sleep.
In the morning we look through the results and enjoy amusing model generations. Our "overnight" run even gets very close to the GPT-3 (124M) model. This video builds on the Zero To Hero series and at times references previous videos. You could also see this video as building my nanoGPT repo, which by the end is about 90% similar.

Github. The associated GitHub repo contains the full commit history so you can step through all of the code changes in the video, step by step.
https://t.co/BOzkxQ8at2

Chapters.
On a high level Section 1 is building up the network, a lot of this might be review. Section 2 is making the training fast. Section 3 is setting up the run. Section 4 is the results. In more detail:
00:00:00 intro: Let’s reproduce GPT-2 (124M)
00:03:39 exploring the GPT-2 (124M) OpenAI checkpoint
00:13:47 SECTION 1: implementing the GPT-2 nn.Module
00:28:08 loading the huggingface/GPT-2 parameters
00:31:00 implementing the forward pass to get logits
00:33:31 sampling init, prefix tokens, tokenization
00:37:02 sampling loop
00:41:47 sample, auto-detect the device
00:45:50 let’s train: data batches (B,T) → logits (B,T,C)
00:52:53 cross entropy loss
00:56:42 optimization loop: overfit a single batch
01:02:00 data loader lite
01:06:14 parameter sharing wte and lm_head
01:13:47 model initialization: std 0.02, residual init
01:22:18 SECTION 2: Let’s make it fast. GPUs, mixed precision, 1000ms
01:28:14 Tensor Cores, timing the code, TF32 precision, 333ms
01:39:38 float16, gradient scalers, bfloat16, 300ms
01:48:15 torch.compile, Python overhead, kernel fusion, 130ms
02:00:18 flash attention, 96ms
02:06:54 nice/ugly numbers. vocab size 50257 → 50304, 93ms
02:14:55 SECTION 3: hyperpamaters, AdamW, gradient clipping
02:21:06 learning rate scheduler: warmup + cosine decay
02:26:21 batch size schedule, weight decay, FusedAdamW, 90ms
02:34:09 gradient accumulation
02:46:52 distributed data parallel (DDP)
03:10:21 datasets used in GPT-2, GPT-3, FineWeb (EDU)
03:23:10 validation data split, validation loss, sampling revive
03:28:23 evaluation: HellaSwag, starting the run
03:43:05 SECTION 4: results in the morning! GPT-2, GPT-3 repro
03:56:21 shoutout to llm.c, equivalent but faster code in raw C/CUDA
03:59:39 summary, phew, build-nanogpt github repo

[Qurbani App @QurbaniApp]

@ZFellows_ Your posts are a fresh air of motivation. Thank you!

[Tanvir Anjum @codemindtanvir]

Robots are taking over the world at CES 2024!

Top 10 Most Amazing Robots that will blow your mind:🤖

[Nicolas Cole 🚢👻 @Nicolascole77]

Meet John Grisham.

He’s the author of 49 consecutive #1 bestsellers.

His books have been made into movies starring Tom Cruise, Sandra Bullock, and Samuel L. Jackson.

Here are 10 of his timeless writing insights (on talent, routine, and dealing with criticism):

[Curiosity @MAstronomers]

The most accurate map of the entire known Universe in one image:

[Qurbani App @QurbaniApp]

@frankdotlee @ycombinator @paulg Great post, looking forward to reading it!

[Lenny Rachitsky @lennysan]

Inside @Canva's famous SEO strategy

[Epoch AI @EpochAIResearch]

How many large AI models are out there, who developed them, and for what applications? To answer this question, we present a new dataset tracking every AI model we could find trained with over 10^23 FLOP.

Highlights in thread 🧵

[Robert Silverman @Robert_Academic]

💥🇪🇺 EU AI ACT Cheat Sheet Series 💥🇪🇺

Part 1. Definitions, Scope & Applicability

Last week, EU member state governments formally approved the EU AI Act.

To mark this moment, I am launching a new EU AI Act Cheat Sheet Series.