Riddle

[Riddle @Riddle4045]

@levelsio If you torture data long enough, it’ll tell you anything.

[Riddle @Riddle4045]

@earthcurated @earthcurated are there hi-def versions of these m, that you could share?

[Riddle @Riddle4045]

@iRobot your move

[SpaceX @SpaceX]

Mechazilla has caught the Super Heavy booster!

[Riddle @Riddle4045]

LOL

[TENDO XI @TendoXI]

If Frieza ever fought a super saiyan 4, we would’ve heard some of the most creative slurs in history

[Riddle @Riddle4045]

💯

[Massimo @Rainmaker1973]

#Today in 1997, the game Age of Empires was released by Ensemble Studios. It is now generally regarded as one of the best video games ever made.

[Riddle @Riddle4045]

Incredible!

[SpaceX @SpaceX]

Mechazilla has caught the Super Heavy booster!

[Riddle @Riddle4045]

@hamptonism Credits : @3blue1brown

[The Nobel Prize @NobelPrize]

BREAKING NEWS
The Royal Swedish Academy of Sciences has decided to award the 2024 #NobelPrize in Physics to John J. Hopfield and Geoffrey E. Hinton “for foundational discoveries and inventions that enable machine learning with artificial neural networks.”

[Riddle @Riddle4045]

@notsky Google “Netflix engineering” ;) enjoy!

[Riddle @Riddle4045]

Someone tell me what I need to learn to understand this code

[Andrej Karpathy @karpathy]

Day 24 of llm.c: we now do multi-GPU training, in bfloat16, with flash attention, directly in ~3000 lines of C/CUDA, and it is FAST! 🚀

We're running ~7% faster than PyTorch nightly, with no asterisks, i.e. this baseline includes all modern & standard bells-and-whistles: mixed precision training, torch compile and flash attention, and manually padding vocab. (Previous comparisons included asterisks like *only inference, or *only fp32 etc.) Compared to the current PyTorch stable release 2.3.0, llm.c is actually ~46% faster. My point in these comparisons is just to say "llm.c is fast", not to cast any shade on PyTorch. It's really amazing that PyTorch trains this fast in a fully generic way, with ability to cook up and run ~arbitrary neural networks and run them on a ton of platforms. I see the goals and pros and cons of these two projects as different, even complementary. Actually I started llm.c with my upcoming education videos in mind, to explain what PyTorch does for you under the hood.

How we got here over the last ~1.5 weeks - added:

✅ mixed precision training (bfloat16)
✅ many kernel optimizations, including e.g. a FusedClassifier that (unlike current torch.compile) does not materialize the normalized logits.
✅ flash attention (right now from cudnn)
✅ Packed128 data structure that forces the A100 to utilize 128-bit load (LDG.128) and store (STS.128) instructions.

It's now also possible to train multi-GPU - added:
✅ First version of multi-gpu training with MPI+NCCL
✅ Profiling the full training run for NVIDIA Nsight Compute
✅ PR for stage 1 of ZeRO (optimizer state sharding) merging imminently

We're still at "only" 3,000 lines of code of C/CUDA. It's getting a bit less simple, but still bit better than ~3 million. We also split off the fp32 code base into its own file, which will be pure CUDA kernels only (no cublas or cudnn or etc), and which I think would make a really nice endpoint of a CUDA course. You start with the gpt2.c pure CPU implementation, and see how fast you can make it by the end of the course on GPU, with kernels only and no dependencies.

Our goal now is to create a reliable, clean, tested, minimal, hardened and sufficiently optimized LLM stack that reproduces the GPT-2 miniseries of all model sizes, from 124M to 1.6B, directly in C/CUDA.

A lot more detail on: "State of the Union [May 3, 2024]"
https://t.co/eDgbngHrZ9

[Riddle @Riddle4045]

@karpathy What do I need to learn to understand this😅

[Riddle @Riddle4045]

@ModernWarzone they just discovered git revert and are going bananas with it

[ModernWarzone @ModernWarzone]

Sledgehammer Games is killing it this year, I just wish so much of their time didn’t have to be spent fixing mistakes from last year / Modern Warfare 2 / Warzone 2.

[Riddle @Riddle4045]

@anerdguynow I use it almost everyday to enable devbox debugging config/changes

[Riddle @Riddle4045]

Rest in peace Akira Toriyama and thank you for of the best childhood memories

[Riddle @Riddle4045]

@elonmusk @OpenAI Open source FSD source code

[Riddle @Riddle4045]

@HusKerrs 👋

[Riddle @Riddle4045]

@iamgingertrash Says “simp 4 Satoshi” aka “gingertrash” 🤣

[World of Engineering @engineers_feed]

This is what your brain often does just willing to comfort you.
https://t.co/DXVK4nh2rH

[Bojan Tunguz @tunguz]

Pro tip

[Riddle @Riddle4045]

Devastating if true

[GURGAVIN @gurgavin]

JUST IN : COMCAST SAYS HACKERS MAY HAVE STOLEN USERS PASSWORDS, FULL NAMES, CONTACT INFORMATION, LAST FOUR DIGITS OF SOCIAL SECURITY NUMBERS, DATE OF BIRTH AND SECRET SECURITY QUESTIONS AND ANSWERS PER BLOOMBERG

$CMCSA