Rick Izzo

[Soumith Chintala @soumithchintala]

Why do 16k GPU jobs fail?
The Llama3 paper has many cool details -- but notably, has a huge infrastructure section that covers how we parallelize, keep things reliable, etc.
We hit an overall 90% effective-training-time.
https://t.co/5gngOZJHBO

[Andrew Ng @AndrewYNg]

It is only rarely that, after reading a research paper, I feel like giving the authors a standing ovation. But I felt that way after finishing Direct Preference Optimization (DPO) by @rm_rafailov @archit_sharma97 @ericmitchellai @StefanoErmon @chrmanning and @chelseabfinn. This beautiful paper proposes a much simpler alternative to RLHF (reinforcement learning from human feedback) for aligning language models to human preferences.

RLHF has been a key technique for training LLMs. In brief, RLHF (i) Gets humans to specify their preferences by ranking LLM outputs, (ii) Trains a reward model (used to score LLM outputs) -- typically represented using a transformer network -- to be consistent with the human rankings, (iii) Uses reinforcement learning to tune an LLM, also represented as a transformer, to maximize rewards. This requires two transformer networks, and RLHF is also finicky to the choice of hyperparameters.

DPO simplifies the whole thing. Via clever mathematical insight, the authors show that given an LLM, there is a specific reward function for which that LLM is optimal. DPO then trains the LLM directly to make the reward function (that’s now implicitly defined by the LLM) consistent with the human rankings. So you no longer need to deal with a separately represented reward function, and you can train the LLM directly to optimize the same objective as RLHF.

Although it’s still too early to be sure, I am cautiously optimistic that DPO will have a huge impact on LLMs and beyond in the next few years.

You can read the paper here: https://t.co/m14qRYszVa I also write more about this in The Batch (linked to below).
https://t.co/8h2ag2plIa

[Rick Izzo @TheRickIzzo]

@mylifcc @LightningAI @mylifcc you're in! enjoy!

[Lightning AI ⚡️ @LightningAI]

All-in-one. Zero setup. It’s finally here - Lightning AI Studios

Launch a free Studio - https://t.co/IcYvVsd88b

[Rick Izzo @TheRickIzzo]

@mylifcc @LightningAI Hey @mylifcc , Rick from @LightningAI here. Shoot me a DM with your email you used to sign up and I’ll admit you from the waitlist!

[Rick Izzo @TheRickIzzo]

@Mickey_English @LightningAI You’re on! Enjoy!

[Rick Izzo @TheRickIzzo]

@Mickey_English @LightningAI Hey @Mickey_English, Rick from @LightningAI here. Send me a DM with your email you used to sign up and I’ll give you access!

[Noah Hein @TheNoahHein]

Devs when github is down

[Sebastian Raschka @rasbt]

You love using PyTorch for Deep Learning but want it a bit more organized, so it's easier to take advantage of more advanced features?

Great news: Unit 5 is finally live! In Unit 5, I'll show you how to train PyTorch models with the Lightning Trainer!

🔗 https://t.co/MJB1pr3npi

[Rick Izzo @TheRickIzzo]

@LinuxSeb Linux Mint

[Rick Izzo @TheRickIzzo]

@nevrekaraishwa2 @savetobookmarks

[Rick Izzo @TheRickIzzo]

@nixcraft Pop!_OS. It kindof just works

[Presley Mullinax @PresleyMullinax]

Is @Google really down right now?!? Or is this just a me problem? #googledown

[Kyunghyun Cho @kchonyc]

hmm.. what is this!? @PyTorchLightnin

[William Falcon ⚡️ @williamfalcon]

1/3 As a researcher from a non-traditional background, I take attribution very seriously and embrace constructive scientific discussion that accelerates learning and advances progress. Flash was developed to serve the evolving needs of PyTorch Lightning users.

[Lightning AI ⚡️ @LightningAI]

We've surpassed 1 million downloads of Lightning! 🥳 Huge thank you to our contributors and #community. #opensource #deeplearning #communitylove

[Lightning AI ⚡️ @LightningAI]

New project from @facebookai and @nyulangone. Helping covid efforts.

written in #pytorchLightning

[Rick Izzo @TheRickIzzo]

@raymondh I think I first learned of tau in some undergrad course on physics of nuclear imaging.

Never heard it used by anyone but a physicist.

[Rick Izzo @TheRickIzzo]

@ravinwashere Apple. No better mix between *nix and a good desktop experience.

[Jon Favreau @jonfavs]

I’m sorry did the President just say that catching Covid is a blessing from God???