Orizuru

[Magnus Carlsen @MagnusCarlsen]

The definition of insanity is doing the same thing over and over and expecting different results.

🏆🏆🏆🏆🏆
🏆🏆🏆🏆🏆
🏆🏆🏆🏆🏆
🏆🏆🏆🏆🏆

[Orizuru @OriSenbazuru]

🕺🏆 https://t.co/48s40Nxvwv via @YouTube

[Chess.com @chesscom]

Congratulations on an incredible, undefeated #FreestyleChess tournament in South Africa, @LevAronian 🔥

[Andrej Karpathy @karpathy]

nanoGPT - the first LLM to train and inference in space 🥹. It begins.

[Chess.com @chesscom]

In memory of Daniel Naroditsky ❤️

[Quanta Magazine @QuantaMagazine]

As a child, Hannah Cairo learned math by taking online lessons from Khan Academy. By the time she was 14, she had taught herself the equivalent of an advanced undergraduate math degree.
https://t.co/gTOeUIa9cZ

[Orizuru @OriSenbazuru]

FINAL MOMENTS of WORLD #1 MAGNUS CARLSEN vs. WORLD CHAMPION GUKESH https://t.co/s6rJYmnHwH via @YouTube

[Daniel Rensch @DanielRensch]

I knew @LevAronian was lying about this one! 🤣

[MIT CSAIL @MIT_CSAIL]

Happy birthday to Caltech professor Katie Bouman! As a PhD student at MIT she was part of the team that helped create the first-ever image of a black hole.

[Orizuru @OriSenbazuru]

https://t.co/KwJ3MJNKW8

[Grant Sanderson @3blue1brown]

I just put up a new video on the fundamentals of quantum computing, building up to a step-by-step walk-through of something known as Grover’s algorithm. Here's an excerpt of a quiz used to introduce the topic.

[Orizuru @OriSenbazuru]

What could Alphafold 4 look like? (Sergey Ovchinnikov, Ep #3) https://t.co/M9UcjyZPlJ

[Kevin K. Yang 楊凱筌 @KevinKaichuang]

ProGen3: scaling protein language model data and parameters improves the quality of generations, especially further away from natural sequences.

@AadyotB @jeffruffolo @thisismadani

[MIT CSAIL @MIT_CSAIL]

60 years ago this month, the Fast Fourier Transform was created, a powerful tool for image compression & data analysis.

Watch a classic MIT breakdown of FFT, perhaps the most-taught algorithm at the Institute: https://t.co/R7zdspBswx

v/@MITOCW

[Chess.com @chesscom]

9 games, 9 wins @MagnusCarlsen 👏

[Nathan Lambert @natolambert]

First draft online version of The RLHF Book is DONE. Recently I've been creating the advanced discussion chapters on everything from Constitutional AI to evaluation and character training, but I also sneak in consistent improvements to the RL specific chapter.

https://t.co/2oveIOpdCD

RLHF has a long future ahead of it and this will do a lot to make it more accessible to the next generation.

What's next: Getting a physical copy in your hands (may not be exactly 1to1, we'll see) and minor fixes at a slower cadence (thanks to many github contributors, some of you will get a copy from me).

Here are all the chapters.
1.Introduction: Overview of RLHF and what this book provides.
2.Seminal (Recent) Works: Key models and papers in the history of RLHF techniques.
3.Definitions: Mathematical definitions for RL, language modeling, and other ML techniques leveraged in this book.
4.RLHF Training Overview: How the training objective for RLHF is designed and basics of understanding it.
5.What are preferences?: Why human preference data is needed to fuel and understand RLHF.
6.Preference Data: How preference data is collected for RLHF.
7.Reward Modeling: Training reward models from preference data that act as an optimization target for RL training (or for use in data filtering).
8.Regularization: Tools to constrain these optimization tools to effective regions of the parameter space.
9.Instruction Tuning: Adapting language models to the question-answer format.
10.Rejection Sampling: A basic technique for using a reward model with instruction tuning to align models.
11.Policy Gradients: The core RL techniques used to optimize reward models (and other signals) throughout RLHF.
https://t.co/IJ0TeZ6hps Alignment Algorithms: Algorithms that optimize the RLHF objective directly from pairwise preference data rather than learning a reward model first.
13.Constitutional AI and AI Feedback: How AI feedback data and specific models designed to simulate human preference ratings work.
14.Reasoning and Reinforcement Finetuning: The role of new RL training methods for inference-time scaling with respect to post-training and RLHF.
15.Synthetic Data: The shift away from human to synthetic data and how distilling from other models is used.
16.Evaluation: The ever-evolving role of evaluation (and prompting) in language models.
17.Over-optimization: Qualitative observations of why RLHF goes wrong and why over-optimization is inevitable with a soft optimization target in reward models.
https://t.co/BoIMqWyK9h and Information: How RLHF is often underestimated in its role in improving the user experience of models due to the crucial role that style plays in information sharing.
19.Product, UX, Character: How RLHF is shifting in its applicability as major AI laboratories use it to subtly match their models to their products.

[lichess.org @lichess]

Our December 2024 free and open databases https://t.co/NB0uzLwxFG are now available for download, bringing the total number of rated games to over 6.2 billion in standard chess and 125 million across variants!

[Orizuru @OriSenbazuru]

I only have so many tokens left for my patience https://t.co/faZ4RaImEu via @YouTube

[Orizuru @OriSenbazuru]

I Made Chess 2.0 https://t.co/fhOSkvTa8l via @YouTube

[Joel David Hamkins @JDHamkins]

What is a chess piece, mathematically?
I posted an answer to this question some years back on MathOverflow at https://t.co/scgp2PGflR.