reinforcement learning, because the model sequentially learns a policy to solve a problem. It’s also like generative modeling, because it can sample solutions in a way that corresponds to making a probabilistic inference." --Bengio
GFlowNets by Yoshua Bengio's group at the Université de Montréal provides a way for developing neural nets that can reason
arxiv: https://t.co/yIqFmk2V1E
"I’m excited by generative flow networks, or GFlowNets, an approach to training deep nets that my group started about a year ago. This idea is inspired by the way humans reason through a sequence of steps, adding a new piece of relevant information at each step. It’s like ...
Google research released a paper on a neural net that can forecast rain up to 12 hours ahead, as opposed to using time and compute intensive physics simulations on supercomputers. It can run on a colab notebook!
https://t.co/84vpXFlA2Z
Great article on LLMs - MLE training is exactly lossy data compression! I know RLHF is changing the loss to maximize predicted human ratings, but it still feels like a bit like lipstick on a pig to me (although who doesnt like bbq now and then? ;) https://t.co/VBBH11MtB7
Automating mundane tasks should be Artificial Intelligence's ultimate goal, allowing us all to retire earlier. Ironic statement made by @elonmusk, a powerful player in the AI ecosystem:
https://t.co/I7IW7qZFHp
@WallStreetSilv Macron is doing the difficult, but right thing. The retirement age of 62 was set when lifespans were much shorter. It is impossible for a small number of workers to support a massive number of retirees.
Extracting Training Data from Diffusion Models
Extracts over a thousand training examples from SotA models (e.g. Stable Diffusion), ranging from photographs of individual people to trademarked company logos.
https://t.co/uxYo17Tczn
If you think #ChatGPT is going to destroy Google search, have you heard of PaLM from Google Research? It has 540B parameters, almost three times bigger than #gpt3 with 175B parameters
PaLM on AI Tribune 👉 https://t.co/gTds90HJRo
Abstract on Arxiv 👉 https://t.co/9up4C60DTw
Best of AI Twitter - January 11-25 (megathread):
Andrej Karpathy builds "GPT from Scratch",
A class-action lawsuit against Stability AI,
Track human pose using WiFi alone,
An LLM-native IDE,
InstructPix2Pix & AdA,
Training a world-class Rocket-league RL bot,
... and more:
1/22