Great discussion on self supervised learning with awesome people! Thank you all for attending. I enjoyed and learned a lot.๐๐ really impressed by more than 400 people attending on such a special topic:)
The @OpenAI retrieval API seems to be doing basic top-k RAG on limited context if there's context overflows.
Asking for a summary of the Uber 10-K (~150k-200k tokens), returns oddly specific stuff around acquisitions and legal proceedings ๐
Huge day indeed for AI and LLMs, congrats to Meta ๐
This is now the most capable LLM available directly as weights to anyone from researchers to companies.
The models look quite strong, e.g. Table 4 in the paper: MMLU is good to look at, the 70B model is just below GPT-3.5. But HumanEval (bad misnomer) shows coding capability is quite a bit lower (48.1 vs 29.9).
I'm excited about Segment Anything released from FAIR today. It tackles an old problem (find objects in images) at large scale: trained on 11M images and 1B objects.
This is a new Foundation Model for Computer Vision - it recognizes any object in any context.
Excited to share the first of a series of @GoogleAI blog posts summarizing our research work from 2022. This covers language & multimodal models, computer vision, and generative models. We'll have ~7 posts covering other areas over next few weeks!
https://t.co/WuHSZ6qKKu
This is a great thread, +1e100. @karpathy didn't mention my (biased!) favorite example of 2021. Models designed to generate words (transformers) & model language (BERT) were reused in #AlphaFold to solve the protein folding problem, mapping a bunch of letters, to 3D coordinates๐คฏ