I don't wanna say "I told you so", but I told you so.
Quote: "Ilya Sutskever, co-founder of AI labs Safe Superintelligence (SSI) and OpenAI, told Reuters recently that results from scaling up pre-training - the phase of training an AI model that uses a vast amount of unlabeled data to understand language patterns and structures - have plateaued." ...
https://t.co/wPzUOQ1Pb3
BREAKING NEWS
The Royal Swedish Academy of Sciences has decided to award the 2024 #NobelPrize in Chemistry with one half to David Baker “for computational protein design” and the other half jointly to Demis Hassabis and John M. Jumper “for protein structure prediction.”
Geoffrey Hinton says he is "flabbergasted" about being awarded the Nobel Prize in Physics and he believes AI will exceed people in intellectual ability so we should worry about it "getting out of control"
BREAKING NEWS
The Royal Swedish Academy of Sciences has decided to award the 2024 #NobelPrize in Physics to John J. Hopfield and Geoffrey E. Hinton “for foundational discoveries and inventions that enable machine learning with artificial neural networks.”
I've read a ton of research papers this year. And to conclude this eventful year in AI, I've compiled a selection of 10 noteworthy papers from 2023 that I am discussing in my new article: https://t.co/FfYERa23g0
To get a sneak peak, I'm covering:
- insights into LLM training runs
- new openly available LLMs
- efficient finetuning methods
- improving "small" LLMs
- pretraining on domain-specific data
- new techniques to align LLMs with human preferences
- enhancing LLMs with high-quality data and instruction sets
- ConvNet vs vision transformer comparisons
- recent developments in image segmentation
- video synthesis with latent diffusion models
I'm wishing you all a great start to 2024!
Is this the end of @Google/@DeepMind as leading presences at machine learning conferences? @JeffDean said: Things had to change. Google would take advantage of its own AI discoveries, sharing papers only after the lab work had been turned into products.
https://t.co/zMKqG64uvG
In the NYT today, Cade Metz implies that I left Google so that I could criticize Google. Actually, I left so that I could talk about the dangers of AI without considering how this impacts Google. Google has acted very responsibly.
Announcing OPT-IML: a new language model from Meta AI with 175B parameters, fine-tuned on 2,000 language tasks — openly available soon under a noncommercial license for research use cases.
Research paper & more details on GitHub ⬇️
Always a pleasure to give a talk at DSC conference. This year DSC is blasting with over 1500 participants and inspiring talks. Thank you Aleksandar Linc-Djordjevic for the invitation and @dsccrew for excellent organization.
Research Class SemTech Lab 20.7.2022. 10:00 O-S32
10:00 Dino Pitoski, Networks and factors of migration flows
10:50 Karlo Babić, RNN with pyramidal embeddings
11:40 Milan Petrović, Graph neural networks: introduction and applications
Congrats to the BigScience team!! 🎉 4 months of training.
More info:
https://t.co/nWr1lOOuCL
Technical logs:
https://t.co/afiPsCvMVC
I believe you can forward on HF Hub, or if you have an 8XA10080GB node lying around :). But offloading work is ongoing, evaluation too. Cool!!
Thank you @uniri for recognising our effort during the #research of COVID-19 communication on social media. It was exciting to work on the @InfoCoV3 project funded by the @hrzz_science.
We will continue our #research as the @SemTechLAB1, FIDIT, @UniRijeka