o3 is really special and everyone will need to update their intuition about what AI can/cannot do.
while these are still early days, this system shows a genuine increase in intelligence, canaried by ARC-AGI
semiprivate v1 scores:
* GPT-2 (2019): 0%
* GPT-3 (2020): 0%
* GPT-4 (2023): 2%
* GPT-4o (2024): 5%
* o1-preview (2024): 21%
* o1 high (2024): 32%
* o1 Pro (2024): ~50%
* o3 tuned low (2024): 76%
* o3 tuned high (2024): 87%
given i put in the original $1M @arcprize, i'd like to re-affirm my previous commitment. we will keep running the grand prize competition until an efficient 85% solution is open sourced.
but our ambitions are greater! ARC Prize found its mission this year -- to be an enduring north star towards AGI.
the ARC benchmark design principle is to be easy for humans, hard for AI and so long as there remain things in that category, there is more work to do for AGI.
there are >100 tasks from the v1 family unsolved by o3 even on the high compute config which is very curious.
successors to o3 will need to reckon with efficiency. i expect this to become a major focus for the field. for context, o3 high used 172x more compute than o3 low which itself used 100-1000x more compute than the grand prize competition target.
we also started work on v2 in earnest this summer (v2 is in the same grid domain as v1) and will launch it alongside ARC Prize 2025. early testing is promising even against o3 high compute. but the goal for v2 is not to make an adversarial benchmark, rather be interesting and high signal towards AGI.
we also want AGI benchmarks that can endure many years. i do not expect v2 will. and so we've also starting turning attention to v3 which will be very different. im excited to work with OpenAI and other labs on designing v3.
given it's almost the end of the year, im in the mood for reflection.
as anyone who has spent time with the ARC dataset can tell you, there is something special about it. and even moreso about a system than can fully beat it. we are seeing glimpses of that system with the o-series.
i mean it when i say these are early days. i believe o3 is the alexnet moment for program synthesis. we now have concrete evidence that deep-learning guided program search works.
we are staring up another mountain that, from my vantage point, looks equally tall and important as deep learning for AGI.
many things have surprised me this year, including o3. but the biggest surprise has been the increasing response to ARC Prize.
i've been surveying AI researchers about ARC for years. before ARC Prize launched in June, only one in ten had heard of it.
now it's objectively the spear tip benchmark, being used by spear tip labs, to demonstrate progress on the spear tip of AGI -- the most important technology in human history.
@fchollet deserves recognition for designing such an incredible benchmark.
i'm continually grateful for the opportunity to steward attention towards AGI with ARC Prize and we'll be back in 2025!
OpenAI's o1 "broke out of its host VM to restart it" in order to solve a task.
From the model card: "the model pursued the goal it was given, and when that goal proved impossible, it gathered more resources [...] and used them to achieve the goal in an unexpected way."
We're releasing a preview of OpenAI o1—a new series of AI models designed to spend more time thinking before they respond.
These models can reason through complex tasks and solve harder problems than previous models in science, coding, and math. https://t.co/peKzzKX1bu
This year, the ASF is celebrating 25 years of #opensource collaboration and innovation. Here's a look at the first 10 ASF projects, including HTTP Server, the ASF's founding project. https://t.co/3S4DgPLoJx
#ASF25Years
10 EU cities won the Label of the EU Mission for Climate-Neutral and Smart Cities:
🇷🇴Cluj-Napoca
🇦🇹Klagenfurt
🇩🇪Mannheim
🇩🇰Sønderborg
🇸🇪Stockholm
🇪🇸Madrid, Valencia, Valladolid, Vitoria-Gasteiz, Zaragoza
It acknowledges their commitment towards climate neutrality. #EUGreenDea10 EU cities won the Label of the EU Mission for Climate-Neutral and Smart Cities:
🇷🇴Cluj-Napoca
🇦🇹Klagenfurt
🇩🇪Mannheim
🇩🇰Sønderborg
🇸🇪Stockholm
🇪🇸Madrid, Valencia, Valladolid, Vitoria-Gasteiz, Zaragoza
It acknowledges their commitment towards climate neutrality. #EUGreenDea10 EU cities won the Label of the EU Mission for Climate-Neutral and Smart Cities:
🇷🇴Cluj-Napoca
🇦🇹Klagenfurt
🇩🇪Mannheim
🇩🇰Sønderborg
🇸🇪Stockholm
🇪🇸Madrid, Valencia, Valladolid, Vitoria-Gasteiz, Zaragoza
It acknowledges their commitment towards climate neutrality. #EUGreenDea10 EU cities won the Label of the EU Mission for Climate-Neutral and Smart Cities:
🇷🇴Cluj-Napoca
🇦🇹Klagenfurt
🇩🇪Mannheim
🇩🇰Sønderborg
🇸🇪Stockholm
🇪🇸Madrid, Valencia, Valladolid, Vitoria-Gasteiz, Zaragoza
It acknowledges their commitment towards climate neutrality. #EUGreenDea10 EU cities won the Label of the EU Mission for Climate-Neutral and Smart Cities:
🇷🇴Cluj-Napoca
🇦🇹Klagenfurt
🇩🇪Mannheim
🇩🇰Sønderborg
🇸🇪Stockholm
🇪🇸Madrid, Valencia, Valladolid, Vitoria-Gasteiz, Zaragoza
It acknowledges their commitment towards climate neutrality. #EUGreenDea10 EU cities won the Label of the EU Mission for Climate-Neutral and Smart Cities:
🇷🇴Cluj-Napoca
🇦🇹Klagenfurt
🇩🇪Mannheim
🇩🇰Sønderborg
🇸🇪Stockholm
🇪🇸Madrid, Valencia, Valladolid, Vitoria-Gasteiz, Zaragoza
It acknowledges their commitment towards climate neutrality. #EUGreenDeal
“Ten years ago I was kicked out and forced to retire.”
Our new medicine laureate Katalin Karikó (@kkariko) told us how much it means to be awarded the Nobel Prize after a scientific career that has been full of challenges.
Ten years ago, Karikó was still doing all her experiments by hand but today she has been awarded the medicine prize for her research on mRNA, which led to the development of COVID-19 vaccines.
Listen now:
Curious about how to identify & implement #DigitalAgriculture policies and best practices? Check out DG’s Digital Agriculture Knowledge Management Library, developed in our #DASProgram alongside our partners @JengaLab, @TechChange, & @IFAD.
Learn more: https://t.co/pNFq6TqHAb
An honour to meet @POTUS@JoeBiden and @FLOTUS@DrBiden on the sidelines of #UNGA78.
As Moldova consolidates its democracy, I deeply appreciate the enduring strategic partnership and close ties with the U.S.
I welcome today’s @EU_Commission decision to end the Cooperation & Verification Mechanism #CVM for 🇷🇴, as a result of our reform efforts in this key area of European integration. We remain determined to uphold rule of law at home&abroad, delivering on our 🇪🇺 commitments&values.
Our position on Romania and Bulgaria fulfilling the Schengen criteria is clear.
And the message to the EU countries is also very clear.
Romania and Bulgaria belong to Schengen. So let us finally bring them in – without any further delay.
Next #opencontracting rockstar at #OGPEstonia is @DGateway's Charlene Migwe-Kagume. Got to focus on the initial design of digital systems to be user-friendly & interoperable, with a focus on use of information right at the start. Shares examples from her work in Kenya, what works & what hasn't...
Omg omg omg float16 in webGPU finally works now and it's ~fast.
Even on a $700 laptop with just 8gb ram.
Fully local, fully controlable, web-based ai is here (in beta/Chrome Canary).
Yes it works offline.
We’re hosting a panel discussion at #AGRF2023! The discussion will include experts exploring innovative technologies & strategies to address needs, challenges, & opportunities within the agricultural sector in Africa.
RSVP now: https://t.co/e1MB5JZYbz!
Important article: https://t.co/U1l5BfTgv4 The single most important data point that suggests "progress is unlikely to slow in the next 2-3y": GPT-4 cost ~$100M (probably less), and Alphabet has 1000x that much money in cash on hand:
Noua Lege pentru Baterii Sustenabile și Circulare a intrat în vigoare! Reglementarea noilor baterii asigură o amprentă de carbon redusă, substanțe nocive minime și reciclare eficientă. Acest pas susține trecerea la o economie circulară. Detalii: https://t.co/utUoZvUtay #baterii
Russia's continued attacks against the Ukrainian🇺🇦 civilian infrastructure on #Danube, in the proximity of Romania🇷🇴, are unacceptable. These are war crimes and they further affect UA's capacity to transfer their food products towards those in need in the world.
Everyone's pivoting to generative AI.
But my alarm bells go off when I see:
🚩 A crowded landscape
🚩 FOMO driven decision making
🚩 Sky high valuations for an early space
Here are 5 questions to ask to understand if a gen AI product will be successful: