JAS

Oye, pues sí. Me gusta enseñar a máquinas

[BuBBliK @k1rallik]

> Google publishes ARC-AGI-3 benchmark on March 25
> no hype, no product launch, just a grid and a timer
> 48 hours later:
> GPT-5: 0.26%, Claude: 0.25%, Grok: 0%
> a 4-layer CNN built in weeks: 12.58%
> Jensen Huang 2 days before: "I think we've achieved AGI"
> random unemployed people from SF streets: 100%
> one dev replicates the winning approach over a weekend
> analysts say the benchmark is unfair
> $700K prize still untouched
> DRAM stocks drop, AI bubble discourse returns

the number is 0.26%. Remember it next time someone says AGI is already here.

[Marcin Krzyzanowski @krzyzanowskim]

my productivity skyrocketed when I stopped setup agents and started to write code again like a normal fucking human being

[Mehrdad Farajtabar @MFarajtabar]

1/ Can Large Language Models (LLMs) truly reason? Or are they just sophisticated pattern matchers? In our latest preprint, we explore this key question through a large-scale study of both open-source like Llama, Phi, Gemma, and Mistral and leading closed models, including the recent OpenAI GPT-4o and o1-series.
https://t.co/2tv8Pp9MSz

Work done with @i_mirzadeh, @KeivanAlizadeh2, Hooman Shahrokhi, Samy Bengio, @OncelTuzel.

#LLM #Reasoning #Mathematics #AGI #Research #Apple

[Santiago @svpino]

AI won't kill software developer jobs; it will multiply them.

Every person using AI to write code that doesn't work or scale creates new jobs for real developers who know how to fix the mess.

[Firefox 🔥 @firefox]

While everyone is solving for X, we'll just be here minding our own business and protecting your privacy 🙂

[JAS @ja_sdv]

@rasbt Congrats! Any plans for a printed version?

[AK @_akhaliq]

Faith and Fate: Limits of Transformers on Compositionality

Transformer large language models (LLMs) have sparked admiration for their exceptional performance on tasks that demand intricate multi-step reasoning. Yet, these models simultaneously show failures on surprisingly trivial problems. This begs the question: Are these errors incidental, or do they signal more substantial limitations? In an attempt to demystify Transformers, we investigate the limits of these models across three representative compositional tasks -- multi-digit multiplication, logic grid puzzles, and a classic dynamic programming problem. These tasks require breaking problems down into sub-steps and synthesizing these steps into a precise answer. We formulate compositional tasks as computation graphs to systematically quantify the level of complexity, and break down reasoning steps into intermediate sub-procedures. Our empirical findings suggest that Transformers solve compositional tasks by reducing multi-step compositional reasoning into linearized subgraph matching, without necessarily developing systematic problem-solving skills. To round off our empirical study, we provide theoretical arguments on abstract multi-step reasoning problems that highlight how Transformers' performance will rapidly decay with increased task complexity.

paper page: https://t.co/TYtpSvMvrx

[nuriaoliver @nuriaoliver]

Some say that #AI poses an #existential #risk to #humanity. To which humanity? to the 3.6 billion people who are highly vulnerable due to climate change? to the ~1 billion people who live on less than 1 USD/day? To the 2 billion people affected by war, conflict and fragility?

[JAS @ja_sdv]

@karpathy I don't think we can talk to anything close to AGI until some trivial tasks like L-5 autonomous driving are solved. So yeah.

[François Chollet @fchollet]

Q1-Q2 2023 is the peak of the AI bubble.

[Christian Szegedy @ChrSzegedy]

While AI is the most transformative technology we had since
Internet, I think its effect will be much less direct than some expect/advertise, in the sort/medium run and maybe beyond that, even if we achieve several impactful breakthroughs.

[Kevin Fischer @kevinafischer]

I don't talk much about this - I obtained one of the first FDA approvals in ML + radiology and it informs much of how I think about AI systems and their impact on the world. If you're a pure technologist, you should read the following:

There's so much to unpack for both why Geoff was wrong, and why his future predictions should not be taken seriously either.

Geoff made a classic error that technologists often make, which is to observe a particular behavior (identifying some subset of radiology scans correctly) against some task (identifying hemorrhage on CT head scans correctly), and then to extrapolate based on that task alone.

The reality is that reducing any job, especially a wildly complex job that requires a decade of training, to a handful of tasks is quite absurd.

Here's a bunch of stuff you wouldn't know about radiologists unless you built an AI company WITH them instead of opining about their job disappearing from an ivory tower.

(1) Radiologists are NOT performing 2d pattern recognition - they have a 3d world model of the brain and its physical dynamics in their head. The motion and behavior of their brain to various traumas informs their prediction of hemorrhage determination.

(2) Radiologists have a whole host of grounded models to make determinations, and actually, one of the most important first order determination they make is whether there is anything notably wrong with a brain structure that "feels" off. As a result, classifiers aren’t actually performing the same task even as radiologists.

(3) Radiologists, because they have a grounded brain model, only need to see a single example of a rare and obscure condition to both remember it and identify it in the future. This long tail of rare conditions to avoid missing is a large part of their training, and no one has any clue how to make a model that acts similar in this way.

(4) There’s so many ways to make Radiologist lives easier instead of just replacing them, it doesn’t even make sense to try. I interviewed and hired 25 radiologists, whose primary and chief complaint was that they had to reboot their computers several times a day.

(5) A large part of the radiologist job is communicating their findings with physicians, so if you are thinking about automating them away you also need to understand the complex interactions between them and different clinics, which often are unique.

(6) Every hospital is a snowflake, data is held under lock and key, so your algorithm might not work in a bunch of hospitals. Worse, the imagenet datasets have such wildly different feature sets they don’t do much for pretraining for you.

(7) Have you ever tried to make anything in healthcare? The entire system is optimized to avoid introducing any harm to patients - explaining the ramifications of that would take an entire book, but suffice to say even if you had an algorithm that could automate away radiologists I don’t even know if you could create a viable adoption strategy in the US regulatory environment.

(8) The reality is that for every application, the amount of specific and UNKNOWABLE domain knowledge is immense.

LONG STORY SHORT: thinkers have a pattern where they are so divorced from implementation details that applications seem trivial, when in reality, the small details are exactly where value accrues.

Should you be worried about GPT5 being used to automate vulnerability detection on websites before they’re patched? Maybe.

Should you be worried GPT5 is going to interact with SOCIAL systems and destroy our society single-handedly? No absolutely not.

[Yann LeCun @ylecun]

Haha.
Auto-Regressive LLMs gonna auto-regress.
Your hands must remain on the keyboard at all time.
Level-2 Writing assistance? Yes!
Level-5 autonomous writing? No!

"Here’s What Happens When Your Lawyer Uses ChatGPT"
https://t.co/j5QDLzyNCt

[Marc Skov Madsen, PhD, CFA® @MarcSkovMadsen]

Finally! @Panel_org 1.0 supports reacting to events on ECharts.

👉 Check out https://t.co/oVagWHNuR4

@ApacheECharts #python #databiz #datascience #analytics

[Yann LeCun @ylecun]

MMS: Massively Multilingual Speech.
- Can do speech2text and text speech in 1100 languages.
- Can recognize 4000 spoken languages.
- Code and models available under the CC-BY-NC 4.0 license.
- half the word error rate of Whisper.

Code+Models: https://t.co/NIGfUZ8KZg
Paper: https://t.co/W15aEWHGIR
Blog: https://t.co/TFKXFtlPwc

[JAS @ja_sdv]

@chrisalbon Wait to try StarCoder. IMO it's way better than GPT4. GOT4 slowed me down actually

[hardmaru @hardmaru]

AI Twitter these days. 👇🧵

[JAS @ja_sdv]

@simonw No there isn't. All you have is a bunch of cherry-picked tweets and their webpage "demos". Isn't that enough by today's standards?

[Stanford HAI @StanfordHAI]

Foundation models are powerful machine learning algorithms that sit at the core of many generative AI tools today. How are they built and deployed and how are they changing society? Here’s a quick intro for non-experts:

[JAS @ja_sdv]

@ykilcher The problem is the amount of effort required. If anyone can publish any random text "for free", then there is no value in text anymore. Creating fake news from scratch is a tiresome process that barely pays out. But if the barrier of entry dissappears, then we have a problem