Ben Crulis

AI, computer vision, science

[Ben Crulis @BenCrulis]

@tonylfeng LLMs are superhuman in some aspects, but are not even human level in the ones that matter to be considered AGI, IMO.
I am talking about continual learning, mostly.

[Ben Crulis @BenCrulis]

@gabriberton LLMs are not even bitter lesson pilled according to Sutton himself.
But yeah, people still mistakenly believe it applies this way.

[Ben Crulis @BenCrulis]

@VictorTaelin I would be already wary enough of a free proprietary programming language, but there is no chance at all I would use a subscription based one, no matter how good it is. Unless maybe I am paid by a company to use it, of course.

[Ben Crulis @BenCrulis]

@TrisH0x2A What about stopping at the first mismatch, but compare bytes in a different random order at each call?
Maybe not as secure as comparing every byte, now that I think about it.

[Ben Crulis @BenCrulis]

@gabriberton The brain does not seem to be doing backprop, but it might still compute the same thing as backprop (perhaps approximately). Backprop is just here for credit assignment, which the brain also needs.
But yeah, JEPA and HRM are about as inspired by the brain as anything else in DL.

[Ben Crulis @BenCrulis]

@fraserpricee It is old but you should take a look at project Tierra by Thomas S. Ray as well as the more recent SALIS simulation that it inspired (great video series on youtube).
For recent academic research, maybe look into what was published at the Alife conference.

[Ben Crulis @BenCrulis]

@tautologer There are other materialist options at least. For instance, electromagnetic consciousness or the theory of Andrés Gómez Emilsson.
Functionalism is just one materialist theory of consciousness and it is not even the best one IMO.

[Ben Crulis @BenCrulis]

@VictorTaelin Finding ideas to test is easy, if only I had compute.
Last time I ran a GPU intensive training run, it nearly killed my PC.
Compute is in the hands of people that are afraid of betting on something new.

[Ilir Aliu @IlirAliu_]

The man who invented RRT put his entire textbook ONLINE FOR FREE.

If a robot moves from A to B without hitting anything, there's a good chance it's using an algorithm Steve LaValle created.

The Rapidly-Exploring Random Tree (RRT) is inside autonomous vehicles, robotic arms, surgical robots, and space systems worldwide.

LaValle then wrote the definitive book on planning algorithms. 826 pages. Published by Cambridge University Press.

Then he put the whole thing on his personal website. Free. No login. PDF.

It covers everything: motion planning, planning under uncertainty, sensor-based planning, game theory, reinforcement learning, trajectory planning, and nonholonomic systems.

The full picture of how robots decide where to go and how to get there.

Free at https://t.co/yhZuA7kKvu

Follow for more robotics classics & free resources like this.

——

Weekly robotics and AI insights.
Subscribe free: https://t.co/9Nm01QUcw3

[Ben Crulis @BenCrulis]

@VictorTaelin @lookingforgames "optimize the logic for size, keep everything else the same" maybe?

[Ben Crulis @BenCrulis]

@francoisfleuret RNNs + agentic search seems like a good combo for maximum efficiency.

[Ben Crulis @BenCrulis]

@lost_in_tech @BambulabGlobal Aren't they shrink lines? It matches the external ridges, maybe there is internal geometry that makes the layer times different at these heights.

[Ben Crulis @BenCrulis]

@_fracapuano I find it a bit ironic that to avoid pretraining on a proxy-task, JEPA also ends up relying on a proxy task, which is learning embeddings invariant to a set of arbitrarily chosen data augmentations.
If you choose the invariants badly, you will also get poor representations.

[Ben Crulis @BenCrulis]

@DeepDishEnjoyer The UAT alone does not explain why we can find ~0 loss models this easily with SGD and why solutions we usually find generalize so well (for some things).
The UAT just says that a model fitting the data exists in the hypothesis class if the architecture is large enough.

[Ben Crulis @BenCrulis]

@nic_carter The way I see it, LLMs are general in the same sense that assembling a bunch of specialized AIs together into one system is general.
But the number of different tasks a system can do is actually irrelevant, only the ability to learn to solve novel tasks is. LLMs fail at that.

[Ben Crulis @BenCrulis]

@jiaxinwen22 @segyges Interesting, what did you try?

[Ben Crulis @BenCrulis]

@jiaxinwen22 You should take a look at the research work of the late Naftali Tishby on deep learning.

[Ben Crulis @BenCrulis]

@TrueAIHound Frame prediction models are already susceptible to the same kinds of illusions that fool us https://t.co/BYQr9fmcUJ

[Ben Crulis @BenCrulis]

@jordyw35 @GaryMarcus The human brain managed to do it, and it is probably not even optimal.

[Ben Crulis @BenCrulis]

@jordyw35 @GaryMarcus Hardware will continue to improve, but most importantly, training algorithms will improve so that training becomes sufficiently efficient.
For instance, the continual learning algorithm might involve filtering irrelevant information in real time, to learn only the relevant.