A primordial Von Neumann Probe, chilling and entropy maxxing | ML research | Pattern hunting across Math, Physics, Systems | CV @Nvidia | Ex @Toyota, @Qualcomm
if you have a qwen3.5 4b around, please try steering neurons L14 N8949 +1 and L13 N7956 -1
now you have evil qwen
recipe: dose × median(‖resid‖ leaving layer L, tokens 1:) / ‖W_down[:, n]‖` at the down_proj at every input
Two years of work, and PySR v2 is finally out!
If you haven't used it before: PySR is a machine learning library that finds short equations that fit your data.
I've always wanted a PyTorch for symbolic learning. PySR v2 is a big step towards that:
📍 Can VLMs actually navigate? Introducing MapReason-OSM: a new benchmark for graph-verifiable mobility decisions on OSM maps. 🗺️ A core use case for autonomous agents on personal devices and their ability to do reasoning on Maps.
Beyond text-matching, it scores structured routes & pins against hidden street graphs across 10 cities.
🚨 Finding: Frontier models "read" maps but fail to "reason" network costs, and they flip decisions when zooming !
#VLM #OSM #SpatialReasoning #AI
We'll be spending a lot more time trying to understand the outputs of language models. A few thoughts, tips & tricks:
Writing. Something I've had success with: Ask your LLM to explain something in ASD-STE100, it's a controlled language specification originally developed for aerospace maintenance documentation. LLMs well-versed in this language and it comes with heavy constraints on clean writing style that I often find a lot more readable. Sometimes I've tried to soften it a bit e.g. ask for "80% of the way to ASD-STE100" because the spec is quite stringent. But even better:
Diagrams / images. Instead of writing, ask your LLM to create a diagram. These can be a lot easier to process, parse, and understand. But even better:
Web pages. Ask for output "in HTML" to get a beautiful, interactive webpage. LLMs are getting really good at frontend and can create beautiful experiences, animations, etc. But even better:
Explainer videos. The output format I am most bullish on is fully custom / bespoke explainer videos generated on any arbitrary topic. Experiment with things like "Create a 3b1b style video explainer on X. Use my ElevenLabs API key for audio narration". (you'd need an API key for the latter or you can ask your LLM to find you decent free alternatives that use your local compute). This is actually starting to work!
In summary:
- As LLMs get better, they will do more and more of the legwork autonomously, and a lot more of our work will rise up the abstractions into oversight and understanding.
- Luckily, LLMs can help here too because as intelligence and code are increasingly abundant, you can ask for large, custom, discardable software artifacts (e.g. web apps, video explainers) that would have never made sense to create before. Push the boundaries here and you'll be surprised.
📍 Can VLMs actually navigate? Introducing MapReason-OSM: a new benchmark for graph-verifiable mobility decisions on OSM maps. 🗺️ A core use case for autonomous agents on personal devices and their ability to do reasoning on Maps.
Beyond text-matching, it scores structured routes & pins against hidden street graphs across 10 cities.
🚨 Finding: Frontier models "read" maps but fail to "reason" network costs, and they flip decisions when zooming !
#VLM #OSM #SpatialReasoning #AI
MapReason-OSM: Can Vision-Language Models Make Graph-Verifiable Mobility Decisions from Street Maps?
Srinivas Venkatanarayanan, Clement Pakkam Isaac
https://t.co/vOZZgu5f9I [𝚌𝚜.𝙲𝚅]
💬Code: https://t.co/rAB21KZSdY
Terence Tao: AI isn’t hype anymore in Math discovery.
Terence Tao is one of the greatest living mathematicians, in his new lecture explains how AI and human professional mathematicians are now complementary.
"There has been a really visible increase in capability. It is not pure hype by any means. To me, these advances show there is a complementary way to do mathematics. Humans traditionally work in small groups on hard problems for months, and we will keep doing that.
But we can also now set AI to scale: sweep a thousand problems and pick up all the low-hanging fruit. Figure out all the ways to match problems to methods. If there are 20 different techniques, apply them all to 1,000 problems and see which ones can be solved by these methods. This is the capability that is present today."
From 'Institute for Pure & Applied Mathematics (IPAM)' YT channel.
Hugo Duminil-Copin, French mathematician and 2022 Field Medalist told me he never participated in math competition and was very bad at it.
Innovative mathematics requires creativity, intuition, intense concentration, and long reflections, sometimes spread over several years.
Good performance at a math olympiad merely tests fast problem solving abilities. AI can do that nowadays.
One of the big activities of a researcher, in mathematics and elsewhere, is not to answer questions but to ask the right questions.
We take randomness for granted.
Early PRNGs were BAD.
Thousands of scientific papers used to rely on RANDU, created by IBM in the 1960s. In 1D space, it looks ok!
Map in 3D…you start to see the issues. Now, there *was* a better solution...but it would cost you.
Transformers require explicit positional encoding with global reference. While in nature, processes like morphogenesis happens without a global system of reference. Positional awareness with concentration gradients would be an interesting development in Transformers.
There are internal models of reasoning and understanding that is different for different authors, thats why you have interpretations of linear algebra from Shilov to Strang each different on different intuitions for same formalisms
As much as Jensen is my super boss and i absolutely respect him, I don’t align to this well enough. There is a qualia even for objective knowledge, the deep intuition a human has on a mathematical or a scientific concept which you get to understand that’s impossible from an AI
Absolutely love Jensen Huang's prediction on this 🎯
“In 2-3 years, 90% of the world’s knowledge will be generated by AI. ”
whether we learn from books written by unknown people or AI that is combining assimilating knowledge, it makes little difference
What is the future of intelligence? The answer could lie in the story of its evolution.
Nice essay by @blaiseaguera published in @Nature.
Essay: https://t.co/dh1Zu0QElI
Some nice ideas in the piece:
“Hunting is a prime example of this predictive modelling. A predator must predict actions that will get the prey into its stomach; the prey must predict the predator’s behaviour to stop that from happening. Starting in the 1970s, neuropsychologists and anthropologists began to realize that other intelligent entities are often the most important parts of the environment to model — because they are the ones modelling you back, whether with friendly or hostile intent. Increasingly intelligent predators put evolutionary pressure on their prey to become smarter, and vice versa.”
“Humans did not invent computation any more than they did electric current or optical lenses. We merely re-discovered a phenomenon nature had already exploited, developed mathematical theories to understand it better and worked out how to engineer it on a different substrate.”
Why do some corals grow into tall and narrow columns, and others into massive domes? The answer is buried deep in each coral polyp’s biological programming.
https://t.co/uuBli2Min8