Some fields work in theory but not in practice. Some fields work in practice but not theory.
The uniqueness of economics is that it works in neither theory nor practice.
One of the most jarring things about current AI is its lack of introspection ability and metacognition. It doesn't know what it doesn't know, how it knows, or how it could find out. It's a one-way system.
I am delighted to see a new version of the book by @_sdbuchanan, @druv_pai , @pengwang2003 and @YiMaTweets . This is the best book on the foundations of deep representation learning! In this era of coding agents, the math is all you need to learn :)
https://t.co/3IvoZeFUYA
The Claude C Compiler is the first AI-generated compiler that builds complex C code, built by @AnthropicAI. Reactions ranged from dismissal as "AI nonsense" to "SW is over": both takes miss the point.
As a compiler🐉 expert and experienced SW leader, I see a lot to learn: 👇
Amazing test of Gemini 3’s multimodal reasoning capabilities: try generating a threejs voxel art scene using only an image as input
Prompt: I have provided an image. Code a beautiful voxel art scene inspired by this image. Write threejs code as a single-page
Here @stephen_wolfram accurately points out that scientists that come up with new ideas & frameworks are more often the experienced ones, not the young as we tend to believe.
https://t.co/EGWePVREme
Oldies but goldies: J Blinn, A Generalization of Algebraic Surface Drawing, 1982. "Metaballs" are levelsets of mixtures of radial basis functions, which generate blobby lava-lamp-like behaviors. https://t.co/yOpB1Or5Uf
Mike Jordan gave an excellent talk on the need to take the multi agent setting as central to AI, because humans and companies have different information states and incentives, and thus centralized “AGI” is inevitably suboptimal and potentially socially harmful.
I'm pleased to announce a new blog post from @RuiqiGao and some other folks on my team, showing that diffusion == Gaussian flow matching.
https://t.co/6AwqPiktu0
A surprising number of people have asked me versions of "Is the Physics Nobel Prize today really for Physics?"
What counts as a field is surprisingly complicated. As a rough and incomplete classification, a field can be:
1. Based on exploration of and development of a set of agreed-upon deep principles. The Maxwell-Lorentz equations, quantum field theory, general relativity, and quantum computing are each (separately!) examples of fields in this sense
2. Based on exploration of some (more or less) agreed-upon set of questions. The search for a basic fundamental theory and quantum information are both examples of fields having this flavour. The questions tend to shift over time, sometimes substantially, and such fields sometimes fission or fusion or change substantially as a result
3. A group of people investigating a common domain of application. Examples of fields with this flavour are atomic physics, optical physics, and condensed matter physics. This can be viewed as a case of 2. It's striking how much internal variation there can be - understanding (say) the fractional quantum Hall effect is very different than understanding spin glasses, yet both are part of condensed matter physics. In some sense the question behind condensed matter unifies many different fields of type 1
4. A philosophical, organizational, and political treaty among fields of types 1-3. That's what "Physics" is, in the sense of the prize. It's interesting that antenna design is currently not regarded as part of Physics (it's more EE), while quantum computing to some extent is part of Physics. That's partly a contingent choice: it could have been different if history had just been a tiny bit different. However, to some extent it also represents some general philosophy of "what Physics is about". Ideas like quantum error-correction and topological quantum computers required deep fundamental insights into physics. My guess - it's just a guess - is that, over time, quantum computing will become more detached from Physics, as it becomes more and more commercial, and more and more engineering
Today's Physics prize falls outside the usual type 4 philosophical, organizational, and political treaty of "what physics is". People at Caltech used to tell me that John Hopfield had "left physics", and that's why he'd gone to Princeton (from Caltech). But then, a lot of physicists in the 1990s didn't think quantum computing was part of physics. I'm sure some still don't. So: it really is somewhat contingent
My own point of view: there is just one nature. I'm delighted when people have and share deep insights into nature, and I don't care so much what we label it. I'm especially delighted by the incredible progress in the past few decades in developing the design sciences. That is: understanding the fundamental principles underlying the incredible systems latent in nature, and which we humans are gradually learning to build. John Hopfield and Geoff Hinton have made enormous contributions to understanding what possibilities lie latent in nature. It so happens that their work falls largely outside the usual Nobel classification, but I am happy to celebrate them for their remarkable contributions, and physics seems as apt an area as any
Congratulations to them both!
The world’s architects need to be investigated immediately. Think of the countless awful, illegal things that have taken place in rooms they’ve designed and built. Those who installed secret government cameras and microphones into the walls are okay but the rest should be jailed.
Identifying the true structure of the latent data generating process (causal world model) by learning from passive observation alone is impossible without reasonable priors, which transformers don’t have. This is why LLMs are brittle and can’t reliably do counter factual reasoning .
One of the coolest lesser-known Keras feature is the FeatureSpace API: a one-stop shop for tabular data preprocessing. It takes care of indexing, hashing, encoding, normalizing, and even crossing any type of categorical or numerical feature you have.
You can use inside a data pipeline, or directly inside of a Model, since it acts as a Keras layer. It's also saveable just like a model.
I am happy to share our new ICML paper on WOLF, the "Weighted Observation Likelihood Filter", which is a simple way to robustify many existing Bayesian filtering techniques (such as extended and ensemble Kalman filter). It uses generalized Bayes to replace the likelihood function with a robust loss function, which avoids the need to define a generative model of the outlier process. This is simple, fast, and provably robust.
Details in thread below.
Paper: https://t.co/VkMCdnzlsK
Code: https://t.co/UPeEYgv56z
Hiring & promotion decisions should always be meritocratic, both because that's what's in your organization's best interest, and because that's what maximizes the flourishing of human potential.
My views on nature vs nurture:
1. Humans are intelligent from the very beginning. From the first year of their life, they have a high ability to adapt to new situations and efficiently acquire new skills.
2. Almost all knowledge is acquired. Humans are born with very little prior knowledge about the world. They don't have a pre-trained visual system -- they must learn to see. They don't have a pre-trained gait-generation program -- they must learn to walk. Etc.
3. Early knowledge/skills acquisition is strongly guided by metacognitive priors, which is why everyone acquires essentially the same Core Knowledge foundations, under roughly the same timeline, despite experiencing very different environments.
4. How intelligent you are is largely determined by your genes -- it's how your brain is wired. Environmental factors (pollutants, malnutrition, sensory deprivation, trauma...) can lower your intelligence but cannot meaningfully increase it.
5. Intelligence decreases with age. My 3 year old is smarter than me. Which makes perfect sense, because as you age you can trade-off fluid intelligence for crystallized skills. The less you know the more valuable intelligence is.
I’ve never understood why modern nonconvex line search did not become more popular, since it’s clearly hard to find a single step size which is good throughout the loss landscape, even when using Adam-style diagonal scaling.