Cameron Gordon

PhD student at the Australian Institute for Machine Learning, University of Adelaide. A cheerfully dismal scientist.

[Daniel P. Gross @daniel_p_gross]

🚨 New working paper: what happened when the US military pulled back from its broad-based support for university research?

A short 🧵 on our latest work on the Mansfield Amendment era of the 1970s, and what we think we can learn from it.

joint w/ Bhaven Sampat and @HansenZhang13

Ungated copy: https://t.co/4RQNqMEV7L

[Mark Schmidt @MarkSchmidtUBC]

Slides from my ICML tutorial "Is numerical optimization theory irrelevant to machine learning practice in 2026?":
https://t.co/nRmKdzTrdA

(Updated to fix some typos, incorporate feedback, and add some things I did not have time for. Will link to the video here when posted.)

[Peyman Milanfar @docmilanfar]

What do polar coordinates, polar matrix factorization, & Helmholz decomposition of a vector field have in common?

All are implied by Brenier’s Thm: a cornerstone of Optimal Transport theory. It’s a fundamental decomposition result that’s not widely known

1/6

[François Fleuret @francoisfleuret]

It's telling about the teaching of math that even thought I studied a lot of maths, and I am obsessive about nice representations, I never saw that before.

What kind of global culture and educationfailure is that.

[César A. Hidalgo @cesifoti]

1/ Academics often specialize in a few fields. But their impact can extend far into others.

The left network shows the fields of Geoffrey Hinton's papers. The one on the right shows the fields citing these papers.

Dive in to see other examples of such impact. 👇

[Ehsan Pajouheshgar @Esychology]

Neural CAs are amazing, but they've never scaled past low resolution.

We propose a simple solution that allows an ~8x resolution boost with minimal extra parameters.

The core idea: Treat cells as local neural fields instead of pixels.

Try the demo: https://t.co/Nnq8VFOGbB

🧵

[Steven Brunton @eigensteve]

Optimization Bootcamp now live on YouTube!!!
https://t.co/tx6TLLvfUD

Videos will release every few days, so keep tuned!

Book available on Amazon July 16
Free PDF of book soon!

[César A. Hidalgo @cesifoti]

1/ What is the structure of the Research Space?
(the network connecting fields that share authors).

The research space doesn't have a center. It is shaped like a ring connecting the life sciences and engineering through two paths.

[Amiri Hayes @amirihayes_]

What if attention were code? We show that many attention heads in transformer LMs can be replaced by human-readable Python programs.
Swap them in and the model barely notices.

See our experiments here: Explaining Attention with Program Synthesis [https://t.co/tkFopEYtaV]

[Andrej Karpathy @karpathy]

# on shortification of "learning"

There are a lot of videos on YouTube/TikTok etc. that give the appearance of education, but if you look closely they are really just entertainment. This is very convenient for everyone involved : the people watching enjoy thinking they are learning (but actually they are just having fun). The people creating this content also enjoy it because fun has a much larger audience, fame and revenue. But as far as learning goes, this is a trap. This content is an epsilon away from watching the Bachelorette. It's like snacking on those "Garden Veggie Straws", which feel like you're eating healthy vegetables until you look at the ingredients.

Learning is not supposed to be fun. It doesn't have to be actively not fun either, but the primary feeling should be that of effort. It should look a lot less like that "10 minute full body" workout from your local digital media creator and a lot more like a serious session at the gym. You want the mental equivalent of sweating. It's not that the quickie doesn't do anything, it's just that it is wildly suboptimal if you actually care to learn.

I find it helpful to explicitly declare your intent up front as a sharp, binary variable in your mind. If you are consuming content: are you trying to be entertained or are you trying to learn? And if you are creating content: are you trying to entertain or are you trying to teach? You'll go down a different path in each case. Attempts to seek the stuff in between actually clamp to zero.

So for those who actually want to learn. Unless you are trying to learn something narrow and specific, close those tabs with quick blog posts. Close those tabs of "Learn XYZ in 10 minutes". Consider the opportunity cost of snacking and seek the meal - the textbooks, docs, papers, manuals, longform. Allocate a 4 hour window. Don't just read, take notes, re-read, re-phrase, process, manipulate, learn.

And for those actually trying to educate, please consider writing/recording longform, designed for someone to get "sweaty", especially in today's era of quantity over quality. Give someone a real workout. This is what I aspire to in my own educational work too. My audience will decrease. The ones that remain might not even like it. But at least we'll learn something.

[Cameron Gordon @CameronGordon0]

@MahdiKahou Very nice work!

[Sakana AI @SakanaAILabs]

Introducing Sakana Fugu: A full multi-agent orchestration system accessible via a single model API.

Our ‘Fugu Ultra’ model matches the performance of Fable and Mythos, delivering frontier capability without the risk of export controls.

Try it: https://t.co/hhO6qTawgb 🐡

[Yi Ma @YiMaTweets]

Why diffusion denoising-based generative methods do not suffer the curse of dimensionality even though the data may lie in extremely high-dim spaces? Our new work, accepted by the JMLR: https://t.co/njMEqzH3TF reveals the not-so-surprising secret: as long as the intrinsic dimension of the distribution is very low, the generative process can be extremely efficient and effective! It seems that a mixture of low-rank Gaussians is a universal model for all informative real-world data. as we stipulated in a former textbook of mine: Generalized Principal Component Analysis: https://t.co/nEy8qcFN7e, published exactly ten years ago!

[Peyman Milanfar @docmilanfar]

Applying a denoiser twice is essentially applying it once, minus a correction term driven by the residual (the score) and steered by the denoiser’s Jacobian. If you drop the noise level by ε between steps, standard diffusion also adds a temporal drift term:

f (f (x, α), α) ≈ f (x, α) - Jₓ(f) [x − f (x, α)] - [∂f/∂α] ε

The diffusion literature is largely an exercise in controlling these two terms. The Jacobian term is the most geometrically consequential - its spectrum dictates the directions the score can move across the manifold from one step to the next.

To keep the drift term negligible, standard solvers take tiny steps. Flow matching bypasses this by enforcing straight-line trajectories, neutralizing the drift term and reducing the residual to a constant velocity vector. This is what enables high-fidelity sampling in just a few steps.

[Peyman Milanfar @docmilanfar]

Highly under-appreciated fact: Jacobian of your denoiser is the most mathematically consequential part of a diffusion model. Architecture gets all the attention, but the Jacobian governs everything - even if standard training methods mean you never actually have to look at it

[Gabriel Peyré @gabrielpeyre]

The alpha version of my new book "Optimal Transport
for Machine Learners" is out, with in particular an online version with interactive figures
https://t.co/xEdZpMXgjx

[Cameron Gordon @CameronGordon0]

@JosephNWalker This is a good essay.

An additional point is that the risk profile is highly non-symmetric. There's substantial upside for technical spillovers and AI research.

Risks with strategic exclusion / dependence (as seen with Fable) are high.

Domestic capacity is a form of insurance.

[Cameron Gordon @CameronGordon0]

@MahdiKahou Geometrically gives the best intuition. When teaching PCA I first spend some time on Grant Sanderson's 3Blue1Brown eigenvector visualisations. It makes it much easier to see how / why the method works.

https://t.co/Grf1ehDG4B

[Zac Gross @ZacGross]

New working paper on AI and Job displacement!

I use high-frequency, timely Australian administrative data to ask whether AI is already disrupting work.

The answer, at least up to October 2025, is:

No!

Thread.

[Keenan Crane @keenanisalive]

Giving a talk in the @Stanford SCIEN seminar this Wednesday (1/3) at 4:30pm:

https://t.co/EgDLBwv2YU

The topic is “normal coordinates”: a shape representation little-used outside of mathematics—but which turns out to have nice applications in geometry processing & learning.