alex rubinsteyn

Genomics + immunology + ML = personalized cancer immunotherapy | |

[alex rubinsteyn @iskander]

The thing I feel most about coding agents is tremendous gratitude. Every half-finished project that has haunted me for a decade gets to actually come to life

[alex rubinsteyn @iskander]

I think this is a good framing:

Could an unusually bright 15yo totally revolutionize your field? If so, AI will do the same but relentlessly.

On the other hand, does expertise arise from apprenticeship, permission, slow and/or expensive feedback cycles, &c?

[Jake Wintermute 🧬/acc @SynBio1]

Why are there no child prodigies in biology?

This question seems to me to reveal something important about the nature of biological knowledge

It has new relevance as try to distinguish questions that can be answered with pure intelligence from those that require large datasets

[alex rubinsteyn @iskander]

@alphabreacher I think the main catch is how do you get depth / compositional learning without gradients

[alex rubinsteyn @iskander]

Are any of these frontier labs distinguishing themselves by focusing on non-gradient based learning methods or entirely non-neural approaches?

(I’ve been thinking about a few old school ideas for reasoning and LMs, curious how well they scale on modern data/GPUs)

[alex rubinsteyn @iskander]

@FilthyWeeb42069 @andersonbcdefg Wait, they go up to 768gb unified memory? That's crazy

[alex rubinsteyn @iskander]

@DimitrisPapail Seems related to the general decline in writing skills? Novel jargon invented everywhere, often repurposing familiar words in narrower meanings. Huge digressions into details before landing on higher level framing (if ever getting there at all), &c

[alex rubinsteyn @iskander]

@nyttypos @adamwren We will mourn your old ways when they are gone

[alex rubinsteyn @iskander]

I’m afraid to ask but which exact approach does Gary Marcus mean by neurosymbolic?

Like, tool calls to Prolog?

[zeta @zeta_globin]

do you guys ever get stressed out by the fact that your red bone marrow is being replaced by yellow bone marrow as you age

[Kevin Hartnett @KSHartnett]

It strikes me that by the time computers could solve integrals, mathematicians no longer cared about them as research. The move away from explicit computation had happened decades earlier, in the late 19th century.

That period — Riemann, Dedekind, Hilbert — and the climb into algebraic abstraction seems like the right analogy for now.

As AI mows down conjectures (many of which maybe weren't so deep to begin with), the question is what abstraction layer sits above the coming avalanche of individual results.

[alex rubinsteyn @iskander]

@caglar_ee @alphabreacher Do you really get the same kind of decision tree fitting hyperrectangles across the whole dataset? There's something distinct I think about the recursive splits on subsets

[alex rubinsteyn @iskander]

@caglar_ee @alphabreacher It was much faster than the sklearn implementation in 2013 in the era of eg GeForce 590 GPUs, but we didn't maintain or optimize it, or adapt to larger/newer cards. We eventually dropped it once sklearn was tolerably fast but it left me thinking you could really scale up on GPUs

[alex rubinsteyn @iskander]

@caglar_ee @alphabreacher We did two different batching schemes in CudaTree (2013) depending on how deep into splitting you got, wonder if there is a more modern incarnation of that idea

[alex rubinsteyn @iskander]

@alphabreacher Also curious about the many flavors of associative memories that popped up in the 80s and 90s, do any of them show consistent learning and scaling laws when you put them on a H200?

[alex rubinsteyn @iskander]

@alphabreacher One idea I had was looped training of tree model ensembles on target token states and then estimating the optimal new token representation from the error of the concatenation of ensembles.

So like “layers” but for Random Forests

Things like that!

[Daniel Litt @littmath]

Incidentally I am conceding this bet. Strictly speaking it hasn’t resolved (I think we’ve yet to see an Annals-quality number theory paper) but it’s clear I was wrong about what capabilities were necessary to produce one, and it’s just a matter of time.

[alex rubinsteyn @iskander]

@ZhugeYY This was great! …is there a podcast to subscribe to?

[Jared Duker Lichtman @jdlichtman]

Wow! Sphere packing:
"New upper bounds on sphere packing density down to the Cohn–Elkies threshold."

Maryna Viazovska won the Fields Medal for optimality in dimensions d=8 and 24. (In March, Gauss autoformalized it in Lean.)

When I spoke to Maryna, she suggested that it may be possible to approximate Cohn–Elkies for general dimension d, but it sounded like science fiction at the time! (only 4 months ago)

[Noam Brown @polynoamial]

@iamgroguu @DaveShapi We still haven’t solved math. Astra isn’t building new branches of mathematics, or posing interesting new conjectures.

Though I admit it’s hard to believe that tweet was only a year ago. A lot has happened since o3 was released.

[François Fleuret @francoisfleuret]

People are joking with "Gaussian processes" as if it was some kind of ancient arcane ml magic.

I wrote this 17y ago.

https://t.co/ymABMfopF2

[David Pfau @pfau]

Pure math is rare among STEM fields in how little it had been affected by computational advances until now. It's a field that prides itself on still being practiced the same way as Euclid. It's the department that kept chalkboards when everyone else switched to whiteboards.