Pabnau

Math PhD: data scientist by trade, mathematician at heart. Interested in science and skepticism.

[Pabnau @pabnau]

@NateSilver538 From your book, it honestly feels like you have a weird caricature of risk-averse agreeable academics in your head. At least in the hard science, these are people who went into PhD programs which often have ~30% success rates, and they can gain clout by proving each other wrong

[Pabnau @pabnau]

@NateSilver538 I find it to be quite the opposite tbh: that science journalists tend to uncritically accept what researchers say. That’s on top the of the issue that they rarely have the expertise to understand a research paper: so researchers have to baby it down so they can understand

[Pabnau @pabnau]

@TonyTheLion2500 It means there exists a way of continuously deforming your space/object without changing the underlying structure. For example, the laws of physics don’t care if we scooch everything in the universe to the right, so the magic of Noether’s theorem gives us conservation of momentum

[Pabnau @pabnau]

@TonyTheLion2500 That makes sense. I think the advantage of the infinitesimal approach is that it can be more intuitive to some people and can be more direct because doesn’t require you to build as much of a mathematical background. On the other hand, it feels less mathematically natural to me

[Pabnau @pabnau]

@TonyTheLion2500 Not surprising that the cited book was written by computer scientists. imo this is a bizarre way of phrasing it, intended for shock value. The definition of random variables tries to model random events. But like any math concept, if you strip out the context, you lose it

[Pabnau @pabnau]

@SashaMTL Of note, since the title of the figure was omitted, that is the total electricity consumption of data centers for all purposes. As mentioned in the article, the largest driver of this is an enormous 25-30% average yearly growth in global IP traffic over the last 8 years

[Pabnau @pabnau]

@TonyTheLion2500 In my experience, it could mean either they are sketchily canceling infinite values from a summation because they can’t be bothered to do it carefully, or it could be one of the deeper gaps in math physics like renormalization, where they get the right answer through invalid math

[Pabnau @pabnau]

@galendruke Oh no, I was worried this might happen eventually. You’re an absolute gem Galen, and the main reason I kept listening to the podcast after the last layoff. I hope I will still get to hear or read from you

[Pabnau @pabnau]

@SashaMTL Do you have a source for that? As far as I can find, its architecture is a standard MoE, which is more efficient at inference than a traditional transformer model of the same size

[Pabnau @pabnau]

@kareem_carr Ha, yeah, you generally want to avoid hot loops in Python. Not the languages strong suit

[Pabnau @pabnau]

@kareem_carr For real! Every time people just go nuts extrapolating exponential growth far into the future and making wild claims

[Pabnau @pabnau]

@kareem_carr It would probably be a fun exercise, but might be better to keep it private. Election modeling seems like a great way to get every partisan to hate you. Whenever the model tells them something they don’t want to hear, they’ll just claim you’re financial compromised or whatever

[Pabnau @pabnau]

@kareem_carr Generally speaking, ensembles help a little bit, but the errors ML models make can be highly correlated, especially if they are of a similar type, so it usually isn’t sufficient to push a method from unreliable to reliable enough

[Pabnau @pabnau]

@kareem_carr Haha, yeah…very true. Also improved pedagogy and notation. Not fun reading the equations-written-as-complete sentences of the Arabic mathematicians from 1000 years ago

[Pabnau @pabnau]

@sp_monte_carlo You’re missing out. You could be one of those people who uses the loosest possible constant in your bounds, then promote it to the media to run one of their cookie-cutter “look how big this number is” stories

[Pabnau @pabnau]

@sp_monte_carlo Oh, well that is somewhat reassuring. I found the same result in some specific cases, but I wasn’t sure if something was just weird about our model or data

[Pabnau @pabnau]

@sp_monte_carlo That’s a very interesting question. It’s a bit farther afield, but I’ve run into some creative approaches where they basically try to “learn” a prior in Variational Inference. I guess the hope would be that the loss function approaches something better
https://t.co/ZznvO7LwUI

[Pabnau @pabnau]

@sp_monte_carlo @kevinjtian @dianarycai Hmm, well you can get similar grouping-type iterative methods. Like Alternating Least Squares instead of SVD for matrix factorization, or more generally (block)-coordinate descent. I guess the key is for part of the model to be easy-to-solve conditioned on the other parts

[Pabnau @pabnau]

@sp_monte_carlo @kevinjtian @dianarycai Relatedly I think exact inference in non-tree graphical models is possible, but really expensive computationally. The work-arounds I’m familiar with use some kind of inherently approximate approach (eg: mean fields or loopy BP), so not sure if it’s exactly what you want though

[Pabnau @pabnau]

@sp_monte_carlo @kevinjtian @dianarycai Maybe EM algorithms fit that definition? Often you have an easy-to-solve likelihood based on the (increasingly accurate) inferred distribution of latent variables