Amit Poonia

NLP, Machine Learning, Functional Programming.

[Amit Poonia @NoisyChannel]

@mycoliza Nice. I suspect a lot of engineers already following this to some extent at workplace, but its good that someone explicitly spelled it out.

[Amit Poonia @NoisyChannel]

@ChrisGPotts I get its stated in context of LLMs but this has been always the case, no? The ML model being part of a larger system.

[Christopher Potts @ChrisGPotts]

So this is an article about the limited value of AI, and the (alleged) use of AI to write it added very limited value while inflicting a reputational cost on the author. This is almost too neat. (If no AI was used in the writing, it simply becomes a different parable.)

[Amit Poonia @NoisyChannel]

Here is more about their type based dispatching: https://t.co/JskgokzMfu

[Amit Poonia @NoisyChannel]

One thing nice about JAX is that the ecosystem around it emphasizes type driven design and patterns you often see in functional programming languages. For e.g. came across today this neat library called plum from beartype which provides Julia style multiple dispatch.

[Amit Poonia @NoisyChannel]

@cHHillee Doesn't this go against the whole "good abstraction are simple and scalable"? I mean attention/transformer was simplification of preceding methods which were more complex and hard to scale. I suspect infra pain is a symptom that something is too complex and should be simplified.

[Martin Odersky @odersky]

New blog post that describes the ideas behind our award-winning paper on securing AI agents: https://t.co/Kj0kmZ1JBW

[Amit Poonia @NoisyChannel]

@aryaman2020 From personal experience all I can say that only thing got 10x'ed for me with LLMs is quality of search and ability learn new things, even that sometimes have negative impact as I keep going on tangents and lose focus on the initial task at the hand.

[Amit Poonia @NoisyChannel]

"Statistical Modeling: Two cultures" is a great read, wish I knew about it when I started.

While reading it again when I saw the line "The goal is not interpretability, but accurate information" I thought Claude wrote it, such is the sensitivity to this pattern of LLM writing :)

[vicki @vboykis]

@_arohan_ ML and recsys:

Machine Learning: The High-Interest Credit Card of Technical Debt
https://t.co/8UNQXiEMoZ

Neural Word Embedding as Implicit Matrix Factorization
https://t.co/9jxUNPW3Lg

Statistical Modeling:Two cultures https://t.co/Lyh5Ff6QIZ

Pinnersage https://t.co/5mE9pMfzZV

[Amit Poonia @NoisyChannel]

A bit of tangent on this: recently I was reading about LLM self-distillation using reverse KL, which led to me revisiting variational inference, latent models and plate diagrams!

The old methods still remain relevant, often necessary building blocks, and/or as a baseline.

[(((ل()(ل() 'yoav))))👾 @yoavgo]

i got a gig to consult some org on some ML problem, and after understanding their domain, the thing that best fits (by far) is some bespoke graphical model solution (say, an LDA-variant) of the form that was very popular a decade ago. but its impossible to recommend that.

[Amit Poonia @NoisyChannel]

@cwolferesearch Nice write up. Couple of additional points which IMO helps with jump from SL to RL, is that RL allows us to work with non differentiable objectives (reward mechanism) unlike SL, and SL itself can be thought as a special case of RL where supervision is dense/fixed and no sampling.

[Amit Poonia @NoisyChannel]

@Vtrivedy10 Maybe a stupid question, but what happened with RETRO? Did it get adapted into something someway? I mean it looks one possible answer to that small cognitive core Karpathy and others talked about, what am I missing here?

[Amit Poonia @NoisyChannel]

@chaotictransfem Modal and Runpod mainly, and google colab sometimes.

[Amit Poonia @NoisyChannel]

@willccbb Because of the recent publication of negative results for reasoning models?

[Amit Poonia @NoisyChannel]

Also openai's spinningup tutorials is a great resource.

[Amit Poonia @NoisyChannel]

I have been watching youtube videos on RL from John Schulman & Sergey Levine, mainly on policy gradients, , they are incredible good, while watching some questions come up to your mind and before you know it they get addressed in the lecture in a way that is very satisfactory.

[Amit Poonia @NoisyChannel]

@j_golebiowski Yeah I guess depends on the task. I tried mainly with multi-turn search tasks with qwen3 base models and found that 1 to 2 epochs were enough, though learning rate needed some tweaking.

[Amit Poonia @NoisyChannel]

@TheZachMueller I gave it a try here, though a bit too minimal: https://t.co/JVJnpg4Qib

[Amit Poonia @NoisyChannel]

@MainzOnX TPUs connected with nearest neighbors (happened to be reading JAX scaling book right now).