Rob Neuhaus

LLM obsessed.

[Rob Neuhaus @rob_renaud]

@natolambert Do you think EMO is a promising direction to get even more juice from tiny models?

[Demis Hassabis @demishassabis]

https://t.co/PTeDiv1b6L

[Rob Neuhaus @rob_renaud]

@PackBropagated @prajdabre Optimizers are for LLMs are not finding local minima. If you keep training, the loss will keep going down (slowly). There are too many directions to to lower the loss from any given point.

[Rob Neuhaus @rob_renaud]

@PackBropagated @prajdabre Does local minima have anything to do with high dimensional optimization? I thought local minima are essentially practically impossible to find.

[Rob Neuhaus @rob_renaud]

@prajdabre They are in different optimization basins, their weights are in wildly different spaces. You either need to start from a single trunk that has been somewhat trained to break the symmetries. Or you can do some intense remapping to put them in the same space. See linear mode con

[Rob Neuhaus @rob_renaud]

@prajdabre Intermediate activations, the teacher loss curve/checkpoints.

[Rob Neuhaus @rob_renaud]

@pmddomingos @andrewgwils What kinds of errors do people make when they forget NFLs? Can you explain one case where thinking about NFL would have saved someone time/effort?

[Rob Neuhaus @rob_renaud]

@neur_reps @SuryaGanguli Hi from the 3000 guy at neurips. Does the cat classifier neuron have implications for the platonic representation hypothesis?

[Rob Neuhaus @rob_renaud]

@yoavgo Is the sycophancy problem non-technical? It seems very intertwined with effective reward modelling.

[Rob Neuhaus @rob_renaud]

@dwarkesh_sp are there any Linus Torvalds or Guido van Rossum's of open source ai? If not, why not? What should someone do to rise to that level?

[Rob Neuhaus @rob_renaud]

@srush_nlp Any new intuitions about this in the intervening two years?

[Rob Neuhaus @rob_renaud]

@LightningAI any chance I can get priority verification for Lightning Studios?

[Rob Neuhaus @rob_renaud]

@dwarkesh_sp @fchollet Why did pytorch win ML researcher hearts and minds?

[Rob Neuhaus @rob_renaud]

@CitiBikeNYC as an annual member, please stop asking me for payment info everytime I scan a bike. It's so annoying.

[Rob Neuhaus @rob_renaud]

@BlackTeamKQB https://t.co/u6hPNcnBbo . Warning, contains an outburst of joy.

[Rob Neuhaus @rob_renaud]

@hearluminary please give us some faster than 1x player options. As is, using your paid app feels worse than my previous free podcast app.

[Rob Neuhaus @rob_renaud]

@joshdebonis @nmikros considered fixing the "oh shit, can't find my bee problem"? down down down -> visual indicator would be famazing

[Seth Chambers @sethchambers]

@lagunitasT @SierraNevada how about a collab brew w/ proceeds to solve a "real" problem (like clean water)? Call it "oopsies" or "mulligan"?

[Rob Neuhaus @rob_renaud]

@Ednever Bad analysis re Seattle. You can subscribe to a city subreddit without living in that city

[Rob Neuhaus @rob_renaud]

@jeroenhjanssens I posted your SOS article here, and someone asks you a question http://t.co/fHpj4GTeSG