I work on privacy-focused ML for📱 @Google. I used to work on 🛰 at Skybox Imaging (RIP) and @Stanford. Space + exploration robotics are my jam. Go blue!
@s_shiels03@ammalusty There’s definitely a Bluey GWOT, b/c in one episode they show Rusty’s dad (who’s in the army) deployed somewhere hot and resembling the Middle East
@agarwl_ Respectfully disagree; ICL is useful but we’ve veered to overuse and it’s now a crutch. ICL is like if everytime you drive you first have to read the car manual. That’s useful the first time? But “muscle memory” (transfer of task spec from token to weight space) should kick in
@byjoelanderson In the early days of the moon race, JFK was visiting a NASA center, and happened to cross paths with a janitor and ask him what his job was. “I’m helping to put a man on the moon”. JFK loved that answer, and made sure that mentality was spread
@byjoelanderson Exactly. Grew up in S Florida, rooting for the U. Distinctly remember this new years, watching this game with cousins who were FSU fans. I remember feeling dejected, that this was the exact moment the torch of most dominant program got passed
@keithnewton@Michael_Druggan I would laugh my ass off if it turns out the doctor consulted with an LLM in formulating the medical opinion he gave to Michael.
@Turbinetraveler Not sure if this is correct, but I once read that the pilot involved was demoted (for disobeying orders to eject), and then immediately promoted (for doing something no one had ever done before), lol
In our work, we introduce Gatekeeper: a novel loss function that calibrates smaller models in cascade setups to confidently handle easy tasks while deferring complex ones. Gatekeeper significantly improves deferral performance across a diverse set of architectures and tasks.
Thanks to all my amazing collaborators at Google for hosting me for this internship in Zurich and for making this work possible: Nathalie Rauschmayr, Achin (Ace) Kulshrestha, Petra Poklukar, @wittawatj, @seanAugenstein, @ccwang1992, and @fedassa!
@fauxfresh@edsbs you were once discussing the relative merits of Drake’s emo rap, and you said something to the effect of “sometimes I just want to listen to cash money list off their recent purchases” 🎯
We just put out a key step for making distributed training work at larger and larger models: Scaling Laws for DiLoCo
TL;DR: We can do LLM training across datacenters in a way that scales incredibly well to larger and larger models!
@janecoaston I’m not sure he even has to hear it from another, I fully believe he’s capable of inceptioning. Were people in Columbus really saying ‘Maserati Marv’ before Gus decided to put some stank on it?
@_marimartins04@cafedujord No, it’s just bullshit. You bike on the road, with the cars; and they paint some pictures of bikes on the road surface and put up one or two signs. It’s the same as if there was no infrastructure for bikes at all.