Are you looking for an independent research position in Europe?
I highly recommend checking this out!
The ELLIS Institute in Tübingen is special. Full academic freedom, great funding, competitive salaries and a co-appointment at @MPI_IS.
https://t.co/sTHP9iVVgQ
The year is 2030, ChatGPT has been RLHF’d so much that prompt hackers have found that 1000 tokens dedicated to small talk about how ChatGPT’s hallucinated children are doing in school improves performance
Anthropic’s Claude is pretty decent at ingesting a pdf and responding to neurips reviews. If only reviewers put their own reviews through Claude…probably would save everyone time lmao
@peter_richtarik@karanchadha1729 We also analyze the post-personalization test error of minimizing the loss in equation (2) of your paper. We show it does do better than MAML, but we also show that adding a ridge penalty during your fine-tuning phase gives you the same benefit.
A couple of years ago, with John Duchi and @karanchadha1729, we showed in linear models that FedAvg becomes a meta-learner (in particular, MAML) as model sizes got larger (https://t.co/iVTz2v2QK0). Very cool to see this behavior observed in practice!
1/ Our work is out! 🚨
Towards Federated Foundation Models:
Scalable Dataset Pipelines for Group-Structured Learning
We push federated learning research closer to LLM scales.
Paper: https://t.co/thgBT4m3PK
Joint with @nicki_mitch & @KrishnaPillutla
Thread below.
While our theory does suggest high-dimensionality is a crucial ingredient, it does not explain the effect the number of client steps has on meta-learning ability. Still more to discover!
@gowerrobert In work with @karanchadha1729 and John Duchi, we develop theory for accelerated versions for model-based stochastic optimization methods in https://t.co/m2codfzq1Z, including models which leverage nonnegative losses. Would love to hear some discussion of how your work relates.
@YiTayML@Eric_Wallace_ Honest Q: What is the alternative that you think would have been better? Seems to me like trying it and then finding out shortcomings is still better than not trying it all.
@randall_balestr@sarahookr However, if the answer to my question is yes, this suggests that maybe there is some irreducible level of churn which can only be overcome via some type of ensembling.
@randall_balestr@sarahookr I ask because if the answer is no, then isn't the punchline less about ensembling and more about training time? i.e., "fairness naturally emerges from more training"