Can we tell whether data domains cooperate or compete during pretraining?
Adding code to the mix makes models better at math, while some other combinations hurt each other. We call this data synergy. Turns out you can incorporate data synergy into scaling laws and estimate it ���
My Lab at the University of Edinburgh🇬🇧 has funded PhD positions for this cycle!
We study the computational principles of how people learn, reason, and communicate.
It's a new lab, and you will be playing a big role in shaping its culture and foundations.
Spread the words!
How do people reason so flexibly about new problems, bringing to bear globally-relevant knowledge while staying locally-consistent? Can we engineer a system that can synthesize bespoke world models (expressed as probabilistic programs) on-the-fly?
Today we’re launching AutumnBench, our benchmark built on @BasisOrg’s Autumn platform. It’s designed to measure world modeling and reasoning by placing humans and AI in unfamiliar worlds—with no rewards or guidance—to test who can figure out how these worlds actually work.
I've struggled to announce this amidst so much dark & awful going on in the world, but with 1mo to go, I wanted to share that: (i) I finally graduated; (ii) In August, I'll begin as an assistant professor in the CS dept. of the National University of Singapore.
Last but not least, the SIGPLAN Robin Milner Young Researcher Award was also announced at PLDI. This year, the award went to Işıl Dillig (@IsilDillig), whose research has had profound and far-reaching contributions to program analysis, verification, and synthesis ⭐️
New paper: World models + Program synthesis by @topwasu
1. World modeling on-the-fly by synthesizing programs w/ 4000+ lines of code
2. Learns new environments from minutes of experience
3. Positive score on Montezuma's Revenge
4. Compositional generalization to new environments
https://t.co/WpTEeF5mRa
[1/n]
We are hosting the MIT Programming Languages Review on April 25th in person here at MIT! The PLR is a student-run workshop that aims to highlight the best papers from the past year that we believe will have a significant impact on shaping the future direction of PL research.
If you're interested in a PhD at the intersection of machine learning and programming languages, consider Yale CS!
We're exploring new ways to build software that draws inferences & makes predictions. See https://t.co/uFSVhlBNvT & apply at https://t.co/pPCQps7Jch by Dec. 15 😃
I am recruiting PhD students at Duke!
Please apply to @dukecompsci or Duke CBB if you are interested in developing new methods and paradigms for NLP/LLMs in healthcare.
For details, see here: https://t.co/6duX8z3rk4. Feel free to retweet!
New ARC-AGI paper
@arcprize w/ fantastic collaborators @xu3kev@HuLillian39250@ZennaTavares@evanthebouncy@BasisOrg
For few-shot learning: better to construct a symbolic hypothesis/program, or have a neural net do it all, ala in-context learning?
https://t.co/zcmxoQzv92
I'm thrilled to share that I have joined Microsoft Research as a Senior Researcher in the amazing @RiSE_MSR team!✨Looking forward to working on AI, symbolic reasoning and human-centric methods to empower programmers and end users.
Should AI be aligned with human preferences, rewards, or utility functions?
Excited to finally share a preprint that @MicahCarroll@FranklinMatija@hal_ashton & I have worked on for almost 2 years, arguing that AI alignment has to move beyond the preference-reward-utility nexus!
Exciting new work @NatureComms led by Josh Rule on how to model human learning symbolically.
The new model out-performs key alternatives, including a code generation LLM and previous symbolic models.
https://t.co/EKxirfbgao
Life update: I’ll be starting (this week!) as an Assistant Professor in the Yale philosophy department and cognitive science program, affiliated with the Wu Tsai Institute’s Center for Neurocomputation. I’ll also be launching the Computational Cognitive Architecture lab – tackling questions in perception and cognition from computational first principles (more on this soon!).
Huge thank you to all who’ve supported to this point. I am beyond excited to join such a vibrant community and to experience everything that’s to come!
The space of human goals is infinitely vast -- yet, people spontaneously infer plausible motivations for others from just a few actions. How?
Excited to share a paper I'll be presenting at #CogSci this week, which introduces an algorithmic account of *open-ended goal inference*!
Why do people take turns exerting effort to benefit one another? In new work with @rebecca_saxe at #CogSci2024@cogsci_soc, we show that, in a cognitive model, the value of communicating equality can specifically give rise to reciprocal generosity
📄https://t.co/z00ONI3HwX
I'm proud of our group's presentations at #CogSci2024 – come check them out!
1: Today 10:30am–noon in J.F. Stall: "Finding structure in logographic writing with library learning" led by @jiang_gy. This work won the Sayan Gul award for best undergraduate student paper!
#ai4code peeps at #ICLR2024: come chat self-repair with me at 10:45 AM on Friday, May 10!
I'll be presenting the camera-ready version of this fan favorite from last year... now with even more data!
Keep reading for a sneak peek of the results 👇