Can we enable robots to develop a sense of touch without forgetting what they learned from large-scale vision-only pretraining?
Introducing MultiSensory World Model (MuSe) 🌍: A new approach for finetuning visuomotor policies on minimal data from new sensor modalities, such as force/torque (F/T)
With Muse, touch learned later improves skills learned earlier — a small amount of F/T data on new tasks improves zero-shot on diverse pretraining tasks that were never supervised with F/T
We believe MuSe provides a practical pathway towards training multisensory foundation models that leverage both abundant vision data, and smaller multisensory datasets 🧵👇
@claudeai Can someone make a benchmark on the efficiency-accuracy of coding agents (how much token burnt for a task and accuracy of the task)? I am low on tokens
"The erosion of creative thinking means young people will struggle to navigate uncertainty." AI can only iterate on what it has seen before- but "our species’ ability to come up with unexpected and original ideas is something to protect and nurture." https://t.co/6AUb9MECNR
@brennan__simon Real data reflects the true distribution of language and knowledge, while synthetic data is just a recursive projection of the prior that current models already believe. The improvement is marginal and could risk long-tail problem that would result in model collapse.
@brennan__simon Cautiously optimistic. In training LLM, syn data is usually used in post-training, especially in finetuning reasoning whereas real data predominates pre-training. However, in biomedical research, the bottleneck is scarcity of real data that represent diverse biology.
Genome wide perturb seq creates powerful perturbations x transcripts datasets. This unveils new biology - but also reveals new off-target effects. New work from @AustinMHartman, a massively talented PhD student in our lab, systematically identifies seed driven off-target effects in genome-wide perturb seq exps. (1 of 3) https://t.co/6kg7cJjCQr
The next inflection point in virtual cell modeling will require large-scale perturbative data. To get there we are partnering with @tahoe_ai and @officialbiohub to help us generate a dataset to push us forward: https://t.co/dZbcKBTxdX
How do we move beyond chain-of-thought to build models that reason more deeply and reliably?
Is generalization, rather than scale, now the real bottleneck?
I’m excited to talk with @Muennighoff, on the frontier of model reasoning and reinforcement learning.
#ai#LLMs#reasoning
In this conversation, we discuss:
Model reasoning research beyond chain-of-thought
The future of reinforcement learning
Test-time scaling and inference-time compute tradeoffs
AI’s impact on the job market, the “AI bubble,” and separating signal from hype
In Silicon Valley, “virtual cells” are suddenly everywhere.
Meta and CZI recently went all in, signaling that this is no longer a fringe research direction.
So what is virtual cell? Check out my new blog about virtual cell: https://t.co/lQx2dc4XsZ
We've got a #Twofer for #TT!! First, we tested 6 different #ML models for Grade IV #glioma prognosis, with improvement after feature selection and high accuracy.
They identified factors such as adjuvant TMZ💊, confusion, and MGMT status🧬 as predictive in survival!
Today is a bittersweet moment with Toby Mao leaving my group and pursing his next step as a prestigious @FulbrightPrgrm fellow as he waits to hear back from med schools. Congratulations to Toby and we wish him luck!
@CleClinicCIRC@CleClinicHVTI