Check out our GitHub repo if you want to integrate this framework into your research! We're really excited to see the scientific and clinical applications of cross-participant mapping
https://t.co/dqL2hxjd0p
7/7
This approach can also generalize across modalities: here we trained models on brain responses to stories and transferred them by aligning brain responses to silent movies. This could be effective for mapping concepts in children and patients with language impairments
6/7
This was an incredibly rewarding project to work on. Thanks to the amazing team that made this possible! Carly Millanski, Allison Chen, Lisa Wauters, Jordyn Anders, @ShilpaShamapant , @smwilsonau, @alex_ander, Maya Henry
8/8
We're excited to share our new study on decoding brain activity in participants with post-stroke aphasia! We think this is an important step towards cognitive brain-computer interfaces for patients with language disorders
https://t.co/UrzC5qgzDO
1/8
We also found good decoding performance from individual brain regions. This suggests that we could move our decoder from fMRI into more portable systems. The best regions differed across participants so we think fMRI will remain very important for localizing recording sites
7/8
How does the human brain represent semantic information from different languages?
Our new preprint suggests that bilingual language comprehension relies on shared semantic representations that are systematically modulated by each language!
1/n
Do brain representations of language depend on whether the inputs are pixels or sounds?
Our @CommsBio paper studies this question from the perspective of language timescales. We find that representations are highly similar between modalities! https://t.co/xVaMHKcCOi
1/8
I’ll be presenting this work at the #NeurIPS2023 5pm poster session on Wednesday — come by to chat about the relationship between language and vision in the brain!
Multimodal transformers make it possible to transfer fMRI encoding models between language and vision! (though mostly from L->V and not V->L 🤔) New paper from @jerryptang@_du_meng@vvobot@vasudev_lal https://t.co/sO8F1vBp65
This paper and dataset is now officially published in Scientific Data!! And not only that but we have nearly doubled the amount of data and stimulus available since the preprint version.
Link to paper here: https://t.co/fxaZGAK0pZ
1/n
Our new study is out today in Nature! We demonstrate a brain-computer interface that turns speech-related neural activity into text, enabling a person with paralysis to communicate at 62 words per minute - 3.4 times faster than prior work. 1/3 https://t.co/6pB7NLAqjG