Emil Wärnberg

Computer scientist venturing into the exciting world of neuroscience

[Emil Wärnberg @emil_warnberg]

@adamsafron Feel free to reach out if you have any questions :-)

[Emil Wärnberg @emil_warnberg]

@xpuente @KordingLab Nevertheless, a more complicated model of RL in the brain probably needs to train some kind of policy network as one component. And for that having access to an error gradient is very beneficial, especially if the action space is continuous.

[Emil Wärnberg @emil_warnberg]

@xpuente @KordingLab In our Discussion, we point at some ideas of how that gradient might appear, but realistically I think that would require a much more complete model of the whole brain - perhaps also including hippocampus for e.g. off-policy learning with some smart sampling strategy.

[Emil Wärnberg @emil_warnberg]

@wegmor @Inscopix In summary, all three striatal pathways independently broadcast a similar, complete representation of the different levels of the task (choice value, trial phase) to downstream targets. This representation looks more like RL state, than actor or critic [8/9]

[Emil Wärnberg @emil_warnberg]

@wegmor @Inscopix Our division of the task into 12 discrete phases was somewhat arbitrary. Could we have chosen a finer division? Turns out the answer is yes: we can accurately decode the animals’ progression through individual phases (example: right to center port turn). [6/9]

[Emil Wärnberg @emil_warnberg]

@wegmor @Inscopix Let’s unpack that a bit. Here’s a video of three Oprm1-neurons during the task: the red one fires in the center to right port turn; the purple one fires in the center port prior to left choices; the cyan one when getting a reward from the left port. [3/9]

[Emil Wärnberg @emil_warnberg]

@wegmor @Inscopix Choice port outcomes were occasionally swapped covertly (75% vs 0% reward). In response, mice adapted a win-stay (blue), lose-switch strategy (red). In short, the striatal activity reflected both the mice’s progression through trials and their win/switch strategy. [2/9]

[Sandro Galea @sandrogalea]

Determining what matters and does not is more complicated than accepting binary “statistical” significance. I signed on to this @nature paper, with 800+ other scientists, which eloquently urges we stop talking about statistical significance: https://t.co/cfP92oCZ3X @societyforepi