Andrew Glennerster

3D vision

[Andrew Glennerster @ag3dvr]

A corrigendum for this paper is out today:
https://t.co/D3eXTDEtIe...
(some quite weird things happened to the text between submission and proofs). Anyway, corrected now. I hope the weird elements of the uncorrected version do not put people off.

[Andrew Glennerster @ag3dvr]

New paper out today:
https://t.co/qV3snkJeKB
Summary: This paper argues that the brain navigates through image space rather than a 3D coordinate-based reference frame.

[Andrew Glennerster @ag3dvr]

The set of saccades that connect points in the scene can be described either as an egocentric representation or a ‘policy’, a set of context-dependent actions. When the observer moves, the angles between distant points endure, providing a stable frame.

[Andrew Glennerster @ag3dvr]

One element of the argument concerns the way that we (and most animals) move. We fixate on a point and move relative to that. The dorsal stream is well set up to control movements in this way.

[Andrew Glennerster @ag3dvr]

@katestorrs Yes. (https://t.co/Yez72iN5OR at 2min 30s).

[Andrew Glennerster @ag3dvr]

@GaryMarcus Which aspects (or anatomical regions) of the brain do you think are well modelled by artificial systems (DNNs primarily) and which are not? Will there be an incentive for artificial systems to mimic the latter?

[Andrew Glennerster @ag3dvr]

@criticalneuro @SussilloDavid @FieteGroup @StefanoFusi2 @Awfidius And an even older one, arguing that eye movements across evolution (head movements if the eyes are fixed in the head, body movements if fixed in the body like a fly) are designed to keep the image on a restricted manifold (until a saccade to a new one). https://t.co/slW1KFT9Rg

[Andrew Glennerster @ag3dvr]

@criticalneuro @SussilloDavid @FieteGroup @StefanoFusi2 An old paper, showing that people do not notice when they walk across a (virtual) room as it expands 4-fold https://t.co/NYfJkNYQZT. Hard to explain this if we reconstruct a scene, easy if we move across a manifold of images. https://t.co/gmeYGdYq9Y

[Andrew Glennerster @ag3dvr]

@WiringTheBrain Exactly.

A similar discussion on mastodon: https://t.co/hQPiugoBU1

[Andrew Glennerster @ag3dvr]

@LintonVision @chrisschuringa Wow! Yes, that is me. I don’t remember being in this photo but Patrick Haggard (in the front row) was my landlord at the time so that might have something to do with why I went along. Yes, it really is another country and yes I am now old too.

[Andrew Glennerster @ag3dvr]

@keithfrankish In philosophy, I have never, ever understood the question.

[Andrew Glennerster @ag3dvr]

@NeilBurgess10 @dileeplearning @chrastil @AjdDavison @Awfidius @jcrwhittington @ElDuvelle @jcaread @LintonVision @robwilliamsiii @UniRdg_CINN Many thanks. This is a wonderful, detailed model (@AndrejBicanski). In the paper, I refer to the Byrne https://t.co/VO9HAO1VrG version. I say if this and/or Carpenter https://t.co/vXW21AXceD is shown to be correct, then the hypothesis I advocate will be blown out of the water.

[Andrew Glennerster @ag3dvr]

Phil Trans B @royalsoc are publishing a theme issue today on 3D vision bringing together work on computer, human and animal vision. It includes some challenging new ideas. Issue: https://t.co/hbLJgOeKTP, including a paper by me: https://t.co/5PD0jEe9Hy (mine is open access). 1/n

[Andrew Glennerster @ag3dvr]

@dileeplearning @chrastil @AjdDavison @Awfidius @jcrwhittington @ElDuvelle @NeilBurgess10 @jcaread @LintonVision @robwilliamsiii @UniRdg_CINN Yes, there is no 'vicarious trial and error' here so, as you say, it only considers compiled-in actions. T(s’|s,a) is left to the world - a simplistic assumption to begin with.

[Andrew Glennerster @ag3dvr]

@dileeplearning @chrastil @AjdDavison @Awfidius @jcrwhittington @ElDuvelle @NeilBurgess10 @jcaread @LintonVision @robwilliamsiii @UniRdg_CINN Many thanks!
I mean a store of potential actions, where the one that is chosen is contingent on the current state, π(a|x).

[Andrew Glennerster @ag3dvr]

Alternative site for discussion:

[Andrew Glennerster @ag3dvr]

In summary, don’t imagine that the brain carries out complex 3D coordinate transformations (retinal -> egocentric -> world-centred). Instead, imagine a point moving across a high dimensional manifold of potential brain states and what that movement could achieve. 21/21.

[Andrew Glennerster @ag3dvr]

The tricky bit is outside the brain (or, more precisely, is best understood by looking outside the brain). It requires understanding images and how they change as an observer moves. This is a liberating, democratic perspective and is applicable beyond vision research. 20/n