I’m starting a weekly blog where I (metaphorically) cut open the putative black box architecture of the transformer! 🤖
Part 1: Are transformers that different from convolutional networks? 🤔
https://t.co/QJC5Jc2QDr
#Neuroscience#NeuroAI#Vision
In neuroscience, a canonical microcircuit is thought to subserve all cortical computation. In AI, neural networks universally subserve intelligent behavior. Is there convergence between computational motifs in natural and artificial intelligence? Work with @dyamins 🧠🤖
Thus, the transformer appears a compelling candidate for a canonical cortical microcircuit, capturing computations happening at the scale of a single neuron or small groups of neurons.