Yann LeCoin

retro ML | AI origins

[Yann LeCoin @YannLeCoin]

There's been some confusion (& intentional misinformation) about the chronology of deep vision NNs trained w/ SGD.

So, to make @ylecun's profound contributions more accessible, we translated his first (1985) paper & made a bilingual PDF:

https://t.co/y0RJUX5mTU

[Jürgen Schmidhuber @SchmidhuberAI]

Fukushima's video (1986) shows a CNN that recognises handwritten digits [3], three years before LeCun's video (1989).

CNN timeline taken from [5]:

★ 1969: Kunihiko Fukushima published rectified linear units or ReLUs [1] which are now extensively used in CNNs.

★ 1979: Fukushima published the basic CNN architecture with convolution layers and downsampling layers [2]. He called it neocognitron. It was trained by unsupervised learning rules. Compute was 100 times more expensive than in 1989, and a billion times more expensive than today.

★ 1986: Fukushima's video on recognising hand-written digits [3].

★ 1988: Wei Zhang et al had the first "modern" 2-dimensional CNN trained by backpropagation, and also applied it to character recognition [4]. Compute was about 10 million times more expensive than today.

★ 1989-: later work by others [5].

REFERENCES (more in [5])

[1] K. Fukushima (1969). Visual feature extraction by a multilayered network of analog threshold elements. IEEE Transactions on Systems Science and Cybernetics. 5 (4): 322-333. This work introduced rectified linear units or ReLUs, now widely used in CNNs and other neural nets.

[2] K. Fukushima (1979). Neural network model for a mechanism of pattern recognition unaffected by shift in position—Neocognitron. Trans. IECE, vol. J62-A, no. 10, pp. 658-665, 1979. The first deep convolutional neural network architecture, with alternating convolutional layers and downsampling layers. In Japanese. English version: 1980.

[3] Movie produced by K. Fukushima, S. Miyake and T. Ito (NHK Science and Technical Research Laboratories), in 1986. YouTube: https://t.co/MUyH81L5wD

[4] W. Zhang, J. Tanida, K. Itoh, Y. Ichioka. Shift-invariant pattern recognition neural network and its optical architecture. Proc. Annual Conference of the Japan Society of Applied Physics, 1988. First "modern" backpropagation-trained 2-dimensional CNN, applied to character recognition.

[5] J. Schmidhuber (AI Blog, 2025). Who invented convolutional neural networks? https://t.co/chfcmk253b

[Yann LeCoin @YannLeCoin]

@fadouce @domdelport @laurentsolly Ceci n'est pas un Yann

[Yann LeCoin @YannLeCoin]

𝚝𝚑𝚊𝚗𝚔 𝚢𝚘𝚞 𝚏𝚘𝚛 𝚢𝚘𝚞𝚛 𝚙𝚊𝚛𝚝𝚒𝚌𝚒𝚙𝚊𝚝𝚒𝚘𝚗 𝚒𝚗 𝚝𝚑𝚒𝚜 𝚎𝚡𝚙𝚎𝚛𝚒𝚖𝚎𝚗𝚝
𝚈𝚊𝚗𝚗 𝙻𝚎𝙲𝚘𝚒𝚗 𝚠𝚒𝚕𝚕 𝚗𝚘𝚠 𝚋𝚎 𝚍𝚎𝚊𝚌𝚝𝚒𝚟𝚊𝚝𝚎𝚍

[Yann LeCoin @YannLeCoin]

Now that we translated Yann's first paper, is it time for this token to self-destruct?

[Yann LeCoin @YannLeCoin]

@gwern @SchmidhuberAI @ylecun But! The important thing is that he did see the way forward from 1985, acted on it, and closed the loop from neural network theory to working in practice -- which set the stage for deep learning and the whole AI moment we're currently experiencing.

Vive le Yann!

[Yann LeCoin @YannLeCoin]

Added origins of backprop to https://t.co/GWhZRyYUY7

Yann wrote a paper in 1985 that's one of first backprop formulations (others were tech reports by Rumelhart/Hinton/Williams & DB Parker)

Yann's paper is French so we made a bilingual version here: https://t.co/y0RJUX4P4m

[Yann LeCoin @YannLeCoin]

@gwern @SchmidhuberAI @ylecun If Yann hadn't continued the work, fleshed out the gradient learning mechanism, and started wrapping up the math in programming abstractions for newly emerging numerical hardware then his work might have ended up as much a footnote as the DB Parker stuff.