Krishna G

#DesignEnthusiast #MultiDisciplinarySystemsResearch #MachineIntelligence

[Krishna G @krish_ondway]

#argumentumadauctoritatum and #argumentumadpopulum

[Insider Mix @InsiderMix1]

ما زلنا لا نستوعب هذه اللقطة.. رونالدو قفز حتى وصلت ركبتاه إلى مستوى رأس منافسه! 🤯🚀

[Jürgen Schmidhuber @SchmidhuberAI]

Who invented convolutional neural networks (CNNs)?

1969: Fukushima had CNN-relevant ReLUs [2].

1979: Fukushima had the basic CNN architecture with convolution layers and downsampling layers [1]. Compute was 100 x more costly than in 1989, and a billion x more costly than today.

1987: Waibel applied Linnainmaa's 1970 backpropagation [3] to weight-sharing TDNNs with 1-dimensional convolutions [4].

1988: Wei Zhang et al. applied "modern" backprop-trained 2-dimensional CNNs to character recognition [5].

All of the above was published in Japan 1979-1988.

1989: LeCun et al. applied CNNs again to character recognition (zip codes) [6,10].

1990-93: Fukushima’s downsampling based on spatial averaging [1] was replaced by max-pooling for 1-D TDNNs (Yamaguchi et al.) [7] and 2-D CNNs (Weng et al.) [8].

2011: Much later, my team with Dan Ciresan made max-pooling CNNs really fast on NVIDIA GPUs. In 2011, DanNet achieved the first superhuman pattern recognition result [9]. For a while, it enjoyed a monopoly: from May 2011 to Sept 2012, DanNet won every image recognition challenge it entered, 4 of them in a row. Admittedly, however, this was mostly about engineering & scaling up the basic insights from the previous millennium, profiting from much faster hardware.

Some "AI experts" claim that "making CNNs work" (e.g., [5,6,9]) was as important as inventing them. But "making them work" largely depended on whether your lab was rich enough to buy the latest computers required to scale up the original work. It's the same as today. Basic research vs engineering/development - the R vs the D in R&D.

REFERENCES

[1] K. Fukushima (1979). Neural network model for a mechanism of pattern recognition unaffected by shift in position — Neocognitron. Trans. IECE, vol. J62-A, no. 10, pp. 658-665, 1979.

[2] K. Fukushima (1969). Visual feature extraction by a multilayered network of analog threshold elements. IEEE Transactions on Systems Science and Cybernetics. 5 (4): 322-333. This work introduced rectified linear units (ReLUs), now used in many CNNs.

[3] S. Linnainmaa (1970). Master's Thesis, Univ. Helsinki, 1970. The first publication on "modern" backpropagation, also known as the reverse mode of automatic differentiation. (See Schmidhuber's well-known backpropagation overview: "Who Invented Backpropagation?")

[4] A. Waibel. Phoneme Recognition Using Time-Delay Neural Networks. Meeting of IEICE, Tokyo, Japan, 1987. Backpropagation for a weight-sharing TDNN with 1-dimensional convolutions.

[5] W. Zhang, J. Tanida, K. Itoh, Y. Ichioka. Shift-invariant pattern recognition neural network and its optical architecture. Proc. Annual Conference of the Japan Society of Applied Physics, 1988. First backpropagation-trained 2-dimensional CNN, with applications to English character recognition.

[6] Y. LeCun, B. Boser, J. S. Denker, D. Henderson, R. E. Howard, W. Hubbard, L. D. Jackel: Backpropagation Applied to Handwritten Zip Code Recognition, Neural Computation, 1(4):541-551, 1989. See also Sec. 3 of [10].

[7] K. Yamaguchi, K. Sakamoto, A. Kenji, T. Akabane, Y. Fujimoto. A Neural Network for Speaker-Independent Isolated Word Recognition. First International Conference on Spoken Language Processing (ICSLP 90), Kobe, Japan, Nov 1990. A 1-dimensional convolutional TDNN using Max-Pooling instead of Fukushima's Spatial Averaging [1].

[8] Weng, J., Ahuja, N., and Huang, T. S. (1993). Learning recognition and segmentation of 3-D objects from 2-D images. Proc. 4th Intl. Conf. Computer Vision, Berlin, pp. 121-128. A 2-dimensional CNN whose downsampling layers use Max-Pooling (which has become very popular) instead of Fukushima's Spatial Averaging [1].

[9] In 2011, the fast and deep GPU-based CNN called DanNet (7+ layers) achieved the first superhuman performance in a computer vision contest. See overview: "2011: DanNet triggers deep CNN revolution."

[10] How 3 Turing awardees republished key methods and ideas whose creators they failed to credit. Technical Report IDSIA-23-23, Swiss AI Lab IDSIA, 14 Dec 2023. See also the YouTube video for the Bower Award Ceremony 2021: J. Schmidhuber lauds Kunihiko Fukushima.

[nikunj shah 🇮🇳 @niku1630]

Three great forces rule the world; stupidity, fear and greed.

[Krishna G @krish_ondway]

@FactcheckEU @JulesDrmnn @FactcheckEU @MEAFactCheck This book provides history of active measures. Do we have any tools for the citizen to discern the truth and actively verify the facts? I@USPoliticsAlert in today's world, the information diet of the citizen matters a lot. https://t.co/yv8F2h5LR4

[Krishna G @krish_ondway]

Some of the finest articles around protocol research and system innovation/dynamics. Worth reading. https://t.co/7ASlsbmwDf

[THE SKIN DOCTOR @theskindoctor13]

Volvo XC90, renowned as one of the safest cars in the world, has recorded zero fatal crashes in UK since its launch in 2002.

Imagine investing in this car for its unmatched safety, only to face a tragic end because a car in the opposite lane suddenly slows down, causing a container truck behind it to lose control, jump the divider, and crush you, your spouse, three kids, and a relative.

Not the Volvo's fault, as any car would have crushed under such heavy weight—just an unfortunate twist of fate. Sad how death can find you in the most unexpected ways, even when you've done everything to stay safe.

Bengaluru, 21 Dec 2024.

[Krishna G @krish_ondway]

A very valid case which should gain traction. Else, we might end up with something like artificial intelligence vs natural stupidity.

[FAR.AI @farairesearch]

✈️😀 Which technologies might we have lost if a worse alternative had won regulatory capture? 🦅😕

Stuart Russell makes the case for solving technical problems over forcing through inferior AI systems at the Vienna Alignment Workshop hosted by FAR AI.

[Krishna G @krish_ondway]

Pole vault World Record #Paris2024Olympics

[Sports on HBO Max 🇸🇪 @sportshbomaxse]

MONDO TAR VÄRLDSREKORD!!!! 6.25! 🤩

📲 Streama varje ögonblick från OS i Paris 2024 – bara på Max

[Krishna G @krish_ondway]

@elonmusk very much true and it reflects the current state of affairs.

[Elon Musk @elonmusk]

😬

[Lawrence H. Summers @LHSummers]

Perhaps the world's greatest debtor can't remain the world's greatest super power. It’s a 1938 like moment and that coupled with the complexity of our internal challenges makes this a very, very serious moment.

Watch my interview from @AspenEcon with @DavidWestin @BloombergTV

https://t.co/KoHApIOskB via @YouTube

[Elon Musk @elonmusk]

Best AI video to date!

[Krishna G @krish_ondway]

https://t.co/UevdoOBJPo

[Subramanian Swamy @Swamy39]

Who cares who won? The nation’s losers are the youth (who despite a globe trotting PM who is without a party majority in Parliament), who are unemployed or semi employed I.e., in jobs well below their education. How heart breaking and dream destroying it must be for them.

[Krishna G @krish_ondway]

https://t.co/EoOZjl9UaQ

[Krishna G @krish_ondway]

@MuskUniverseUsa Open Your Eyes

[Krishna G @krish_ondway]

Check out my latest article: Device Instruction Transpiler https://t.co/vSpfmlDtsR via @LinkedIn

[Tesla @Tesla]

Making Model 3 even better

[Yann LeCun @ylecun]

I propose the creation of the DeTuring Award.
It will be granted to people who are consistently trying (and failing) to deter society from using computer technology by scaring everyone with imaginary risks.
As the Turing Award is the Nobel Prize of computing, the DeTuring Award is the IgNobel Prize of computing.