Fulya Taştan

[Joonas Nättilä @joonas_nattila]

Navier-Stokes Millennium saga continues. Here are my slides and a summary from a recent talk. Disclaimer: this is a physicists view of the problem & solution! 🧵

[Martin Bauer @martinmbauer]

Mathematics

[Fulya Taştan @fuulay]

En sevdiğim bölüm başladı.

[Jürgen Schmidhuber @SchmidhuberAI]

The #NobelPrizeinPhysics2024 for Hopfield & Hinton rewards plagiarism and incorrect attribution in computer science. It's mostly about Amari's "Hopfield network" and the "Boltzmann Machine."

1. The Lenz-Ising recurrent architecture with neuron-like elements was published in 1925 [L20][I24][I25]. In 1972, Shun-Ichi Amari made it adaptive such that it could learn to associate input patterns with output patterns by changing its connection weights [AMH1]. However, Amari is only briefly cited in the "Scientific Background to the Nobel Prize in Physics 2024." Unfortunately, Amari's net was later called the "Hopfield network." Hopfield republished it 10 years later [AMH2], without citing Amari, not even in later papers.

2. The related Boltzmann Machine paper by Ackley, Hinton, and Sejnowski (1985) [BM] was about learning internal representations in hidden units of neural networks (NNs) [S20]. It didn't cite the first working algorithm for deep learning of internal representations by Ivakhnenko & Lapa (Ukraine, 1965)[DEEP1-2][HIN]. It didn't cite Amari's separate work (1967-68)[GD1-2] on learning internal representations in deep NNs end-to-end through stochastic gradient descent (SGD). Not even the later surveys by the authors [S20][DL3][DLP] nor the "Scientific Background to the Nobel Prize in Physics 2024" mention these origins of deep learning. ([BM] also did not cite relevant prior work by Sherrington & Kirkpatrick [SK75] & Glauber [G63].)

3. The Nobel Committee also lauds Hinton et al.'s 2006 method for layer-wise pretraining of deep NNs (2006) [UN4]. However, this work neither cited the original layer-wise training of deep NNs by Ivakhnenko & Lapa (1965)[DEEP1-2] nor the original work on unsupervised pretraining of deep NNs (1991) [UN0-1][DLP].

4. The "Popular information" says: “At the end of the 1960s, some discouraging theoretical results caused many researchers to suspect that these neural networks would never be of any real use." However, deep learning research was obviously alive and kicking in the 1960s-70s, especially outside of the Anglosphere [DEEP1-2][GD1-3][CNN1][DL1-2][DLP][DLH].

5. Many additional cases of plagiarism and incorrect attribution can be found in the following reference [DLP], which also contains the other references above. One can start with Sec. 3:

[DLP] J. Schmidhuber (2023). How 3 Turing awardees republished key methods and ideas whose creators they failed to credit. Technical Report IDSIA-23-23, Swiss AI Lab IDSIA, 14 Dec 2023. https://t.co/Nz0fjc6kyx

See also the following reference [DLH] for a history of the field:

[DLH] J. Schmidhuber (2022). Annotated History of Modern AI and Deep Learning. Technical Report IDSIA-22-22, IDSIA, Lugano, Switzerland, 2022. Preprint arXiv:2212.11279. https://t.co/Ys0dw5hkF4 (This extends the 2015 award-winning survey https://t.co/7goTtI5Uwv)

[Fulya Taştan @fuulay]

If you are a mathematician interested in understanding the mathematical framework of their research, they are inspired by variational optimization methods in calculus of variations to solve energy-based Plateau problems, they arise as solutions to nonlinear PDEs.

[Yoshua Bengio @Yoshua_Bengio]

@HopfieldJohn and @geoffreyhinton, along with collaborators, have created a beautiful and insightful bridge between physics and AI. They invented neural networks that were not only inspired by the brain, but also by central notions in physics such as energy, temperature, system dynamics, energy barriers, the role of randomness and noise, connecting the local properties, e.g., of atoms or neurons, to global ones like entropy and attractors. And they went beyond the physics to show how these ideas could give rise to memory, learning and generative models; concepts which are still at the forefront of modern AI research.

Their ideas inspired me so profoundly that I decided to choose learning in neural networks for my own research as a graduate student. They motivated me to look for abstract principles that could be as simple as the laws of physics, but could explain biological as well as artificial intelligence. I'm truly delighted for them and for our field.

[Jim Fan @DrJimFan]

Hitchhiker's guide to rebranding:
- Machine learning -> statistical mechanics
- Loss function -> energy functional
- Optimize the model -> minimize free energy
- Trained model -> reached equilibrium distribution
- KL divergence -> free energy difference
- Gaussian noise -> random thermal fluctuations
- Random step -> Brownian motion
- SGD -> directional Brownian motion
- GPU -> simulated particle accelerator
- Diffusion models -> Langevin dynamics
- Reinforcement learning -> control theory
- Robotics -> physical computation
- Audio learning -> 1D signal processing
- Image learning -> 2D signal processing
- Video learning -> 3D signal processing
- Multimodal models -> multidimensional signal processing
- Sora -> learned physics engine

You're welcome

[Olexandr Isayev 🇺🇦🇺🇸 @olexandr]

Cats are invariant under SO(3) transformations! 😼

[Fulya Taştan @fuulay]

@hmelisaacar Referans kaynak verebilir misiniz? “Positive dissonance” diye arattığımda cognitive dissonance diye düzeltiliyor. Imposter syndrome kavramından farkı nedir bu kavramin?

[Elon Musk @elonmusk]

Guys I found Q*

[Fulya Taştan @fuulay]

In other words, studying mathematics humbles you.

[Iso (math fool) @IsomorphicPhi]

Math feels frustratingly huge and deep. After so many years of study it seems that I haven't gotten further than knowing some terminology and a few elementary results

[Fulya Taştan @fuulay]

@connntrol Teşekkür ederim. Bana saldıran kişi kadındı ama tarif ettiğiniz kategoriye uyuyordu 😄

[Fulya Taştan @fuulay]

Sonra efendim Amerikalılar niye bu kadar bireysel bilmem ne. Tek sebebi bu sistemin yarattığı korku ve güvensizlik hissiyatı. Burada yaşamaya devam ettikçe çok da yargilayamiyorum artık onları.

[Fulya Taştan @fuulay]

Ancak sonra dükkanına sığınmama izin verdi ve polisi aradı. Yalnızsanız ve güvendiğiniz bir çevreniz yoksa burada yaşamak kafa yedirtir. İyice bireyselleșip herkese korkuyla yaklaşan bir manyağa dönüşme olasılığınız çok yüksek.

[Matematik Dayanışma Platformu @matdayanisma]

Depremden etkilenen üniversite öğrencilerine matematik derslerinde destek sağlamak amacıyla geçen dönem başlattığımız Matematik Dayanışma Platformu’nu bu eğitim-öğretim yılında da devam ettiriyoruz.
Katılmak isteyen öğrenciler ve danışmanlar için linkmiz:
https://t.co/Jf3AIoggMU

[Fulya Taştan @fuulay]

@ghiblipicture Grave of the fireflies

[Fulya Taştan @fuulay]

@gusholderhaber R.I.P. my brain

[Fulya Taştan @fuulay]

https://t.co/1aXAPavSiY

[Melike Demirbağ Kaplan @melikedkaplan]

Memleket evlatlarının yurtdışında göreceli olarak başarılı olmasını sağlayan şeylerden birinin "Bu şey ne olursa çalışır?"dan ziyade "Bu şey ne olursa çalışmaz?"a odaklı düşünce sistemi olduğunu söyleyebilirim. Bi işin nereden patlayabileceğini düşünmek iliklerimize işlemiş.

[Ahmet Erdem @a_erdem4]

Are you ever stuck deciding between two options? GetOpinion is the app that can help. Just ask whatever is on your mind and get immediate votes from other users.
It is available on both the App Store and Google Play. I would be happy if you can give it a try 🫂

[Ebrar Karakurt @karakurtebrar18]

BİZ TÜRKİYE'YİZ BİZ ŞAMPİYONUZ! 🇹🇷

Tribünlerde ve ekran başında sizler, sahada biz! Birlikte savaştık ve kazandık!! Bu şampiyonluk güzel ülkemiz için...
Avrupa'nın en büyüğü TÜRKİYE! 🐺❤️🏆