Kameron Decker Harris

Math, computation, networks, neurosci, biology / snow and other fun things. Here for the strange animals.
Assistant Prof @WWU Computer Science

[Kameron Decker Harris @KameronDHarris]

"Structured random receptive fields enable informative sensory encodings"

work headed by @tweetbarrage with @marius10p @bingbrunton

https://t.co/CpqFiSkFQa

[Kameron Decker Harris @KameronDHarris]

@MelMitchell1 Bezos has enough money not to care, and will probably profit from a T**** admin enough to make it worth it, I suppose

[Laureline Logiaco @LLogiaco]

@reziliusReza Science paper in the 1960s ... https://t.co/BNA86trQYb

The question is which papers published today will make future scientists laugh...

[Kameron Decker Harris @KameronDHarris]

@MountRainierNPS Boooo

[Kameron Decker Harris @KameronDHarris]

@FlorentinGuth Nice work, Florentin

[Kameron Decker Harris @KameronDHarris]

@StphTphsn1 @CellTypist @crozSciTech @SaraASolla This is such bad exposition. Let me tell you about my super technical theorem and conditions, but I give no proof or idea to its technique. Proof by intimidation & confusion

[Jane Manchun Wong @wongmjane]

Twitter’s algorithm specifically labels whether the Tweet author is Elon Musk

“author_is_elon”

besides the Democrat, Republican and “Power User” labels

https://t.co/fhpBjdfifX

[Kameron Decker Harris @KameronDHarris]

@zamakany I remember Steve and Nathan constructing their first board system in an abandoned windowless room in the applied math building. We were always a bit puzzled by why it was happening. Now they're YouTube stars!

[The Robert Larner, M.D. College of Medicine @UVMLarnerMed]

This month, #UVMLarnerMed Associate Professor of Neurological Sciences DaviBock , Ph.D., along with colleagues from Princeton, Cambridge and @NIH's The BRAIN Initiative®, presented their research on the mapping of the entire fruit fly brain, at #sfn2024 @UVMResearch @uvmvermont

[Kameron Decker Harris @KameronDHarris]

@TPVogels haha thank you

[Kameron Decker Harris @KameronDHarris]

@DarioRingach @TonyZador @ylecun I didn't know that! Good fact

[Kameron Decker Harris @KameronDHarris]

Registration for DeepMath is now *open*!
https://t.co/HvIz1zeqqF

[Kameron Decker Harris @KameronDHarris]

@MtBakerSnoNF Awesome!

[Kameron Decker Harris @KameronDHarris]

@Sauers_ @StphTphsn1 Fourier is showing up because your are looking for orthogonal basis & your data are ~stationary/smooth. Try a different kind of matrix factorization, e.g. nonnegative or ICA.

[Kameron Decker Harris @KameronDHarris]

Some fundamental contributions to neural nets outlined in this thread. Formative, essential stuff

[Sebastian Seung @SebastianSeung]

🧵on Japan's underrated contributions to neural nets. Shun-ichi Amari @UTokyo_News_en @riken_en is another one of my heroes. His 1972 paper on associative memory models modeled Hebbian plasticity using an outer product weight matrix.

[Sebastian Seung @SebastianSeung]

🧵on Japan's underrated contributions to neural nets. Shun-ichi Amari @UTokyo_News_en @riken_en is another one of my heroes. His 1972 paper on associative memory models modeled Hebbian plasticity using an outer product weight matrix.

[Kameron Decker Harris @KameronDHarris]

@beenwrekt It's the modern version of Jevons' paradox that I teach in my algorithms class. "More efficient hardware/algorithms can still end up being a crappier experience"

OTOH my 12 year old ThinkPad running debian is still fully useful so long as I don't care about battery life

[Kameron Decker Harris @KameronDHarris]

There's a lot more to this backstory:
https://t.co/LwJamkN1pL

[Kameron Decker Harris @KameronDHarris]

Read this ⏬ if you care about science

[Jürgen Schmidhuber @SchmidhuberAI]

The #NobelPrizeinPhysics2024 for Hopfield & Hinton rewards plagiarism and incorrect attribution in computer science. It's mostly about Amari's "Hopfield network" and the "Boltzmann Machine."

1. The Lenz-Ising recurrent architecture with neuron-like elements was published in 1925 [L20][I24][I25]. In 1972, Shun-Ichi Amari made it adaptive such that it could learn to associate input patterns with output patterns by changing its connection weights [AMH1]. However, Amari is only briefly cited in the "Scientific Background to the Nobel Prize in Physics 2024." Unfortunately, Amari's net was later called the "Hopfield network." Hopfield republished it 10 years later [AMH2], without citing Amari, not even in later papers.

2. The related Boltzmann Machine paper by Ackley, Hinton, and Sejnowski (1985) [BM] was about learning internal representations in hidden units of neural networks (NNs) [S20]. It didn't cite the first working algorithm for deep learning of internal representations by Ivakhnenko & Lapa (Ukraine, 1965)[DEEP1-2][HIN]. It didn't cite Amari's separate work (1967-68)[GD1-2] on learning internal representations in deep NNs end-to-end through stochastic gradient descent (SGD). Not even the later surveys by the authors [S20][DL3][DLP] nor the "Scientific Background to the Nobel Prize in Physics 2024" mention these origins of deep learning. ([BM] also did not cite relevant prior work by Sherrington & Kirkpatrick [SK75] & Glauber [G63].)

3. The Nobel Committee also lauds Hinton et al.'s 2006 method for layer-wise pretraining of deep NNs (2006) [UN4]. However, this work neither cited the original layer-wise training of deep NNs by Ivakhnenko & Lapa (1965)[DEEP1-2] nor the original work on unsupervised pretraining of deep NNs (1991) [UN0-1][DLP].

4. The "Popular information" says: “At the end of the 1960s, some discouraging theoretical results caused many researchers to suspect that these neural networks would never be of any real use." However, deep learning research was obviously alive and kicking in the 1960s-70s, especially outside of the Anglosphere [DEEP1-2][GD1-3][CNN1][DL1-2][DLP][DLH].

5. Many additional cases of plagiarism and incorrect attribution can be found in the following reference [DLP], which also contains the other references above. One can start with Sec. 3:

[DLP] J. Schmidhuber (2023). How 3 Turing awardees republished key methods and ideas whose creators they failed to credit. Technical Report IDSIA-23-23, Swiss AI Lab IDSIA, 14 Dec 2023. https://t.co/Nz0fjc6kyx

See also the following reference [DLH] for a history of the field:

[DLH] J. Schmidhuber (2022). Annotated History of Modern AI and Deep Learning. Technical Report IDSIA-22-22, IDSIA, Lugano, Switzerland, 2022. Preprint arXiv:2212.11279. https://t.co/Ys0dw5hkF4 (This extends the 2015 award-winning survey https://t.co/7goTtI5Uwv)

[Kameron Decker Harris @KameronDHarris]

I'd add that not everything is fairly represented in this thread

[Kameron Decker Harris @KameronDHarris]

@aliceschwarze Some context which I wasn't aware of to dampen our passions thanks to @tyrell_turing : https://t.co/LwJamkN1pL