DB

Loss Function Observing Enthusiast

[DB @bhdebayan]

Indigo aviation fiasco is the reason why competitors are required.

[DB @bhdebayan]

Also, vibe coding entire software systems is the easiest way to lose track of it. 100% do not recommend.

[DB @bhdebayan]

Vibe coding is fun but the dopamine kick of accomplishing something via vibe code is less.

We are hardwired to get reward from tasks in which we put noticeable effort.

I enjoy vibe coding certain parts of the logic, but the entire app/software system, no way amigo.

[DB @bhdebayan]

Prompting till you get to a solution looks fun but in practice, doing hard things that you consciously recognize and accomplishing it, hits a reward that is unmatched.

[DB @bhdebayan]

In any case, this ai assistants are not going anywhere.

[DB @bhdebayan]

As I’m using the AI coding assistants more often, two things are taking hits in different ways.
Throughput and efficiency are two things that matter when coding.
Throughput is how much code is written. AI coding assistants really increase throughput.

[DB @bhdebayan]

It’s so convenient to go on autopilot and just work on solving problems from a high level through prompts. But also, losing touch with the nitty gritty of coding is also worrying.

[DB @bhdebayan]

@iScienceLuvr @welchlabs Excellent video as always! I feel inspired seeing Welch Labs videos. If anyone is interested in understanding RAG which is visually intuitive, please check out my video
https://t.co/aldt3NoN6m

#shamelessselfpromotion

[DB @bhdebayan]

What does this mean? In a future not too far from now, humans can pose questions & I mean some really difficult questions and have models like o3 perform an initial assessment. The solutions provided will be used by humans to refine answers or act as starting point.

[DB @bhdebayan]

This is a significant moment in history. The coveted ARC-AGI is beaten by o3. Although it comes at infeasible costs now but costs will reduce as more people investigate RL + Chain of Thought.

[Riley Goodside @goodside]

ARC-AGI scores for past five years of OpenAI models (updated w/ release dates)

[DB @bhdebayan]

@MaxLaves @AravSrinivas Thanks Max 🙏

[DB @bhdebayan]

I try my best to explain what is happening in RAG visually. I hope you guys like this video.

@AravSrinivas I love Perplexity. I did use some of perplexity UI to explain some parts.

https://t.co/MYqs7u994g

[DB @bhdebayan]

@3blue1brown heavily inspired by everything you do. I hope you watch it, that itself would be an honour.

[DB @bhdebayan]

I have a stories to tell about the things I find interesting. I hope you enjoy this topic as much as I did. I am happy for your feedback! I am just starting out. :)

https://t.co/T8HGStgNfJ

[DB @bhdebayan]

https://t.co/OBe54hu8fN

Find me on Bluesky

[Jürgen Schmidhuber @SchmidhuberAI]

The #NobelPrizeinPhysics2024 for Hopfield & Hinton rewards plagiarism and incorrect attribution in computer science. It's mostly about Amari's "Hopfield network" and the "Boltzmann Machine."

1. The Lenz-Ising recurrent architecture with neuron-like elements was published in 1925 [L20][I24][I25]. In 1972, Shun-Ichi Amari made it adaptive such that it could learn to associate input patterns with output patterns by changing its connection weights [AMH1]. However, Amari is only briefly cited in the "Scientific Background to the Nobel Prize in Physics 2024." Unfortunately, Amari's net was later called the "Hopfield network." Hopfield republished it 10 years later [AMH2], without citing Amari, not even in later papers.

2. The related Boltzmann Machine paper by Ackley, Hinton, and Sejnowski (1985) [BM] was about learning internal representations in hidden units of neural networks (NNs) [S20]. It didn't cite the first working algorithm for deep learning of internal representations by Ivakhnenko & Lapa (Ukraine, 1965)[DEEP1-2][HIN]. It didn't cite Amari's separate work (1967-68)[GD1-2] on learning internal representations in deep NNs end-to-end through stochastic gradient descent (SGD). Not even the later surveys by the authors [S20][DL3][DLP] nor the "Scientific Background to the Nobel Prize in Physics 2024" mention these origins of deep learning. ([BM] also did not cite relevant prior work by Sherrington & Kirkpatrick [SK75] & Glauber [G63].)

3. The Nobel Committee also lauds Hinton et al.'s 2006 method for layer-wise pretraining of deep NNs (2006) [UN4]. However, this work neither cited the original layer-wise training of deep NNs by Ivakhnenko & Lapa (1965)[DEEP1-2] nor the original work on unsupervised pretraining of deep NNs (1991) [UN0-1][DLP].

4. The "Popular information" says: “At the end of the 1960s, some discouraging theoretical results caused many researchers to suspect that these neural networks would never be of any real use." However, deep learning research was obviously alive and kicking in the 1960s-70s, especially outside of the Anglosphere [DEEP1-2][GD1-3][CNN1][DL1-2][DLP][DLH].

5. Many additional cases of plagiarism and incorrect attribution can be found in the following reference [DLP], which also contains the other references above. One can start with Sec. 3:

[DLP] J. Schmidhuber (2023). How 3 Turing awardees republished key methods and ideas whose creators they failed to credit. Technical Report IDSIA-23-23, Swiss AI Lab IDSIA, 14 Dec 2023. https://t.co/Nz0fjc6kyx

See also the following reference [DLH] for a history of the field:

[DLH] J. Schmidhuber (2022). Annotated History of Modern AI and Deep Learning. Technical Report IDSIA-22-22, IDSIA, Lugano, Switzerland, 2022. Preprint arXiv:2212.11279. https://t.co/Ys0dw5hkF4 (This extends the 2015 award-winning survey https://t.co/7goTtI5Uwv)

[DB @bhdebayan]

About time that two people who
have really revolutionised the world & ushered the Al age get recognised by the Nobel prize committee.
Unconventional pick for Nobel prize in physics, but absolutely deserving. I would also add @ylecun & @SchmidhuberAI .

[The Nobel Prize @NobelPrize]

BREAKING NEWS
The Royal Swedish Academy of Sciences has decided to award the 2024 #NobelPrize in Physics to John J. Hopfield and Geoffrey E. Hinton “for foundational discoveries and inventions that enable machine learning with artificial neural networks.”

[The Nobel Prize @NobelPrize]

BREAKING NEWS
The Royal Swedish Academy of Sciences has decided to award the 2024 #NobelPrize in Physics to John J. Hopfield and Geoffrey E. Hinton “for foundational discoveries and inventions that enable machine learning with artificial neural networks.”