Pritom Gogoi

grad student @Uni_Stuttgart

[Andrew Carr 🤸 @andrew_n_carr]

I found a partial version. It's missing the "Meta Learning" selection of papers:

VARIATIONAL LOSSY AUTOENCODER

KEEP NEURAL NETWORKS SIMPLE BY MINIMIZING DESCRIPTION LENGTH OF WEIGHTS

Quantifying the Rise and Fall of Complexity in Closed Systems: The Coffee Automaton

THE FIRST LAW OF COMPLEXODYNAMICS

CHAPTER 14 KOMOGOROV COMPLEXITY

A TUTORIAL INTRODUCTION TO THE MINIMUM DESCRIPTION LENGTH PRINCIPLE

MACHINE SUPER INTELLIGENCE DISSERTATION

CS231n Convolutional Neural Networks for Visual Recognition

IMAGENET CLASSIFICATION WITH DEEP CONVOLUTIONAL NEURAL NETWORKS

DEEP RESIDUAL LEARNING FOR IMAGE RECOGNITION

MULTI-SCALE CONTEXT AGGREGATION BY DILATED CONVOLUTIONS

IDENTIFY MAPPINGS IN DEEP RESIDUAL NETWORKS

RECURRENT NEURAL NETWORK REGULARIZATION

THE UNREASONABLE EFFECTIVENESS OF RECURRENT NEURAL NETWORKS

UNDERSTANDING LSTM NETWORKS (BLOG)

DEEP SPEECH 2: END-TO-END SPEECH RECOGNITION IN ENGLISH AND MANDARIN

THE ANNOTATED TRANSFORMER

A SIMPLE NETWORK MODULE FOR RELATIONAL REASONING

NEURAL TURING MACHINES

ORDER MATTERS: SEQUENCE TO SEQUENCE FOR SETS

RELATIONAL RECURRENT NEURAL NETWORKS

ATTENTION IS ALL YOU NEED

NEURAL MACHINE TRANSLATION: BY JOINTLY LEARNING TO ALIGN AND TRANSLATE

POINTER NETWORKS

NEURAL MESSAGE PASSING FOR QUANTUM CHEMISTRY

SCALING LAWS FOR NEURAL LANGUAGE MODELS

GPIPE: EFFICIENT TRAINING OF GIANT NEURAL NETWORKS USING PIPELINE PARALLELISM

[Chris Olah @ch402]

Research Intimacy: Internalizing obscure knowledge, equations, relationships, and ways of thinking related to a research topic.

[Pritom Gogoi @rustyelectron]

https://t.co/BBYJW026dQ

[Pritom Gogoi @rustyelectron]

The Mind's A Field 🤔
https://t.co/aL6TLOo2r8

[Vrushank Desai @vrushankdes]

I spent a couple months at the beginning of this year learning about GPU programming through trying to optimize inference for @chichengcc awesome Diffusion Policy paper. I was able to improve inference time for the denoising U-Net by ~3.4x over Pytorch eager mode and ~2.65x over Pytorch compile mode! I wrote a 9-part blog post (linked in the last post of this thread) that builds up from the physical structure of DRAM/SRAM cells all the way up to integrating custom CUDA kernels in Pytorch.

A thread of the most interesting things I learned…🧵

This one is unrelated to the Diffusion inference stuff but imo, more amusing… I was able to get my RTX 3090 inductor coils to play ‘Twinkle Twinkle Little Star’ using kernels (GPU programs) that modulate power draw at the right frequencies! What’s happening here is each kernel launch triggers a surge of in-rush current in the GPU’s DC-DC step-down inductors. The Lorentz force due to the change in current (proportional to change in current divided by the change in time) causes the coil to move slightly. If we play with the kernel launch frequencies we can vibrate the coils and get noises in the audible range. Unfortunately we can’t make sounds lower than 2000Hz because the ‘change in time’ part of the equation becomes too large, and the resulting vibration is too weak to make audible noise. So we end up with Twinkle Twinkle shifted up many octaves 😀

[Riley Goodside @goodside]

AI-generated sad girl with piano performs the text of the MIT License

[Pritom Gogoi @rustyelectron]

One of the most satisfying feelings is that of closing down tabs, articles and papers after the exam is over.

[activewarp @activewarp]

checking in on what RWKV looks like nowadays

[Pritom Gogoi @rustyelectron]

I look for a video on a obscure topic on youtube.
No result is found or irrelevant ones are returned.

A few days later, I randomly open youtube.
A video on the exact topic that I searched pops up.

Nice.

[Pritom Gogoi @rustyelectron]

Ever since I discovered https://t.co/k1wF8ndwGu, I have been adding 10+ links to my never-ending reading list every day.

[Yam Peleg @Yampeleg]

Men will literally buy 600,000 GPUs instead of going to therapy

[James Campbell @jam3scampbell]

People are really bad at understanding just how big LLM's actually are. I think this is partly why they belittle them as 'just' next-word predictors

[shaurya @shauseth]

guy who despises the course all semester only to fall in love with it 24 hrs before the final

[Pritom Gogoi @rustyelectron]

New era for video game NPCs?

[Aleksa Gordić (水平问题) @gordic_aleksa]

Creating intelligent simulated worlds is increasingly becoming a reality. 🌎🤖 This paper blows my mind. 25 agents (ChatGPTs + memory streams in the background) go about their day, observe, plan, reflect, and communicate with each other to achieve user-specified goals 🤯

1/ 🧵👇

[Amit Chaudhary @amitness]

Developed a browser extension to access and edit latex of any https://t.co/0GNtNbJSii paper directly on @overleaf

Install: https://t.co/wbgvFY0OcP

Hope it's useful to students/researchers working a lot with latex to see how other papers implement stuff

[AK @_akhaliq]

Its over

run Large Language Models like LLaMA, llama.cpp, GPT-J, Pythia, OPT, GALACTICA, gpt4all, auto-gpt easily in a web ui, free, and open source

github: https://t.co/ubmHm7QCWW

[Aleksa Gordić (水平问题) @gordic_aleksa]

The rate of progress in LLM optimization is just mind-blowing! 🤯

Running 13B LLMs like LLaMA on edge devices (e.g. MacBook Pro with an M1 chip) is now almost a breeze.

I remember when this looked like a distant future not too long ago. I feel old lol. 😅

1/ 🧵👇

[0xDesigner @0xDesigner]

Probably the best thing you'll see today.

In 2017, a group of developers hilariously competed for who could create worst volume control interface in the world.

The results 🧵

1/22

[Rajat Kanti Bhattacharjee @simpuleguy]

Curious and looking for genuine inputs.

How relevant is a Cognitive Science course / background for A.I research. Are the subject/ skills something that should be in your belt. Ignoring the answers that give the idea that I can learn the same from YouTube
#ArtificialIntelligence

[Pritom Gogoi @rustyelectron]

I once wrote a boring explainer article on 2's complement but this explains it so much better.