Kyle

NeuroAI Scholar @CSHL | EE PhD @UWMadison | Comp. Eng. / math @RoseHulman | CS + neuro + AI

[Cold Spring Harbor Laboratory @CSHL]

CSHL’s Kyle Daruwalla discovered a novel approach for AI algorithms to process data more efficiently, resembling the ways in which the human brain learns. 🧠 https://t.co/T9qlWC708b

[Helen Hou @HelenxHou]

We present Cheese3D, a system to sensitively track 3D movement of the entire face in lab mice.

First #preprint from the lab!
https://t.co/bbKML7nn2Y 1/n

[Kyle @darsnack]

@DimitrisPapail Also Makie.jl and Pluto.jl for interactive demos

[Kyle @darsnack]

@DimitrisPapail If it’s for linear algebra, then I would say Julia hits the sweet spot of intuitive syntax. Even better than Matlab. But if this course goes all the way up to DL pipelines…hard to beat PyTorch.

[Kyle @darsnack]

@DimitrisPapail @agi_watcher At best you have the exact gradient and at worst a really noisy one. Reducing how animals learn to computing an approximate forward gradient is unlikely to result in a “better” model.

[Kyle @darsnack]

@DimitrisPapail Data movement is the energy hog in large, distributed training right? Less memory = less to move. Plus any forward algo is presumably converging in the presence of gradient noise. Those tricks might also mean resistance to noise from asynchronous, distributed updates.

[Kyle @darsnack]

@DimitrisPapail Those seem like important implications as you scale up the model size. Or if you have recurrence in play (I’m thinking scaling up https://t.co/NvWhG9wvit). The best answer for gradient propagation through time has been “make time into space” (i.e. transformers).

[François Chollet @fchollet]

The capabilities of LLMs are now causing a resurgence in folks saying that humans are "statistical parrots" mindlessly predicting the next word based on what they've heard before.

5 minutes of observing a young child still learning to speak should cure you of this notion.

[317070 @317070]

Did you know, that you can build a virtual machine inside ChatGPT? And that you can use this machine to create files, program and even browse the internet? https://t.co/15IwHwr2on

[Kyle @darsnack]

@DimitrisPapail Aren’t they both evidence for the prediction? Second shows that shortcuts are limited the # of steps in the training data unless you change the problem to force it to learn the recurrence relation. Adding recurrence to the model is a more flexible solution.

[Brad Aimone @jbimaknee]

A dramatic example of this is how we engineer randomness into computers. Our amazing pseudo random number generators are reliable, predictable and rely on deterministism all the way down to transistors whereas biology is random at all levels and becomes deterministic when needed

[Timoleon (Timos) Moraitis @timos_m]

#NeuroAI: Could principles of embodied sensorimotor neuroscience unify and improve the various Self-Supervised Learning (SSL) methods? How could the brain self-supervise itself?

We are happy to share our #NeurIPS2022 paper https://t.co/gLOgi44knA with @franz_scherr and Q. Guo🧵:

[Kyle @darsnack]

@neurograce For stuff that’s mostly complete but you decided isn’t worth pursuing further. Somehow it seems wrong that these ideas are never disseminated. Anything that could be a preprint but still worth pursuing is something that I assume you’ll hang onto for next year.

[Kyle @darsnack]

This is a joke, but also is a neat idea. On a per lab basis, if you can only publish one paper per year (unlimited pre-prints), what would you work on?

This constraint kinda forces within a lab all the good behaviors that we aspire towards.

[Tony Zador @TonyZador]

White paper —Rallying cry for NeuroAI to work toward Embodied Turing Test !

Let’s overcome Moravec’s paradox: Tasks “uniquely” human like chess and even language/reasoning are much easier for machines than “easy” interaction with the world which all animals all perform.

[Kyle @darsnack]

Sometimes you come across numbers that are so mind boggling that you have to share them. Some people in America pay $100 / Mbps that others pay $0.25 / Mbps (in the same city)! We desperately need a scientific corps in Congress.

https://t.co/mHe2d0SqGl

[typedfemale @typedfemale]

who would win? millions of TPU-hours of RL or two cheeky austrians 😜 https://t.co/80S5gHZ6bW

[Neuromatch @neuromatch]

Scholars in all disciplines & countries, we invite you to sign this open letter to @WHOSTP calling for equitable access & participation in research for all

Why we think you should sign
https://t.co/VzgUMMxdnk

Or go straight to the letter
https://t.co/Yok9PzAeso

#OSTPLetter

[Kyle @darsnack]

@lerlerimed @gta_rollergirl @UWMadison Wow. I didn’t even know that rule for international students. That’s messed up.