Jirko Rubruck

PhD student @summerfieldlab and @saxelab. Interested in all learning from humans to machines.

[summerfieldlab @summerfieldlab.bsky.social @summerfieldlab]

new paper. During learning, why does it help to practice easy examples before hard ones? We provide a theory, based on deep learning models. TL;DR - curricula help with unlearning rather than learning. https://t.co/avgpgzbKHC. Led by @RubruckJirko and Alice Zhang, with @SaxeLab

[summerfieldlab @summerfieldlab.bsky.social @summerfieldlab]

@JacquesPesnot and I are very happy to share this paper. https://t.co/VNJpegYhWn. Building on work by @scychan_brains we show that human in-context and in-weight learning vary with the distributional properties of the training data in a very similar way to transformer networks.

[Satwik Bhattamishra @satwik1729]

📢New Preprint

Are there tasks that are theoretically easier for Transformers to represent but not for SSMs/RNNs (or vice versa)? Are there tasks that are easier for two-layer Transformers but not for one-layer Transformers?

🧵Our new paper explores these questions.

[Stefano Sarao Mannelli @stefsmlab]

Happy to share our paper accepted to @icmlconf where we identify a reason (of possibly many) why most ML applications don't seem to benefit from curricula

TLDR; overparameterisation-while helping in optimisation-is eroding the benefits of curricula

📄https://t.co/JQWxil8Bsv
1/5

[summerfieldlab @summerfieldlab.bsky.social @summerfieldlab]

This project was a long time coming. We used deep RL to find distribution mechanisms that encourage humans to behave sustainably, in a common-pool resource game. Led by @Miruna_Pislar and Raphael Koster
https://t.co/YHQClSL4rZ

[Ellie Holton @EleanorHolton1]

Very excited our study on how goals shape choice is out now in @NatureHumBehav with @NilsKolling @jangrohn @BrainInTheMind and Jill O'Reilly! https://t.co/rr5UhPoImS

[Aaditya Singh @Aaditya6284]

In-context learning (ICL) circuits emerge in a phase change...

Excited for our new work "What needs to go right for an induction head (IH)?" We present "clamping", a method to causally intervene on dynamics, and use it to shed light on IH diversity + formation.

Read on 🔎⏬

[Kai Sandbrink @akaijsa]

Excited that our new position piece is out! In this article, @summerfieldlab and I review three recent advances in using deep RL to model cognitive flexibility, a hallmark of human cognition: https://t.co/FX52syfUCr (1/4)

[summerfieldlab @summerfieldlab.bsky.social @summerfieldlab]

If you want to do world class AI research from within government, this is your chance. AISI is building a really strong technical team to work on AI safety - several positions now available: https://t.co/ZgRxat9Tjc

[Armin Lak @ArminLak]

How do animals learn from naïve to expert? What role does dopamine play? Find out in our new paper by @LiebanaSamuel @aeronlaffere @toschi_chiara , Louisa Schilling, Jacek Podlaski, @matfritsche, Peter Zatka-Haas, @yulonglilab, Rafal Bogacz, @SaxeLab https://t.co/rrRpUKfobs (1/5)

[Aaditya Singh @Aaditya6284]

Training your transformer for longer to get better performance? Be careful! We find that emergent in-context learning of transformers disappears in "The Transient Nature of In-Context Learning in Transformers" (https://t.co/nMM3FB1fI8, poster at #NeurIPS2023).

Read on 🔎⏬

[Satwik Bhattamishra @satwik1729]

Recently, Transformers have been shown to implement learning algorithms in-context. Key questions:
What are their limits?
Can they exploit informative examples to learn more efficiently?
How does this relate to pretrained LLMs?

Our new preprint explores these questions.
🧵

[Ben Lonnqvist @lonnqvistben]

Neural noise is a feature, not a bug!🤖🧠 New pre-print out now on how, counterintuitively, neural noise leads to unsupervised segmentation and perceptual grouping in DNNs
Read it now: https://t.co/b4M4WnTPTM
Or read the thread below 🧵👇 1/13

[Clémentine Dominé, Phd 🍊 @ClementineDomi6]

Introducing NeuralPlayground, an open-source software package that offers a reproducible way to compare hippocampal and entorhinal models against a centralised library of experimental results.🐭🤖

The beta version is available here: https://t.co/T6uRYekT5r
Give it a go!

[Ellie Holton @EleanorHolton1]

🚨New preprint alert🚨 Why do we find it so hard to abandon our goals? We use converging evidence from healthy fMRI participants and brain lesion patients to understand how goal pursuit changes neural processing! With @NilsKolling , Jill O'Reilly, @BrainInTheMind, @jangrohn (1/7)

[Jirko Rubruck @RubruckJirko]

@RevolutApp @EkaterinaJanss1 I have a similar issue but no one is replying to me, via email or in direct messages. Could you please assist me.

[Aaditya Singh @Aaditya6284]

Excited to be at #eacl2023 in Croatia this week for my first conference. Presenting this work from my @DeepMind internship last year at:

Poster session 2: May 2, 16:30
Oral session 3, Language Grounding and Multimodality: May 3, 11:15

[Andrew Saxe @SaxeLab]

Excited to share our paper at #ICLR2023 titled: On the Specialization of Neural Modules.

Led by @devonjarvi5 with @kleinric and @BenjaminRosman

Paper: https://t.co/ltxVSJALUT

Poster #115, Wed 16:30: https://t.co/1xuq1WtPbq

(1/10)

[Machine Learning Street Talk @MLStreetTalk]

New special edition show featuring Prof. Christopher Summerfield @summerfieldlab from Oxford University / @DeepMind - we discuss his brilliant new book "Natural General Intelligence" in detail https://t.co/tK8EdJg8AA

[Aniruddh Galgali @argalgali_neuro]

I am delighted to share that my doctoral research was published online @NatureNeuro last week. Incredibly grateful to my advisor @ValerioMante, Maneesh Sahani, and John Reppas/Bill Newsome (for sharing data). A brief summary.1/13
https://t.co/MWTxZwEWSo