Rainer Engelken

Assistant Professor, ECE, CS, Neuroscience at UIUC | Theoretical neuroscience, neural dynamics, spiking networks | Recruiting PhDs & Postdocs

[Dmitri "Mitya" @chklovskii]

What do control theory and neuroscience have in common? Shared origins in cybernetics — and, lately, “poor cousin” status next to ML. To reconnect them, we’ve been organizing workshops at this interface, most recently at the Flatiron Institute. Thanks to all who joined!

[Rainer Engelken @RainerEngelken]

We are recruiting PhD students and postdocs at UIUC ECE interested in learning dynamics, spiking networks, and dynamical systems. Details: https://t.co/Ge1hy1EC5K

[Rainer Engelken @RainerEngelken]

Belated update: I joined UIUC ECE as an Assistant Professor. Our lab works at the intersection of theoretical neuroscience, machine learning, and dynamical systems, with a focus on learning and spiking networks. I had to miss #COSYNE2026 for visa reasons.

[Rainer Engelken @RainerEngelken]

@GregKamradt Friendly note: Shouldn't it be 'intelligence per Joule' instead of 'per Watt' here? Alternatively per g CO2e or per FLOP.

[Rainer Engelken @RainerEngelken]

@sreejan_kumar Congratulations, Sreejan!

[Rainer Engelken @RainerEngelken]

@loganclarkhall "Once the rockets are up, who cares where they come down? That's not my department," says Wernher von Braun."
https://t.co/jAycbt3r6s

[Rainer Engelken @RainerEngelken]

@RadishHarmers "Once the rockets are up, who cares where they come down? That's not my department," says Wernher von Braun.
https://t.co/jAycbt3r6s

[Rainer Engelken @RainerEngelken]

@StphTphsn1 @jeffreyseely @pfau @gradientpub Yes

[Dan Goodman @neuralreckoning]

The #SNUFA24 final program is out and the event is next Tue-Wed! If you love spiking neural networks, click on the link below to check it out and register (free).

https://t.co/U7TGfWN815

[Rainer Engelken @RainerEngelken]

@SussilloDavid 'convergent learning' https://t.co/E1lFbWwunv
One could also call it 'convergent evolution of representations'.

[Rainer Engelken @RainerEngelken]

@ai_ngrosso @DondersInst @Radboud_Uni Great news, well deserved. Congrats Alessandro!

[Rainer Engelken @RainerEngelken]

@Antihebbiann @HSompolinsky @yasharadian See also the related work by Bruno Cessac et al:
https://t.co/HaBSaq1ZrG
https://t.co/Afi4fyXcWS
https://t.co/qgr1t3wZg5

[Rainer Engelken @RainerEngelken]

@Antihebbiann @HSompolinsky As @yasharadian pointed out, there is previous work on the finite-N effects on the chaos transition in discrete-time networks (with delays) by David Albers et al. that I only recently learned about:
https://t.co/aC2zUp3g0y
https://t.co/2wjzk4pfH7
https://t.co/8djG6bzoYW

[Rainer Engelken @RainerEngelken]

Link to the paper once more: https://t.co/Wz4q1PDAiS

[Rainer Engelken @RainerEngelken]

I thank S. Goedeke, J. Liedtke, R.-M. Memmesheimer, M. Monteforte, A. Palmigiano, M. Puelma Touzel, A. Schmidt, M. Schottdorf, and F. Wolf for fruitful discussions. Moreover, I thank L.F. Abbott, S. Migirditch, A. Palmigiano, Y. Park, and D. Sussillo for comments on a draft.
12/n

[Rainer Engelken @RainerEngelken]

SparseProp doesn't work (yet) for stochastic input most multivariate neuron models. Advantageous scaling of O(log(N)) only for sparse networks (Fixed number of synapses per neuron). A more fundamental limitations: it is not clear how to calculate surrogate gradients.
11/n

[Rainer Engelken @RainerEngelken]

I gave a talk about some of the core idea of SparseProp at #JuliaCon some years ago, but this was an earlier version of Julia and didn't include heterogeneous networks, Chebychev polynomials and network training. https://t.co/zHlmGxN5qY
10/n

[Rainer Engelken @RainerEngelken]

Excited to present this afternoon another #NeurIPS paper on #SparseProp, which is a novel event-based algorithm to simulate and train spiking neural networks, reducing computational cost from N to log(N) per network spike for sparse spiking networks:
https://t.co/Wz4q1PDAiS
1/n

[Rainer Engelken @RainerEngelken]

Example code for networks of QIF, LIF and EIF neurons in julia for SparseProp is available at https://t.co/2WaVrbIzlF
#JuliaLang @JuliaLanguage
9/n

[Rainer Engelken @RainerEngelken]

SparseProp can also be used for event-based training of spiking networks, the backward pass has the same complexity as the forward pass. Caveat: Many people use surrogate gradients and it is not clear to me how to do that in event-based simulations. Any ideas? @hisspikeness
8/n

[Rainer Engelken @RainerEngelken]

For neurons where the next spike time can't be obtained analytically, e.g. exponential integrate-and-fire, SparseProp is still an option. Instead of the analytical phase response curve (PRC), the numerical PRC can be approximated precisely using Chebychev polynomials.
7/n