The first edition of Learning Over Time (LOT) Spring School is starting!
@StefanoMelacci kicking off the school at the wonderful Certosa di Pontignano, Italy
Organized by @StefanoMelacci, @v_lomonaco , @Cossu94 , @aleitteb , and myself
https://t.co/KGi7gEYpPy
#CoLLAs2024 I will present our novel architectural solution for Continual Learning, Memory Head, at the upcoming @CoLLAs_Conf in Pisa!
Paper and video: https://t.co/Ufdx30atIQ
๐Pass by at our poster on Tuesday 30th and to the oral presentation on Wednesday 31th morning!
๐ข "State-space Modeling in Long Sequence Processing: A Survey on Recurrence in the Transformer Era"
https://t.co/VaUfp3LZnl
a joint work by @TiezziMatteo , M. Casoni, @aleitteb , M.Gori (@KleineBottleM) and @StefanoMelacci
3โฃ RNNs evolutions, such as xLSTMs (@gklambauer), ODE-inspired RNNs (Neural Oscillators - @tk_rusch)
as well as novel paths for
4โฃ Learning in RNNs, such as modern evolutions of RTRL (@KhurramJaved_96) or bio-inspired methods (@BerenMillidge)
https://t.co/VaUfp3LZnl
We described several trends:
1๏ธโฃ #Transformers embraced Recurrence exploiting a stateful recurrent representation:
*๏ธโฃ all the latest models RWKV4/5/6 by @BlinkDL_AI
*๏ธโฃ #RetNet by @donglixp et al.
are competitive with quadratic self-attention!
https://t.co/VaUfp3LZnl
We summarized the key ingredients underlying the current models generation:
*๏ธโฃ linear recurrence
*๏ธโฃ element-wise recurrence
*๏ธโฃ gating mechanisms
and where there might be more room for novel research activities: #onlinelearning
https://t.co/VaUfp3LZnl
๐งInterested in the details of the latest SOTA architectures (#RWKV , #RetNet, #Mamba, #Griffin) for long sequences?
๐ข "State-space Modeling in Long Sequence Processing: A Survey on Recurrence in the Transformer Era" https://t.co/VaUfp3LZnl
๐จ1 month left until the #CoLLAs2024 paper submission deadline!๐จ
Donโt miss your opportunity to contribute to the field of lifelong learning. Finalize your research and submit by 15 Feb 2024!
๐ Prepare your submission: https://t.co/xLC1mbEJrS
๐๏ธ Abstract deadline: 09 Feb 2024
The third international Conference on Lifelong Learning Agents (CoLLAs) will happen in Pisa, Italy from 29 July to 01 Aug 2024 (right after ICML in Europe!).
Abstract deadline: 09 Feb 2024
Submission deadline: 15 Feb 2024
Full CFP: https://t.co/xLC1mbEbCk
๐ฃ Last week I presented "Foveated Neural Computation" @ECMLPKDD
โก๏ธ Paper: https://t.co/3cOHoXxLzR
โก๏ธ Code: https://t.co/qxL3EEnOBC
A thread๐๐งต
In some visual tasks, such as image classification, salient information is distributed in regions of limited size ๐ง
1/n