Stefano Melacci

Out of This World

[Matteo Tiezzi @TiezziMatteo]

The first edition of Learning Over Time (LOT) Spring School is starting!
@StefanoMelacci kicking off the school at the wonderful Certosa di Pontignano, Italy
Organized by @StefanoMelacci, @v_lomonaco , @Cossu94 , @aleitteb , and myself

https://t.co/KGi7gEYpPy

[Matteo Tiezzi @TiezziMatteo]

🚀 Learning Over Time (LOT) Spring School 📅 24-27 March 2025 | 📍 Siena, Italy
🔗https://t.co/9FYi3vZAgR

Organizers: @StefanoMelacci ,@v_lomonaco @Cossu94 ,@aleitteb @TiezziMatteo

#ContinualLearning #CL #LifelongLearning #CollectionlessAI

[Stefano Melacci @StefanoMelacci]

@vgaraujov Hi, are you sharing the code?

[Stefano Melacci @StefanoMelacci]

The call for two post-doc positions in #CollectionlessAI is open (University of Siena, Italy)! Have a look at the link in this post 😀

#CollectionlessAI #ContinualLearning #LifelongLearning #MachineLearning #AI

@StefanoMelacci
@KleineBottleM

https://t.co/UuJEBv3747

[Stefano Melacci @StefanoMelacci]

🚀 We’ve launched a new site for Collectionless AI, sharing our take on next-gen AI/ML! 🚀 Explore our papers, events, and more!

https://t.co/WJQxqlj25t

#CollectionlessAI #ContinualLearning #LifelongLearning #MachineLearning #AI 🌐

@StefanoMelacci @KleineBottleM @TiezziMatteo

[Federico Becattini @Fede_Becat]

Heat and social dinner @CoLLAs_Conf #collas2024

[Matteo Tiezzi @TiezziMatteo]

#CoLLAs2024 I will present our novel architectural solution for Continual Learning, Memory Head, at the upcoming @CoLLAs_Conf in Pisa!

Paper and video: https://t.co/Ufdx30atIQ

📝Pass by at our poster on Tuesday 30th and to the oral presentation on Wednesday 31th morning!

[Matteo Tiezzi @TiezziMatteo]

Hurry up for @CoLLAs_Conf registration!

[Matteo Tiezzi @TiezziMatteo]

📢 "State-space Modeling in Long Sequence Processing: A Survey on Recurrence in the Transformer Era"
https://t.co/VaUfp3LZnl

a joint work by @TiezziMatteo , M. Casoni, @aleitteb , M.Gori (@KleineBottleM) and @StefanoMelacci

[Matteo Tiezzi @TiezziMatteo]

We also summarized the most common #benchmarks and settings for long sequence processing and #longrange dependencies https://t.co/VaUfp3LZnl

[Matteo Tiezzi @TiezziMatteo]

3⃣ RNNs evolutions, such as xLSTMs (@gklambauer), ODE-inspired RNNs (Neural Oscillators - @tk_rusch)
as well as novel paths for
4⃣ Learning in RNNs, such as modern evolutions of RTRL (@KhurramJaved_96) or bio-inspired methods (@BerenMillidge)
https://t.co/VaUfp3LZnl

[Matteo Tiezzi @TiezziMatteo]

We described several trends:
1️⃣ #Transformers embraced Recurrence exploiting a stateful recurrent representation:
*️⃣ all the latest models RWKV4/5/6 by @BlinkDL_AI
*️⃣ #RetNet by @donglixp et al.
are competitive with quadratic self-attention!

https://t.co/VaUfp3LZnl

[Matteo Tiezzi @TiezziMatteo]

We summarized the key ingredients underlying the current models generation:
*️⃣ linear recurrence
*️⃣ element-wise recurrence
*️⃣ gating mechanisms
and where there might be more room for novel research activities: #onlinelearning

https://t.co/VaUfp3LZnl

[Matteo Tiezzi @TiezziMatteo]

Goal: efficiently 𝐩𝐫𝐨𝐜𝐞𝐬𝐬 𝐥𝐨𝐧𝐠 𝐬𝐞𝐪𝐮𝐞𝐧𝐜𝐞𝐬 + 𝐩𝐫𝐞𝐬𝐞𝐫𝐯𝐞 𝐥𝐨𝐧𝐠 𝐭𝐞𝐫𝐦 𝐝𝐞𝐩𝐞𝐧𝐝𝐞𝐧𝐜𝐢𝐞𝐬
TL;DR #Transformers + #RNNs, #DeepStatespace models ➡️ progressive confluence of previously disjoint architectures

https://t.co/VaUfp3LZnl

[Matteo Tiezzi @TiezziMatteo]

🧐Interested in the details of the latest SOTA architectures (#RWKV , #RetNet, #Mamba, #Griffin) for long sequences?

📢 "State-space Modeling in Long Sequence Processing: A Survey on Recurrence in the Transformer Era" https://t.co/VaUfp3LZnl

[Matteo Tiezzi @TiezziMatteo]

‼️Short survey paper‼️

📢"On the Resurgence of Recurrent Models for Long Sequences: Survey and Research Opportunities in the Transformer Era"

by @TiezziMatteo M.Casoni @aleitteb @_masaccio_ @KleineBottleM @StefanoMelacci

#Transformers #SSMs

https://t.co/KYLdy4TXNR

[James Smith @jamessealesmith]

📢📢📢The abstract deadline for #CoLLAs2024 is tomorrow 👇

[CoLLAs 2026 @CoLLAs_Conf]

🚨1 month left until the #CoLLAs2024 paper submission deadline!🚨

Don’t miss your opportunity to contribute to the field of lifelong learning. Finalize your research and submit by 15 Feb 2024!

📄 Prepare your submission: https://t.co/xLC1mbEJrS
🗓️ Abstract deadline: 09 Feb 2024

[Stefano Melacci @StefanoMelacci]

See you all in Pisa 😃 !!!!!! (July 2024)

[CoLLAs 2026 @CoLLAs_Conf]

The third international Conference on Lifelong Learning Agents (CoLLAs) will happen in Pisa, Italy from 29 July to 01 Aug 2024 (right after ICML in Europe!).
Abstract deadline: 09 Feb 2024
Submission deadline: 15 Feb 2024
Full CFP: https://t.co/xLC1mbEbCk

[Matteo Tiezzi @TiezziMatteo]

📣 Last week I presented "Foveated Neural Computation" @ECMLPKDD
➡️ Paper: https://t.co/3cOHoXxLzR
➡️ Code: https://t.co/qxL3EEnOBC

A thread👇🧵
In some visual tasks, such as image classification, salient information is distributed in regions of limited size 🧐
1/n