π§΅We discovered a new phenomenon in speculative decoding models that we call attention drift.
As the drafter generates tokens, its attention moves from the "sink" onto its own recently-generated tokens.
Fixing the underlying issue recovered up to 2Γ acceptance length, but why?
Missing #ICLR2026 but our time-series interpretability paper is in good hands, Prof. @zhuqical is presenting tomorrow!
TL;DR: linear composition of symbolic + latent reps β temporal attributions without hurting accuracy.
π Thu 10:30aβ1p BRT
π P3-#1320
https://t.co/BUbPmJnkDu
The best start to a Friday with my favorite (and first) collaborator aka friend, is getting feedback from the youngest minds in Computer Engineering ;)
#joysofresearch
Excited to be at #NeurIPS2025 in San Diego till December 8. I am presenting two spotlight papers on practical ML for time-series! More details on the papers soon, but if you want to catch up/hunt for fun mixers/find good coffee/run along the harbor, hit me up!
Especially the quote -- "An interesting feature is also the band of integers assigned unique tokens in the 1900-2000 region. These represent common dates β i.e. from 1930-2020 are all assigned unique tokens because these dates occur most frequently in the training set" [2/2]
Well I have always had trouble getting on-board with papers on how general-purpose LLMs can be used for zero-shot time-series forecasting, time-series classification or really anything requiring non-trivial analyses of numerical data!
π Paper : https://t.co/qBi0IOPcK6
π» Codebase & processed dataset: https://t.co/SyrGOBLFR5
We invite you to explore our work and would love to hear your thoughts on incorporating linguistic biosignals as a modality for LLMs. [n/n]
Prof @zhuqical will be presenting our poster tomorrow at #ACL2025 - "Can LLMs Understand Unvoiced Speech? Exploring EMG-to-Text Conversion with LLMs".
π Date & Time: July 29, 4:00 pm - 5:30 pm
π Location : Poster Session 3, Hall 4/5
Come and say hello :)
We demonstrate that, under closed-vocabulary conditions, our approach using LLMs (LLaMA 2 and LLaMA 3) achieves an average word error rate (WER) of 0.49, and as low as 0.39. [2/n]
Collaborating with @Boeing since 2021 (my maiden PhD project). From modeling an ambi(tious)guous target, fatigue, to ergonomic assessment with commercial wearables (coming soon...)--this has been the most fun interdisciplinary playground to grow and learn!
#wearablesandAI
This resonates well with me, a fellow interdisciplinary researcher, who has been the recipient of one too many dogmatic encounters with CS researchers claiming to be fundamental-scientists.
Why is academic computer science (an applied field) so enthralled with conceptual purity? Is it because we've achieved similar status but still envy "purer" displines? Are we threatened by interdisclinarity from within as we infiltrate so many other fields?
Aiming to improve occupational health among manufacturing workers, Ping Guo and Qi Zhu led a team of researchers in the development of a wearable multimodal sensor system that leverages ML to enable near-real-time fatigue prediction on the factory floor. https://t.co/GKpYhbZajF
A system that measures heart rate, skin temperature, and locomotive signs continuously monitors manufacturing workers for signs of fatigue, helping prevent mistakes, injuries, and the development of chronic conditions. In PNAS Nexus: https://t.co/lmxz5O8zgM
BREAKING NEWS
The Royal Swedish Academy of Sciences has decided to award the 2024 #NobelPrize in Physics to John J. Hopfield and Geoffrey E. Hinton βfor foundational discoveries and inventions that enable machine learning with artificial neural networks.β