stash

PhD Candidate @OISTedu. Core expertise in ML & Dynamical Systems. Passionate about applications in Psychology & Psychiatry.

[stash @stash_196]

Happy to announce that our paper on Structured State Space Models has been accepted to #AISTATS2026. ☺️

We introduce a geometric framework using warped basis projections to construct Structured SSMs without explicit ODE formulation or discretization.

Interestingly, this formulation naturally leads to a Lag operator interpretation. 📉

Please take a look 👀: https://t.co/KeE1UhFDkf

[stash @stash_196]

🔖

[Earl K. Miller @MillerLabMIT]

Traveling waves may underlie working memory (and many other functions).

Traveling brain waves support flexible storage of human working memory
https://t.co/lwA8KpwfnU
#neuroscience

[stash @stash_196]

🔖

[alphaXiv @askalphaxiv]

New paper advised by Yann LeCun!

"H-JEPA: End-to-End Learning of Hierarchical World Models for Visual Planning"

Most world models plan in a single latent space and at one timescale, which makes long-horizon planning expensive and forces one representation to handle both low-level motion and high-level goals.

H-JEPA instead stacks JEPA world models across multiple timescales, with higher levels predicting farther into the future.

The highest level first figures out roughly where the agent should go, then each lower level turns that into increasingly concrete subgoals until the bottom level outputs actual actions.

On Visual AntMaze, this 3-level setup increases success from 18% to 73% while using less planner compute.

https://t.co/dQrd4iT1T9

[stash @stash_196]

🔖

[CLaE @leafs_s]

Neuron

Inferring brain-wide interactions using data-constrained recurrent neural network models

https://t.co/3shKoSwOfO

[RIKEN Center for Brain Science @RIKEN_CBS]

甘利先生の記事、後編が公開されました��🤖🤖

AI巨大化競争の中、日本が進むべき2つの道　甘利俊一氏が語る「量ではなく理論」で勝負するAI｜JBpress Innovation Review https://t.co/bzxNcfaSEo

[stash @stash_196]

🔖

[佐藤 竜馬 / Ryoma Sato @joisino_]

思考の連鎖 (Chain of Thought; CoT) はいわば「喋りながら考えている」ことに相当しますが、最近の LLM は「黙って」考えられるようになってきているんですね

https://t.co/snYUqQifnY

[stash @stash_196]

🔖

[Kaede Shiohara @KaedeShioharaCS]

新たな生成モデル Empirical Variational Autoencoder (EVA)を提案しました

VAEの事前分布制約を標準正規分布ではなく自己予測分布に置き換える事で、生成モデルとして使われなくなっていたVAEで高品質サンプリングが可能になります
自己予測分布はVAEデコーダーに線形層を1つ足すだけで実��できます

[stash @stash_196]

🔖

[河野健一 Kenichi Kono ｜ 脳外科医 CEO｜AI 医療 MBA｜脳血管内手術支援AI @CeoImed]

なんと！！！

｢脳は意味から、次に来る音を予測｣
Nature Communications

･ ヒト3人、624神経細胞を記録
･ 音韻と意味は別の神経空間で表現
･ 予測した意味に、後の音の活動が整列
･ 脳は音を受け取る���けでなく、文脈から聞こえ方を先回りして組み立てる可能性も

https://t.co/ICigYji6hp

[stash @stash_196]

🔖

[Luiz Pessoa @PessoaBrain]

𝗛𝗲𝘁𝗲𝗿𝗮𝗿𝗰𝗵𝘆 𝗶𝗻 𝘁𝗵𝗲 𝗯𝗿𝗮𝗶𝗻
New preprint with Andrea Gambarotto. Neuroscience often treats control as a hierarchy: the structures behind a behavior hold a stable rank. We argue this mistakes a configuration for an architecture. 🧵
https://t.co/wvvqBPCBWU

[stash @stash_196]

🔖

[橋本幸士 Koji Hashimoto @hashimotostring]

甘利先生が「物理屋のための機械学習講義」にご登壇．
11/6の回．お見逃しなく．#学習物理学
https://t.co/l3LyNoVN2G

[stash @stash_196]

🔖

[Samip @industriaalist]

Backprop has been the only credit assignment algorithm capable of training large neural nets.

Introducing Dust, the first zeroth-order method to pretrain transformers to approach and sometimes even *exceed* backprop with large amounts of computation (bitter lesson!)

- Dust is on the order of 1,000 to 10,000x more compute efficient than EGGROLL, the state-of-the-art ES method, for training transformers.

- We chose pretraining transformers because it's arguably the hardest task possible for zeroth-order optimization.

- Search in high dimensions is really misunderstood: zeroth-order usually gets better with overparameterization, not worse, i.e. at a fixed population size, larger models (up to 120x) reach a lower loss than smaller ones.

- Backprop-like gradients emerge from a large population of activation perturbations, and the alignment holds up at every scale we tested, up to 1B tokens, which is critical for scaling.

The core idea is a *virtual population*, which helps scale to large populations in transformers by perturbing activations in parallel.

w/ @bishmdl76, @cs_serdar, @akshayvegesna

[stash @stash_196]

🔖

[Oliver Sieberling @ COLM @osieberling]

Linear attention is arguably the most naive RNN, but still massively outperforms traditional RNNs by maintaining a matrix state. So... what if we use a (triadic) outer product of three vectors and maintain a three-dimensional state?

Introducing: Triadic Linear Attention 🧵

[stash @stash_196]

🔖

[Yuval Ran-Milo @YuvalRanMilo]

A sweet wrap-up to my visit at @Princeton: my work with @HazanPrinceton and Angelos Assos has been accepted to #NeurIPS2026! 🎉

We show how to learn linear dynamical systems with a small memory footprint. Here’s the idea 🧵

[stash @stash_196]

🔖

[島袋隼士(Hayato Shimabukuro) @bukuro8810]

新しい論文で使ったのがSurrogateという方法。Greigらの論文（https://t.co/brPXcJfScY）でも紹介さ��たこの方法は、信号の位相をランダム化することで、フーリエ振幅情報は維持したまま、信号の形態や相関情報を破壊することができます。つまり、フーリエ振幅は保ちつつ（パワースペクトルはほぼ一緒）、破壊に依って失われた信号の形態や相関情報の持つ非ガウス性の分析に有効です。

[stash @stash_196]

🔖

[Mustafa. @oprydai]

pid control is one of those ideas that looks like an equation until you watch a robot actually move. imagine commanding a robotic arm to follow a trajectory. the controller constantly asks one question: where should the joint be, and where is it right now? the difference is the error. proportional control reacts to the error that exists now. large error means strong correction, small error means gentle correction. increase it too much and the robot becomes aggressive, overshoots and oscillates.

integral control remembers the past. if gravity, friction or some constant disturbance keeps leaving the joint slightly below its target, proportional control may keep producing a persistent error. the integral term accumulates that error over time and keeps increasing the correction until the bias disappears. derivative control looks in the opposite direction: it cares about how quickly the error is changing. if the robot is approaching the target too fast, derivative action pushes against that motion. think of proportional as the spring, integral as the memory, and derivative as the damper.

the interesting part is that pid is really a tiny feedback intelligence loop. observe the system, compare reality with intent, calculate the error, act, observe again. do this hundreds or thousands of times per second and a motor that knows nothing about trajectories can become a precise controlled joint. stack these loops across a robot and you begin turning electrical energy into coordinated physical behavior. before learning exotic control algorithms, understand pid deeply. it teaches the central idea behind control itself: continuously use feedback from reality to correct your model of what should happen.

[柴田 淳 - 「みんなのPython」の著者・機械学習講師 @ats]

RNNは前の状態を使って次を計算するため、時間方向には順番に処理するしかない――これが長年の弱点でした。ところがこの研究は、非線形RNNを時間方向にも並列化し、3万ステップを超える長い時系列で、従来の逐次学習に対して最大870倍の高速化を報告しています。

鍵になるのがDEERという手法です。RNNを「時刻1→2→3と順番に計算するもの」として扱うのでは��く、全時刻の状態を同時に満たす方程式を解く問題へ置き換えます。さらに、カオス系で計算が不安定になる問題をGeneralized Teacher Forcing（GTF）で抑え、非線形なRNNでも長い系列を並列学習できるようにしました。

ここで重要なのは、単にRNNが速くなったことではありません。TransformerやMambaなどが広がった背景には、RNNの逐次処理がGPUと相性が悪いという事情がありました。その制約がアルゴリズム側から外れるなら、「再帰モデルは表現力があるが遅い」というこれまでのトレードオフ自体が変わる可能性があります。

高速化によって、この研究はもう一つ重要な問いを検証できるようになりました。同じ量のデータを使うなら、短い時系列を大量に学習するのと、長い時系列を連続して学習するのは同じなのか、という問いです。
結果は同じではありませんでした。Lorenz-96や神経活動モデルでは、総データ量を揃えても、長い連続系列で学習したRNNほど長期的な力学構造を正確に再現しました。一方で短期予測性能は、比較的短い系列でも早い段階で頭打ちになります。

https://t.co/BjqxbVRxrn

[stash @stash_196]

🔖

[Juno KIM @junokim_ai]

What's the best way to solve linear regression? (No, it’s not least squares.)
In our new paper, we show that principal component regression (PCR) beats, up to constants, every monotone spectral filter (incl. gradient descent & ridge regression) on every problem instance! (1/8)

[stash @stash_196]

🔖

[Masanobu Inubushi 犬伏 正信 @MasaInubushi]

京都大学 数理解析研究所にて講演をしました🎤
「AI���一般に外挿が苦手」と言われますが，カオスが含まれないデータで学習をして，カオスを予測する手法（分岐点を超えた外挿法）を提案しました．
ちゃっかり雑誌「数理科学」の宣伝もしました📣
https://t.co/iAfpdAnKCK

[Naval @naval]

When you look back, you’ll wish you’d just done the long-term version of everything.

[stash @stash_196]

🔖

[よえ @yoeee145]

第1回数理脳科学セミナー 「深層学習の源流と最近の理論の展望」 甘利俊一 https://t.co/Kv47at5qxe via @YouTube

[stash @stash_196]

🔖

[mat @ballforest]

エンコーダの潜在空間の幾何学量推定 CNN・Transformerの情報幾何学を用いた分析 https://t.co/9ojmVkj3fk