paper 3, more LOB trading: RL takes on handcrafted signal execution
i've repeatedly seen the authors names on good papers: Nagy et. al
https://t.co/R1wtMMXMgE
the results (vs baseline) were the best i've seen so far
note: both strategies get the same synthetic signal - this tests signal execution
the details🪡:
1/24
Today we are happy to share one of our first open research releases: Latent Sparse Attention (LSA), a step towards subquadratic attention that matches accuracy of full attention at ultra-long context.
Sparse attention attends to only a few tokens per query, but it has to find the right ones. At 10M tokens, that search takes 98% of the attention layer's time. LSA makes sparse attention up to 22.7× faster and full attention 72.4x faster.
With @kubicek_ales@caciolai@PengLiangzu@SukritSumant@Ben_Hoov@FraserGreenlee@RMBattleday@radim_urban@thoefler@jcrwhittington
We asked GPT-6 Astra to edit Qwen3.5-0.8B weights to improve math.
It directly changed only 0.016% of weights, fixing 0.8B's reasoning loops & improving math up to 6x and even instruction following 3x, without gradients or training.
We call this Weight-Space Programming: frontier LLMs can now natively read/write neural network weights! 🧵
Blog: https://t.co/bOWCcFhCVq
running this lil study. was supposed to release the details as an Article (i literally wrote & finished it)
but a minor tweak & the edge still might be in tact - will release it for readers to up-cycle if it's faded
Two weeks ago a life changing amount of money was wired into my bank account from the sale of my business.
Since I'm no longer a competitor in this particular arena, I'm gonna drop some sauce on how I completely mogged an entire industry on meta ads.
I'm going to be vague so I don't dox myself, but here it goes:
@emollick thing is every closed-model run sends the items through that lab's own api, whereas open weights mostly go via 3rd party hosts.
even if no one trains on api logs that's an asymmetry, depending on how they rotate their items (per run? only per version?) no?
@yacineMTB possible explanation could just be the nature of their verifiers:
>lean kernel says yes or no -> you can't game this
>code rl mostly rewards "tests pass" -> models learn to special-case. so basically a strong optimiser + leaky reward
the stats are a huuge flex.
what are your predictions on the extensibility of this method beyond simple robotic manipulation, to tasks that cannot (due to physical or other constraints) be demonstrated by a human?
e.g. autonomous drones in construction, search & rescue in high-risk scenarios, etc.
looking forward to learning more