Ben Geist

Research Eng @ramplabs / physics + math nerd / Kate Bush fan

[Ben Geist @b_geist]

> ask accounting firm how they’ll scale
> “we’re hiring”
> 300,000 CPAs left the profession
> accounting degrees at a 20 year low
> firms turning away clients they can’t staff
> teach AI the firm’s playbook
> turn it into executable SOPs
> books close in half the time

[Eric Glyman @eglyman]

Introducing Stack.

The AI operating system that lets accounting firms take on more clients without hiring. Learns your firm's process, runs the close, posts the journals. Fully auditable.

We’re living through the biggest shift in accounting since the spreadsheet.

[Eric Glyman @eglyman]

Introducing Stack.

The AI operating system that lets accounting firms take on more clients without hiring. Learns your firm's process, runs the close, posts the journals. Fully auditable.

We’re living through the biggest shift in accounting since the spreadsheet.

[Ben Geist @b_geist]

@a_levitator I like these things, that’s why

[Ben Geist @b_geist]

Imo techniques like this and sparse attn will massively reduce the compute bottleneck that limits smaller labs. Chinese labs are already heavily incentivized to create low compute techniques. This will lead to a Cambrian explosion of AI architectures

[Sakana AI @SakanaAILabs]

Introducing DiffusionBlocks: Block-wise Neural Network Training via Diffusion Interpretation

https://t.co/c9AvsRKybj

What if we didn’t have to hold an entire neural network in memory to train it?

Standard neural net training optimizes all parameters jointly. As a result, the memory required during training grows linearly with the depth of the network.

In our #ICLR2026 paper, we propose DiffusionBlocks, a principled framework to train networks one block at a time, drastically reducing memory requirements while matching end-to-end performance.

With DiffusionBlocks, we split the network into blocks and train them one at a time, so you only need memory for a single block.

How? We explicitly assign each block a role: to move the representation a little closer to the target than the block before it did. That role turns out to be precisely what a diffusion model does, step by step. Each block only needs to optimize its own objective and can be trained independently.

We validated this across five different architectures:

• ViT
• DiT
• Masked diffusion
• Autoregressive transformers
• Recurrent-depth transformers

In each case, performance is competitive with end-to-end training while using a fraction of the memory.

This perspective also extends naturally to recurrent-depth (Looped) transformers, which apply the same network iteratively and normally require expensive backpropagation through time (BPTT). Viewed through DiffusionBlocks, we can replace those multiple iterations with a single forward pass during training.

Read our paper and code, to learn more.

Paper: https://t.co/CRj96VGYQn
GitHub: https://t.co/eNW0K9Xh8E
🐟

[Ben Geist @b_geist]

@RampLabs We have mythos at home?

[Ben Geist @b_geist]

Modern security increasingly looks like probabilistic search over huge state spaces. Attackers can already do this with public frontier models, but defenders have a massive advantage with their internal context, telemetry, architecture knowledge, and production feedback loops.

Given the cost of a single breach, the economics of massively scaled defensive agents make a lot of sense.

[Ramp Labs @RampLabs]

We deployed 10,000 background agents to security-scan our codebase. The system is simple, scales with compute, and runs on publicly available models. From the scan, we fixed several high-severity vulnerabilities.

[Ramp Labs @RampLabs]

https://t.co/YHN5Hy4Ddf

[Ben Geist @b_geist]

@akshat_b @rene_sultan 👀

[Ben Geist @b_geist]

A scaffold for modern memory systems could very well just be optimized block sparse attn

[0x三井瘦 @0xmitsui]

看起来 @MiniMax_AI M3很快就要来了。工程负责人@SkylerMiao7 之前发的一个技术图中可以看到 MiniMax M3 模型确定将会有百万上下文，采用基于GQA(Grouped Query Attention)的动态块稀疏注意力设计。先用 Index Branch 做粗检索，再用 Sparse Branch 对选中的 block 做真实 attention，它的逻辑是：当前 query 不需要看全部历史，只需要看 top-k 相关历史块。打个比方就是看书时候不是把整本书每一页都重读，而是先快速查目录/索引，定位几个相关章节，再精读。这个设计的效果也很明显，一百万上下文，prefill比之前快9.7倍，decode快15.6倍。期待到时候看看DeepSeek V4 和 Minimax M3 谁才是性价比之王。

[Ben Geist @b_geist]

Spikes gradients are my worst fear 😖

[Ben Geist @b_geist]

@mma12261 Yupppp

[Ben Geist @b_geist]

@thesalomander @RampLabs Open your DMs, can help you!

[Ben Geist @b_geist]

@rene_sultan @GoogleDeepMind Goat!

[Ben Geist @b_geist]

@jonaswillett1 Cafe

[Ben Geist @b_geist]

@tryramp 🌱🌱🌱🌱

[Ben Geist @b_geist]

@tryramp 👀🔜🌱

[Ben Geist @b_geist]

Perks of working @tryramp: the free ramps

[Ben Geist @b_geist]

@henrytdowling @RampLabs @PrimeIntellect I wrote this article lol

[Ben Geist @b_geist]

@henrytdowling @RampLabs @PrimeIntellect Yes the majority of effort was building the tasks and reward so that the training was stable prime rl made the rest of the process simplified

[Ben Geist @b_geist]

@rakesh_nori And you dropped this 👑