Neil Adit

Research at Meta. PhD, Cornell. I like peanut butter and oats.

[Neil Adit @neiladit]

Finally got around to writing a blog post about wrapping up 2024 events: https://t.co/eQMilGuZke
I graduated, applied for jobs and attended multiple weddings - pretty eventful!

[Alexander Tian @Alex2262T]

Hardware yearns for block sparse attention, yet it seems largely absent from open weight LLMs. DeepSeek developed NSA, and people speculated DeepSeek v4 would integrate it, yet it was never utilized.

We have a hypothesis as to why.

We found that replacing dense attention with NSA significantly degraded its ability on synthetic retrieval tasks, even when finetuned on them. On our 32k context benchmark, it scored 0.300 compared to dense attention’s 0.904.

We found the reason, and how to fix most of it. Introducing COBS🌽(Cumulant Order Block Sparse Attention), with @AdiGhai18 @sanjitneelam @ZVasania @tensorpro:

• Raises NSA’s 0.300 → 0.820, closing ~86% of the gap to dense
• 15.15x less KV-cache read traffic than dense (just 1.21x the NSA baseline)
• Lower position-wise NLL than dense in our comparison

The key insight: block selection is the keystone to block sparse attention, and existing methods are mathematically stuck in storing a first-order approximation of a cumulant generating function. COBS caches a compressed second cumulant and escapes this ceiling.

Details in the paper.

[Neil Adit @neiladit]

@CapitalOne I made a booking via VentureX travel portal but cannot cancel the flight online. Since I'm traveling outside the country I cannot call the travel helpdesk number. Is there anyway to cancel the flight?

[Neil Adit @neiladit]

@rpopatna @passportsevamea @rpopatna It has already been ~11 months since I applied. Could you tell me what approvals have passed through and which ones are left?

[Neil Adit @neiladit]

@rpopatna @passportsevamea been waiting for the global entry process to proceed since November, 2024 after police verification. Could you update the progress on it?

[Neil Adit @neiladit]

@rpopatna @passportsevamea @rpopatna here is my application number PA1GE7029315024.

[Neil Adit @neiladit]

@rpopatna @passportsevamea Could you send me a message so I can DM you the application number?

[Neil Adit @neiladit]

I put the code (and android APK) on github here: https://t.co/wU1tLVjmRu

[Neil Adit @neiladit]

I spend a considerable time every morning doomscrolling reddit/twitter. So I ~vibe coded~ a digital wellbeing app using Cursor that not only blocks those apps but also website urls on google chrome :)

[Neil Adit @neiladit]

Here is how it looks like at the 2 hour mark!

[Neil Adit @neiladit]

@jacobmbuckman Gold is a finite resource though?

[Neil Adit @neiladit]

@adityakusupati @vishalmisra I feel like an idiot landing in JFK all this while :(

[Neil Adit @neiladit]

@deedydas It's the world we live in though

[Neil Adit @neiladit]

Got a chance to attend the first half of the Democratize Intelligence summit(https://t.co/Mr9arEE3zc) today. Interesting talks and great atmosphere! One of them was about sub-quadratic transformer architectures required to scale context length @jacobmbuckman

[Neil Adit @neiladit]

@notnagma Somebody actually made it: https://t.co/3yIq9Rqkid

[Neil Adit @neiladit]

Me thinking I've optimized my setup with TOTPs.
Next day with my phone in another room...

[Davis from Youform & OneUp @mynameis_davis]

Me: "I put my phone in the other room so I'm going to have a focused and productive working session this morning"

First site I open up:

[Neil Adit @neiladit]

@perplexity_ai @AravSrinivas
where do reminders get stored on the phone? Not synced to google calendar, yet?

[Neil Adit @neiladit]

This feels personal. Anybody got a good hack for keeping phones away? At this point, unlocking the phone on a notification is ingrained too deeply

[Brandon Luu, MD @BrandonLuuMD]

Having your phone on your desk or in your pocket reduces working memory and fluid intelligence — even if you're not using it.

Want to focus? Put your phone in another room.

[Neil Adit @neiladit]

Just asked #perplexity assistant for a 10 minute post run stretch and got the exact video I usually watch. Google assistant will give multiple videos and require manual selection. Small but valuable win

[Neil Adit @neiladit]

SOTA open-source models such as DeepSeek not only bring down the cost of competing LLM providers but also advances all adjacent areas which use transformers.

[Andrej Karpathy @karpathy]

So even though I'm technically in vision, papers, people and ideas across all of AI are suddenly extremely relevant. Everyone is working with essentially the same model, so most improvements and ideas can "copy paste" rapidly across all of AI.

[Andrej Karpathy @karpathy]

Projects like OpenAI’s Operator are to the digital world as Humanoid robots are to the physical world. One general setting (monitor keyboard and mouse, or human body) that can in principle gradually perform arbitrarily general tasks, via an I/O interface originally designed for humans. In both cases, it leads to a gradually mixed autonomy world, where humans become high-level supervisors of low-level automation. A bit like a driver monitoring the Autopilot. This will happen faster in digital world than in physical world because flipping bits is somewhere around 1000X less expensive than moving atoms. Though the market size and opportunity feels a lot bigger in physical world.

We actually worked on this idea in very early OpenAI (see Universe and World of Bits projects), but it was incorrectly sequenced - LLMs had to happen first. Even now I am not 100% sure if it is ready. Multimodal (images, video, audio) just barely got integrated with LLMs last 1-2 years, often bolted on as adapters. Worse, we haven’t really been to the territory of very very long task horizons. E.g. videos are a huge amount of information and I’m not sure that we can expect to just stuff it all into context windows (current paradigm) and then expect it to also work. I could imagine a breakthrough or two needed here, as an example.

People on my TL are saying 2025 is the year of agents. Personally I think 2025-2035 is the decade of agents. I feel a huge amount of work across the board to make it actually work. But it *should* work. Today, Operator can find you lunch on DoorDash or check a hotel etc, sometimes and maybe. Tomorrow, you’ll spin up organizations of Operators for long-running tasks of your choice (eg running a whole company). You could be a kind of CEO monitoring 10 of them at once, maybe dropping in to the trenches sometimes to unblock something. And things will get pretty interesting.