Prithvi Dixit

Math + CS @ucberkeley

[Prithvi Dixit @pd172945]

Making it insanely easy to make things means that if you have a problem with any app (closed/open source), clone it! Ie, I don't like Todoist using trackers, so I had Claude few-shot a clone. Plus, these are personal and local tools, so you can make them as fast as binary

[Prithvi Dixit @pd172945]

Congratulations to the @ollama team for a great fundraise! Recently, I've seen a few interesting dynamic routers on top of ollama . Some thoughts on model routing here: https://t.co/TJoYdZ1iq3

[Prithvi Dixit @pd172945]

watching gpus go brrrr

[Prithvi Dixit @pd172945]

Hey! We put out a really cool paper outlining why sparse attention is the future: https://t.co/pESqHfvJaz

This was a great undertaking and if you read it, I’d be happy to chat about ideas and where you see sparsity emerging :)

[Prithvi Dixit @pd172945]

@cong_ml This is so cool! Gonna try it out for a few tasks 🫡

[Prithvi Dixit @pd172945]

@Venkydotdev Imagining an Anthropic employee sitting there saying “well, this kernel optimization will have to wait for 4 hours and 59 minutes for my limit to reset”. For sure they’re unlimited

[Prithvi Dixit @pd172945]

@mandylu Cholesky, otherwise LU

[Prithvi Dixit @pd172945]

When cost of video generation compresses towards zero I imagine every student will have cheap access to custom blackboard video lectures? I.e., if someone didn't understand how to do matrix row reduction, they could upload problem and have an AI-generated human explain visually.

[Prithvi Dixit @pd172945]

@AlexanderKalian Interesting - would ask if AI models being able to execute basically breadth first search over all of math literature automatically qualifies them above elite human mathematicians? If given choice between two humans with equal reasoning, wouldn’t we prefer the one with breadth?

[SkyLight @skylight_org]

We’ve been assuming that extreme attention sparsity must destroy model quality.

That assumption is now false.

New SkyLight Release: vAttention ⚡️

We added vAttention to the Tier1A leaderboard. The data is striking:

👑 #1 Practical Method: Surpasses the previous lead (PQCache), closing the gap to dense models within 1% at up to 10x sparsity..
👑 Saturates Benchmarks: Dominates the sparsity–quality frontier (w/ oracle top-k).
💡 Introducing Verified-X: A new paradigm for reliable inference-time sparsity.

In this week’s blog, we share an in-depth look at the algorithm and how to run it via SkyLight 👇

[SkyLight @skylight_org]

Selecting the top-k highest scores is the intuitive standard for sparse attention. However, our latest SkyLight Tier 1A benchmarks show a non-trivial gap between Oracle Top-k and dense baselines at high sparsity.

PQCache currently tops the leaderboard as the strongest practical method, closing the gap between approximate-top-k and oracle-top-k for upto 10x sparsity levels.

Closing the gap between dense and practical sparse attention methods will clearly require new ideas. In this week’s blog, PQCache authors share an in-depth look at the algorithm, and how to run it via SkyLight 👇

[SkyLight @skylight_org]

We are open-sourcing the full Tier-1A implementation and the first standardized leaderboards today.

We invite the community to benchmark your methods against the frontier, prove your improvements on the leaderboard, and help us close the gap between sparse and dense.

👉 Leaderboard: https://t.co/RIS1V58prQ
👉 Code: https://t.co/QizTAxhQlG

5/N 🧵

[SkyLight @skylight_org]

Sparse attention offers a viable path to solving the computational and memory issues of long-context LLM serving. However, we observe a surprising gap between the rapid pace of academic research and the reality of open-source adoption.

We identify the core barrier as fragmentation where new methods are published rapidly, but with missing comparative grounding.

Today we’re excited to share SkyLight, where we are building a unified framework designed to bridge this gap. SkyLight standardizes the implementation and evaluation of sparse attention to enable rigorous, controlled benchmarking.

Our first major effort is to understand the frontier of inference time sparse-decoding in LLMs.

1/N 🧵

[SkyLight @skylight_org]

To learn more about SkyLight check out

👉 Blog: https://t.co/M9brzO3JWd

This is a joint collaboration from Sky Computing lab with  @apd10desai ,  @kumarkagrawal, @pd172945, @luisgschroeder  @profjoeyg ,  @matei_zaharia , @istoica05  and @RethinkingAILab