Luis Gaspar Schroeder

ML Systems Researcher @UCBerkeley (Sky Lab). Computer Science at @tu_muenchen and @UCBerkeley.

[Poke @interaction]

Starting today, personal superintelligence is just one tap away.

No download, no signup.

Text Poke for free now:

https://t.co/x9ipMywiNU 🌴

—
0:00 – What's Poke?
0:50 – Introducing Poke Recipes
1:25 –  Create a Recipe in 10 seconds
1:43 – Earn on Poke
2:44 – Build with npx poke
12:58 – Recap
13:36 – Parisian Love

[SkyLight @skylight_org]

Selecting the top-k highest scores is the intuitive standard for sparse attention. However, our latest SkyLight Tier 1A benchmarks show a non-trivial gap between Oracle Top-k and dense baselines at high sparsity.

PQCache currently tops the leaderboard as the strongest practical method, closing the gap between approximate-top-k and oracle-top-k for upto 10x sparsity levels.

Closing the gap between dense and practical sparse attention methods will clearly require new ideas. In this week’s blog, PQCache authors share an in-depth look at the algorithm, and how to run it via SkyLight 👇

[Alejandro Cuadron @Alex_Cuadron]

Wait what!? We robustified tau2-bench and found that the newly released model from @OpenAI (GPT-5.1) performs way worse than GPT-5 and GPT-5-mini.

All while being 5x more expensive than GPT-5-mini!

But, why? We have a theory...

[SkyLight @skylight_org]

Sparse attention offers a viable path to solving the computational and memory issues of long-context LLM serving. However, we observe a surprising gap between the rapid pace of academic research and the reality of open-source adoption.

We identify the core barrier as fragmentation where new methods are published rapidly, but with missing comparative grounding.

Today we’re excited to share SkyLight, where we are building a unified framework designed to bridge this gap. SkyLight standardizes the implementation and evaluation of sparse attention to enable rigorous, controlled benchmarking.

Our first major effort is to understand the frontier of inference time sparse-decoding in LLMs.

1/N 🧵

[Poke @interaction]

Say hi to https://t.co/VIWYU64dUI! 👋🏼🌴

[Ameer Haj-Ali @aha_ml]

🚨 New Podcast 🚨
I spoke with the CEO of Precisely about how Agentic AI & data integrity come together to solve mission-critical problems.
A must-watch for anyone serious about trusted AI at scale.
🎥 Highlight video: https://t.co/SXuso6MSK6
#AI #EnterpriseAI #DataIntegrity

[Luis Gaspar Schroeder @luisgschroeder]

Big thanks to the team that‘s actively supporting vCache:
@profjoeyg, @matei_zaharia Alfons Kemper, Stephan Krusche, Mark Zhao, @LynnLiu41887950, Kyle Chu, @Alex_Cuadron, and @Apd10Desai

[Luis Gaspar Schroeder @luisgschroeder]

What if we can guarantee the correctness of LLM responses? Turns out—we can!

We built vCache, the first semantic cache with error guarantees. For the first time, you can use semantic caching while knowing exactly how often the system is allowed to make a mistake.

[Luis Gaspar Schroeder @luisgschroeder]

🧵The vCache algorithm is workload and model-agnostic and does not require manual fine-tuning. vCache consistently meets the specified error bounds while outperforming state-of-the-art static-threshold and fine-tuned embedding baselines.

[Luis Gaspar Schroeder @luisgschroeder]

🧵We propose vCache, the first verified semantic cache with user-defined error rate guarantees. It employs a probabilistic online learning algorithm to estimate an optimal threshold for each cached prompt, enabling reliable cache responses without additional offline training.

[Luis Gaspar Schroeder @luisgschroeder]

🧵Existing systems use the same static similarity threshold across all requests to determine whether two prompts share similar responses. However, static thresholds do not give correctness guarantees, result in unexpected error rates, and lead to suboptimal cache hit rates.

[Luis Gaspar Schroeder @luisgschroeder]

🧵Semantic caching reduces LLM inference latency and cost by returning cached model responses for semantically similar prompts (not just exact matches)—so you don’t pay for inference cost and latency on repeated prompts that have the same answer.

[Luis Gaspar Schroeder @luisgschroeder]

Explore more:

🔗 GitHub: https://t.co/xzVBiki5xr

📄 Paper: https://t.co/eySryK8vi6

[Andy Konwinski @andykonwinski]

Today, I’m launching a deeply personal project. I’m betting $100M that we can help computer scientists create more upside impact for humanity.
Built for and by researchers, including @JeffDean & @jpineau1 on the board, @LaudeInstitute catalyzes research with real-world impact.

[Lakshya A Agrawal @LakshyAAAgrawal]

Real world AI pipelines are often compound, multi-module, and multi-step programs—unlike most RL/GRPO implementations today which optimize a single agent.

🚨 Super excited to release dspy.GRPO, which lets you GRPO tune any arbitrary multi-module, multi-step DSPy program, with the same, easy interface.

Looking forward to what the community builds with it!

Amazing work with @NoahZiems @dilarafsoylu and @lateinteraction, with more to come soon!

Check the easy to follow guide at: https://t.co/HO41DBeyRp

[Melissa Pan @melissapan]

🚨 Why Do Multi-Agent LLM Systems Fail? ⁉️
🔥 Introducing MAST: The first multi-agent failure taxonomy - consists of 14 failure modes and 3 categories, generalizes for diverse multi-agent systems and tasks!

Paper: https://t.co/BC5YHS8ZRZ
Code: https://t.co/Ea1FvGcaLs

🧵1/n

[Melissa Pan @melissapan]

Running “Creative Coding” workshops for grade5-8 girls at EYH today - co-design the course with @Ponnapalli95

[Jared Quincy Davis @jaredq_]

Ember: an inference-time scaling architecture framework

🧵

(1/8)

[Project Aria @Meta @meta_aria]

Prof. Danfei Xu (@danfei_xu) and the Robot Learning and Reasoning Lab (RL2) present EgoMimic.

EgoMimic is a full-stack framework that scales robot manipulation through egocentric-view human demonstrations via Project Aria glasses.

🔖Blog post: https://t.co/ZCb0KcYQyZ
🔗Github: https://t.co/DZk33WfAjg
📽️YouTube: https://t.co/JgVCaR46w3

[Alejandro Cuadron @Alex_Cuadron]

New discovery! LLMs are just like humans!
Overthinking GREATLY HURTS their performance

If we select the solution with the lower overthinking score. We improve model performance by almost 30% while reducing costs by 43% (o1_low)

Is reasoning really the future of LLMs? 🧵