The idea is to optimize the compiler IR with many small, local graph-rewrite rules (“peepholes”) instead of relying only on big global passes. You inspect a tiny region of the Sea-of-Nodes graph, replace it with a simpler equivalent form, and then reprocess neighboring nodes
Great read! https://t.co/hQHk4kgStX
TLDR; auto-research (@karpathy) applied to databases!
Detail - LLMs to discover better database algorithms (buffer management, index selection, query rewriting) by co-evolving not just the solutions, but the evaluation pipelines themselves.
Retries in distributed systems assume transient failure -- the same request will eventually succeed. Retries with LLMs just re-roll the dice. Same input, different latent trajectory.
« Don’t ever make the mistake [of thinking] that you can design something better than what you get from ruthless massively parallel trial-and-error with a feedback cycle. That’s giving your intelligence much too much credit. » (Linus Torvalds)
« In my field of research (machine learning, and especially deep learning & neural nets), [theory lags practice] is a truth I have experienced first-hand. » (Yoshua Bengio)
Watt invented the engine long before scientists conceived thermodynamics. We built electric circuits before scientists founded electromagnetism. We hacked computers together and then founded computer science. We created large language models and in the future, we will understand why they work.
Daniel Lemire, "Theory lags practice," in Daniel Lemire's blog, January 7, 2015, https://t.co/CcI9wtCRWe
The approach is such a clean way to bridge the gap between empirical observation and formal theory. In distributed systems, we often struggle with “emergent” behaviors we can’t fully prove.. this level of precision in tracking true posterior is huge step for interpretability
New work: Do transformers actually do Bayesian inference?
We built “Bayesian wind tunnels” where the true posterior is known exactly.
Result: transformers track Bayes with 10⁻³-bit precision. And we now know why.
I: https://t.co/e5O5O3QN5U
II: https://t.co/11KMePLUaT 🧵
New work: Do transformers actually do Bayesian inference?
We built “Bayesian wind tunnels” where the true posterior is known exactly.
Result: transformers track Bayes with 10⁻³-bit precision. And we now know why.
I: https://t.co/e5O5O3QN5U
II: https://t.co/11KMePLUaT 🧵
@TheValueist The shift toward disaggregated prefill and decode is essentially turning LLM inference into a classic distributed systems problem: managing state transfer overhead vs. compute efficienc! Spot on!
If databases and their internals fascinate you -- this is quite a treasure trove of lectures from 2014 to now!
https://t.co/yDeFcoMaWA
Especially the Time-Series one from 2017!
Thrilled by the insightful @dist_sys meetup at @nutanix Pune office, July 2023! 🚀 A shout-out to all participants for making it remarkable.
Exciting lineup in store for upcoming meetups.
Stay tuned for your next meetup - https://t.co/iXb7PFBQPZ
@dist_sys We’re setting up a Distributed System Testing group in London and will be having our first event on 21st June. If you would like to meet like minded people, check out the details and sign up! https://t.co/Xc7boE5fAS
Thoroughly enjoyed attending this demo of TLA+ by Markus who works on it at MSR. I liked how he incrementally built up the problem and its pitfalls, how TLA+ scores over manual reasoning and tests, implications of state space explosion, invariants, liveness, starvation ... etc