🎯 Bespoke OLAP makes workload-specific database engines faster
[ADRS Blog #21] We feature Bespoke OLAP, a fully autonomous synthesis pipeline that outperforms general-purpose engines like DuckDB while being ~11.78× faster on the main benchmark comparison. Bespoke OLAP uses LLM-guided code generation to build a complete, custom C++ engine for OLAP workloads for just ~$120.
✍️ Read the blog: https://t.co/aOIkicnmEt
📖 ADRS Blog Series: https://t.co/UxujLFWX8b
📄 Bespoke OLAP Paper: https://t.co/tAWsMbLy8N
👩💻 Bespoke OLAP Code: https://t.co/6vL6VHbEW3
@isDineshHere We database people already worked on this. What came out is the bespoke olap paper, which is from my lab! Btw, we achieve even greater speedups with our approach! https://t.co/sIcNapXaOu
"Rare examples such as @TigerBeetle demonstrate that extreme specialization can pay off when the target workload is narrow yet widely shared"
Bespoke OLAP: DBMS on-demand, for your specific workload.
Also the fact @DuckDB has pretty much become a standard to benchmark against
This code now landed upstream and will be part of the 7.0 kernel when released. The liburing helpers, man page, and test cases have been merged into the master branch.
💿 io_uring is everywhere right now
But for DBMSs it isn’t magic dust:
• drop-in swap epoll/libaio → io_uring: ~+6–10%
• redesign for async + batching + io_uring features: ~2× end-to-end
• Postgres case study: +14% when “done right”
https://t.co/OEhIu9275o
#io_uring#linux
I'm excited about the database performance io_uring will unlock.
Last year I benchmarked Postgres 17 vs 18 to test the initial io_uring upgrades. I was surprised to see they weren't always a clear win for TPC-C.
This paper studies the potential, and the future looks good.
@_devott@TigerBeetleDB@melhindiCS@Tobias__Ziegler@viktor_leis@cbinnig Latency-wise we can go down as low as ~9.3 microseconds for UDP and ~10.3 microseconds for TCP. DPDK for reference is ~7 microseconds on our system. However, DPDK lacks all the nice TCP/IP features. (more Details in the paper)
Interested in io_uring? Check out our new paper, joint work with my former colleagues from TU Darmstadt (@cbinnig and @m4ttze) and from TU Munich (@melhindiCS and @viktor_leis). https://t.co/3QUrSuXbY3
Interested in io_uring? Check out our new paper, joint work with my former colleagues from TU Darmstadt (@cbinnig and @m4ttze) and from TU Munich (@melhindiCS and @viktor_leis). https://t.co/3QUrSuXbY3
HARMONY OF RESILIENCE: Recorded in space and sent to Earth via @SpaceX’s @Starlink constellation, Polaris Dawn crewmember and violinist @Gillis_SarahE invites you to enjoy this music moment in support of @StJude & @ElSistemaUSA → https://t.co/My8cUwAWzg