Building @jazz_tools v2 in public, Ep 3
Main learning: ground your LLMs in your concrete problem, build jigs, religiously optimize code iteration speed.
The last couple days were more perf work, driven by a specific target that a commercial use case gave us.
Optimization un/fortunately has always been extremely addictive to me: a giant puzzle with a clear reward function.
LLMs are good at it for the same reason. They especially help with the tedious parts: run the benchmark, interpret profiles, correlate with code impl, small thesis, small experiment, rebuild, remeasure, repeat.
This already worked well for our synthetic benchmarks in our monorepo, but here we were particularly interested in perf in a concrete adopter app, in their own repo, under NDA, etc.
And what mattered here was end-to-end performance including the jazz server and their use of jazz in their app’s frontend. For a while I was manually getting the app in the right state, profiling, then feeding the profiles to codex. I was the slowest element in the process. Annoyingly, browser automation was similarly slow.
So I set up a private meta-repo that had links to a jazz worktree and a worktree of the adopter app. Made codex set up a small jig that directly imported their app code and ran only the perf-relevant part in bun, bypassing all the app setup around it.
This shortened the iteration cycle to around 10s (plus Rust build times) and meant codex could reliably iterate on the problem autonomously.
In one afternoon of using this setup we found more optimization opportunities than I had in the entire week before, speeding up one particular load pattern by about 24x
I’m now using this same jig to optimize cache re-use of subqueries across similar queries, especially for row-level-security policy evals, allowing me to try different designs super quickly. And again, both performance and correctness are grounded in adopter apps that actually are more demanding and intricate than any synthetic benchmarks we could have come up with this early.
The longer-term goal is to extend this to as many adopter apps as possible and to automate measuring the effect of changes to jazz on real apps even more.
Our North Star here is the equivalent of the “crater run” of the Rust compiler that is run over a large amount of (all?) public rust crates, being a comprehensive check of compile success and performance over the broad ecosystem.
As promised, yapping more on @jazz_tools
After launching the v2 alpha at @ReactMiamiConf our focus has been polish and perf, and there is a lot to do.
Often, refactoring and nailing down semantics go hand in hand with performance.
We realized this again over the last couple days, working on clarifying the difference between “batches” and “transactions”.
Jazz v2 is unique in that it offers both globally consistent MVCC-style ACID transactions as well as local-first, eventually consistent writes (this is actually the default).
But then we realized that even for the eventually consistent writes, it sometimes makes sense to group them into units, which we called “batches”.
Initially, we thought of this as a pure perf optimization primitive and were quite loose with their semantics compared to transactions, where strictness really mattered.
For example, in transactions you clearly want all-or-nothing and rollbacks, but we didn’t really think this was important for batches.
But then, trying to actually implement the perf optimizations, we realized that per-row bookkeeping in large batches made them basically pointless.
So perf guided us towards a clearer meaning for batches: they should also be all-or-nothing, now only being separated from transactions by one clear distinction: multiple concurrent batches can all be accepted and participate in merge-previews, while only one of multiple transactions gets accepted, resulting in global consistency and linearizability.
With all the rows per batch sharing one fate, the bookkeeping overhead disappears and the decision for which tool to use in each case becomes crisp:
- eventual consistency ok but need operations to have truly independent fates? Use separate batches.
- eventual consistency ok and operations sharing fate ok or even desired as a logical unit? Use one batch.
- need global consistency? Use a transaction.
It’s fun to see how this loosens up complexity throughout storage format, query subscription updates, sync protocol etc.
"You still build features from the button to the database, but now they are much closer."
I wrote how building @tillydotsocial with @jazz_tools made me realize there is a "New Kind of Web App".
https://t.co/bTq1MhUGvb
So proud that @jazz_tools is a launch backend that you can use in @vercel’s new Workflow DevKit.
Durable workflows are arguably the compute abstraction of the decade - the best DX for persistence/queues is one that fully disappears.
Jazz is uniquely suited to power workflows…
In SF for @nextjs Conf or Ship AI Conf?
Interested in building apps that sync across devices, work offline, have real-time collab or just need to be super fast?
Come hang out on Friday and learn about Jazz!
Snacks, drinks and mini-electronics on us!
https://t.co/EpTGmbvfWY
New episode of the wonderful https://t.co/zb61fmetHG podcast - this time: the most in-depth story recorded so far about @jazz_tools, how it works, how it’s different, how it came to be. Plus some thoughts about local-first and the web in general! Great conversation and great questions by @schickling!
For those of you based in San Francisco / in town next Wednesday eve:
Local-first curious?
Trying to build a multiplayer app?
Need cross-device sync or offline support?
Drop by and see @jazz_tools in action!
Join us for some hacking, snacks & drinks!
(Event registration below)
Mercoledì 18 Maggio a RomaJS @faber_vt ci parlerà di "Data fetching in React 18 apps" approfondendo il funzionamento delle principali librerie di data-fetching e delle novità su Suspense
Siete pronti al prossimo “unlearn everything”?
Non mancate --> https://t.co/Uc4oBqwVKu
Di nuovo full-remote, torna dopo le vacanze estive il consueto appuntamento con il RomaJS.
Questo mese ci terranno compagnia Alessandro Miliucci e Matteo Manchi. Non mancate mercoledì 16, dalle ore 18:30
Fb: https://t.co/L6V68XoKmn
YT: https://t.co/pA7J9NBBnC
RomaJS is back! 🚀
La quarantena non fermerà la nostra voglia di Javascript
Vi aspettiamo online per il meetup di maggio insieme ai due speaker di questo mese: Marco Liberati e Manuel Salvatore Martone.
Non potete mancare!
https://t.co/ZjFsEAjMT9
@jarrichvdv@vivainio@_developit I think he is trying to say that if you have to write a 200b library by using directly JavaScript is easier to do some code golf.
Today’s episode from “Safari on desktop is different from Safari on iOS”:
Storing a blob in IndexedDB works fine on desktop, but will throw on mobile. ArrayBuffers are fine, tho.