Parallelism is one reason we’re moving Lodestar’s compute-heavy work into Zig.
JavaScript workers run in separate V8 isolates. Ordinary objects sent between them are structured-cloned, which can mean copying large inputs and results. Node can avoid some copies with transferable buffers or SharedArrayBuffer, but that requires explicit ownership and synchronization.
Inside lodestar-z, native worker threads share one address space. Once data is in native memory, workers can operate on the same buffers via pointers rather than serializing inputs for each worker.
That gives us tighter control over how work and memory are shared—and less coordination overhead on parallel hot paths.
👊I've joined the crew at @ClickHouseDB to establish the *ClickHouse Labs* research group. We'll be working side-by-side with ClickHouse devs to cook up new ways to make the system even faster. I'll also be exploring new topics related to PostgreSQL. https://t.co/7VFLQzAL8g
Ello timeline,
Currently looking for work. If you need someone to sculpt computers into moving data around efficiently (databases, vms, backends, compute kernels), I'd love to chat.
Hazard pointer is an interesting data structure that implements a memory management methodology that allows memory reclamation for arbitrary reuse.
This talk from @KernelRecipes 2024 presents a new synchronization API that combines the 2 data structures hazard pointers and reference counters to leverage the benefits of each mechanism. It uses hazard pointers as fast-paths, and falls back to reference counters either explicitly when the reader expects to hold the object for a long time, or when no hazard pointer slots are available. Hazard pointers, being used on the fast paths improve upon the speed and scalability.
Link to the talk -
Hazard pointers with reference counter -
https://t.co/FutiGR0zup
Original paper on Hazard pointers -
Hazard Pointers: Safe Memory Reclamation for Lock-Free Objects by Maged M. Michael -
https://t.co/QFua2pabgb
Lodestar processes SSZ data throughout every slot. That’s blocks, states, attestations, and more. How do we achieve less runtime machinery and lower memory pressure on this hot path?
In Lodestar-z, Zig’s comptime generates specialized code directly from each SSZ schema. Field types, offsets, and serialization paths are resolved ahead of time.
At runtime, the serializer writes directly into a caller-provided buffer. It still processes each field, but it doesn’t interpret a dynamic type descriptor or allocate per-field intermediate objects.
Do the heavy lifting at compile time so the hot path doesn't have to.
A great adventure comes to an end. A crying shame, but I'm grateful to have spent time writing zig, and that it was with such heavyweights as @Gajpower, @GrapeBaBa, @ch4r10t33r , @anshalshukla and @BRUHDWAJ. Rest in peace, champion 🫡
friends, we do have news but a sad one.
zeam grant support from EF has been discontinued.
while we believed that we were doing tremendous work with an excellent team, carrying the PQ implementation effort on our shoulders, unfortunately we have been caught up in this downturn that EF and ethereum core development ecosystem is going through.
without support, we are unable to passionately champion our cause and work, so while we take a step back, we will see how we can continue to involve ourselves in furthering ethereum ecosystem and its goals.
and we thank Ethereum foundation for the support so far. If you would like to support us in anyway, do reach us.
thank you for your attention to the matter 🙂
Since the @bunjavascript folks recently wrote about their experience moving from Zig to Rust, it seemed like a good time to write about our experience rewriting @roc_lang from Rust to Zig!
https://t.co/JkPboqnGTn