Johannes Rudolph

Fiddling bits since 2342
@[email protected]

[Johannes Rudolph @virtualvoid]

My job as a Rust Engineering Lead at a startup became victim of restructuring and I'm looking for a new role.
I enjoy working on infra software, Open Source, with Rust, Scala, or whatever is needed. I'm also available contractually to support previous libs like Pekko Http.

[Johannes Rudolph @virtualvoid]

@hadilq @debasishg If you need fast append/insert/delete, usually the solution is not "just an array" but clever pre-allocation schemes and a mixture between linear and linked data to amortize the cost of updates while still keeping most of the linear memory benefits for sequential access.

[Johannes Rudolph @virtualvoid]

@debasishg @hadilq Another downside is that the reference chain in a linked list adds a dependency chain between elements in the list that prevents potential OOO benefits and SIMD optimizations.

[Johannes Rudolph @virtualvoid]

@forked_franz I see. I looked into uprobes again for https://t.co/ANMuNQKhhD but JIT code / code in anon mappings is still not supported. It seems that you can use mem eXecution watchpoints to trace function entry but something like return probes still seems quite elusive.

[Johannes Rudolph @virtualvoid]

Almost 50 years ago mastermind Loriot created this visualization of what happens during a rolling upgrade if you implemented sharding naively (w/o consistent hashing) https://t.co/Q00ZHCAod4

[Johannes Rudolph @virtualvoid]

@Matsluni Thanks, Matthias!

[Johannes Rudolph @virtualvoid]

In other news, I won another prize at the Vesuvius Challenge for vesuvius-gui, an interactive browser for 3D volumes (like the vesuvius scrolls) written in Rust with egui. Thanks a lot to challenge team! https://t.co/xawhyMKHaY

[Vesuvius Challenge @scrollprize]

Congratulations to the May Progress Prize winners for contributions towards reading the Herculaneum scrolls!! https://t.co/fwMGfzECva

[Johannes Rudolph @virtualvoid]

@lukasz_bialy @WojciechM_dev Great stuff, maybe I should retry llama2.scala on Scala Native as well since it was also hampered e.g. by ByteBuffer issues.

[Johannes Rudolph @virtualvoid]

Thank you, @scrollprize, for awarding me a prize in the Vesuvius Challenge for my Open Source contributions cataloguing released segments and running OS ink detection models on them. It has been a fun journey and great collaboration! https://t.co/orl3NiYAoz

[Jordi Mon Companys @JordiMonPMM]

Amazing contributions to the @scrollprize @spacegaier, @virtualvoid and others and congrats for the prizes won! https://t.co/UKgIgO9ECo

[Johannes Rudolph @virtualvoid]

@ernerfeldt @scrollprize Thanks for egui :) For the first time in 15 years, I have been enjoying working on a gui after avoiding GUIs at all cost because of the lack of good options...

[Johannes Rudolph @virtualvoid]

A short report on trying out different algorithms for selecting top-p elements from a distribution (aka Nucleus Sampling): https://t.co/RdoAVREGnl

[Johannes Rudolph @virtualvoid]

@hetzner The whole point of cloud volumes is to be able to view storage detached from compute. Optimally, storage can be easily migrated to new compute in cases of problems. Quite a bummer if storage cannot be relocated in cases like this... @Hetzner_Online

[Johannes Rudolph @virtualvoid]

Mmh, @hetzner cloud volume hangs while unmounting since one hour. No more actions on the server are possible, after shutting the node down other mounted volumes are also blocked, server cannot be restarted. Big share of cluster storage is unavailable => extended downtime.

[Johannes Rudolph @virtualvoid]

Also, the day, when you first run a code generation model on your engine asking for suggestions about how to improve algorithms of the underlying engine 🤯 (the suggestions are mostly weird and broken, admittedly, but had some ideas to follow up) https://t.co/9vYYIS6Wa4

[Johannes Rudolph @virtualvoid]

llama2.scala now has quantization support, AVX2 kernels, multithreading and can load some GGML models directly. https://t.co/C6DsHB5GLI

[Johannes Rudolph @virtualvoid]

@RealNeilC @karpathy's llama2.c is interesting because it removes all abstraction and breaks down inference into a single file of C (which can be seen as a lingua franca of computation because data structures map closely to actual memory layout and operations to CPU instructions)

[Johannes Rudolph @virtualvoid]

@RealNeilC Python's speed is mostly irrelevant since ML means instructing the GPU which matrix op to do next. In big models, each multiplication takes so long that it dwarfs the overhead of the language. If you are looking at state-of-the-art inference on CPU look at llama.cpp.

[Apache Pekko @ApachePekko]

Apache Pekko Http 1.0.0 has been released, see https://t.co/B2ZjiTTeyM and https://t.co/sCpNVLWwdg for more details.

Apache Pekko Http also includes Scala 3.3 support, the result of an ongoing community effort that spanned years!