@muratdemirbas and I did a fun exercise and read a HotOS paper ("Real Life Is Uncertain. Consensus Should Be Too!") together. We recorded the entire messy process. It is a long video of us reading, questioning, guessing, and arguing with the paper. https://t.co/3dJDB8yFsi
I have played around this a bit. I tried several ideas but what really worked the best for me was a custom WASM runtime. Most of the times the thing that became a roadblock for me in other approaches was making the time a function of number of instructions executed - this was easily solvable in a WASM runtime. What I ended up allowed fine grained control over:
1. File system ops
2. Network ops
3. Scheduling (both threads as well as processes)
4. Cluster and process local time
And a few other things. The only drawback of the approach was that the program must be compiled to WASM (easy for Rust, Zig, etc though).
If someday I will get some time, I will also try to play around with Intel Pin to see if it can be leveraged to do something similar without forcing WASM.
Deep work crushes shallow signals.
Amidst this deepseek frenzy, ask yourself - when did you last think deeply or do something intellectually challenging?
If not, you’re just over-optimizing your career while others innovate.
Take bets. Take on tougher problems. Challenge yourself with harder subjects.
When was the last time you learnt something just for fun and not because it will give you a "better job"?
Until this happens, someone else will keep creating deep-tech, while we crib!
note - this is for those who are now making decent income and out or scarcity.
First post of the year! @andy_pavlo got me thinking about why Confluent didn't build WarpStream.
My conclusion: legacy infrastructure companies are going to have a tough time against cloud native, AI-enabled, post-ZIRP competitors.
https://t.co/k2AsMDne0m
Hoping to be more active here (and other platforms) this year.
I spent most of 2024 working on my relationships (2023 was a roller coaster in that regard) and focusing exclusively on job (finally the product that I was working on got GA’ed in December 🥳).
This week on the unikernel application spotlight we boot up @TigerBeetleDB a new fast OLTP db! Written in @ziglang it makes use of io_uring via @axboe , O_DIRECT, a deterministic simulator and way more than I can put into this tweet - check it out. https://t.co/A1A0YgGEOJ
New post is up! Next-gen infrastructure must support flexible deployments. Embedded, single-node, clustered, BYOC, SaaS, and self-managed. We're finally able to do this with one codebase.
https://t.co/cBnCHs3VxE
@arpit_bhayani Ok, took a quick look at the repo. I could find AOF but not RDB (maybe its called something else there).
I am sorry but AOF looks overly simplistic. I would have loved to contribute however Business Source License wouldn't allow me (Job + Personal POV) :)
@arpit_bhayani That.. looks a little strange. Maybe I am reading it wrong, is the throughput 4x with dicedb and that too with such small ops?
Does dice have both AOF and RDB for persistence like redis?
week 34, streaming kernel dev
topic: debugging wild low level boot issues
(pre-recorded since I'm on vacation, recap vid in thread)
https://t.co/gBgxY6uOZ1
Seokjoo Cho (@seokjoo_cho) presents his work on building ScaleCache, a scalable page cache for multiple SSDs at @IllinoisCDS Systems Research Seminar.
The work appears at EuroSys'24,
https://t.co/32AZiAWQw4
@seokjoo_cho is a 1st-year PhD student advised by @AishwaryaGanlat and Ram Alagappan.
matklad will present TigerBeetle’s LSM-Forest compaction at @P99CONF
- storage determinism across replicas for faster recovery from disk faults
- perfect pacing to solve write stalls for predictable P100s
- and with static memory allocation
https://t.co/Rv7FQTV52e
TigerBeetle’s LSM compaction is “the beating heart” of the database.
Bringing together ideas at the intersection of global consensus protocol and local storage engine.
We just released https://t.co/kciprrC7xX 0.2.0! 🎉 This release has:
- in-memory block cache
- on-disk object cache
- garbage collection
- compressed bloom filters
Next up: admin CLI, range queries, and more cache improevements.
Came across this book today. While the CSAPP book has a couple of chapters on these topics, this book is completely devoted to just memory hierarchy and tries to provide a holistic overview of the system, including cache memory, RAM and spinning disks.
It's a bit dated now and does not discuss SSD or NVME storage. The ToC is very interesting and includes case studies from real-world systems.
Now, only if there was enough time to read it.