Komora

shiny database fragments

[Komora @komora_io]

terrors is an ergonomic and precise error handling library built atop type-level set arithmetic. It lets users specify ad-hoc sum types that can be safely narrowed or broadened, encouraging functions to be precise about the kinds of errors that they return
https://t.co/eD4Od03Ljx

[Komora @komora_io]

When the optimized reference count reaches its maximum size (8 bit or 16 bit) the backing immutable bytes are copied into a new InlineArray. The assumption is that most Arcs don't have many references, and immutable bytes can be copied at-will.

[Komora @komora_io]

inline-array is a DB-focused stack-inlinable byte array with 3 variants:
* inline up to 7 bytes +6 bit length
* up to 255 bytes + 1-byte RC and 1-byte len
* larger + 2-byte RC and 48-bit len

All variants guarantee 8-byte alignment of bytes.

https://t.co/6CyKz1W5GW

[Komora @komora_io]

(260 million accesses per second for 8 billion items in under 33 seconds - smaller datasets are even faster)

[Komora @komora_io]

the extremely efficient wait-free 4-level pagetable crate version 0.4.2 has been released, including some significant space and latency improvements when using smaller ranges of keys.

260 million accesses per second on a MBP, with 0.003% memory overhead.

https://t.co/epyPkhiX9G

[Komora @komora_io]

concurrent-map 5.0.24 adds support for atomic pop_first_in_range, pop_last_in_range, fetch_max, and fetch_min functions.
https://t.co/0So8WMLuq0

[Komora @komora_io]

concurrent-map 5.0.23 now supports two fan-favorite methods from sled: fetch_and_update and update_and_fetch for atomically pushing code to data, letting users avoid a manual read + CAS loop until successful. This makes any pure value mutation atomic.

https://t.co/ti55vGdIc3

[Komora @komora_io]

concurrent-map 5.0.20 improves single-threaded insertion performance by around 400% - something that is only possible due to the intentional avoidance of Sync in concurrent Rust.

It is now MIT/Apache-2.0 in preparation for inclusion in sled.

https://t.co/ORfWpfzXSa

[Komora @komora_io]

concurrent-map 1.0.14 now supports reverse iteration as well as a variety of useful ordering-based methods:

first, last, pop_first, pop_last, get_lt, get_lte, get_gt, get_gte

This was implemented as part of integrating the komora projects into sled.

https://t.co/gcPYHJCU9x

[Komora @komora_io]

concurrent-map is a lock-free B+ tree supporting high-throughput lexicographic scan operations in the presence of concurrent mutations. It supports custom fan-out and control over memory reclamation granularity to tune for a variety of access patterns.
https://t.co/gcPYHJCU9x

[Komora @komora_io]

Marble 14 has been released. It includes several large latency and throughput improvements. Zstd dictionaries can now be generated for each write batch, supporting tunable compression ratios for batches of small objects without harming read latencies.

https://t.co/y7RHepylHD

[Komora @komora_io]

cache-advisor is a scan-resistant sharded low-memory non-blocking concurrent LRU. It does not store items directly, but tells you when to evict items that you maintain.
https://t.co/YAfhIyCsSQ

[Komora @komora_io]

sharded-log is the first crate to showcase how the fault-injection testing crate released earlier today can be used to verify correct behavior when errors occur, even in concurrent and messy situations.

https://t.co/iW300xE9Ha

[Komora @komora_io]

We know why outages happen in data-intensive systems. Most bugs live in error handling code. Most error handling code is executed for the first time in production. The fault-injection crate allows you to test error handling code before it causes an outage.
https://t.co/i8ub1VzxvA

[Komora @komora_io]

The new sharded-log crate is ideal for logging concurrent atomic batches with low contention before periodically compacting corresponding items into a backing store. It pairs extremely well with Marble, the newly-released garbage-collecting object store.
https://t.co/zLT99ajHMe

[Komora @komora_io]

We know why outages happen in data-intensive systems. Most bugs live in error handling code. Most error handling code is executed for the first time in production. The fault-injection crate allows you to test error handling code before it causes an outage.
https://t.co/i8ub1VzxvA

[Komora @komora_io]

Marble is an object store that maps u64 keys to arbitrary blobs. It performs generational garbage collection and supports simple user-specified object partitioning for grouping similar objects together on-disk. It is an ideal base for embedded KVs & DBs.
https://t.co/y7RHepylHD

[Komora @komora_io]

Pagetables form the heart of many pieces of foundational infrastructure. This one provides wait-free access to AtomicU64's based on a u64 key. It is 4-levels deep and has a fan-out of 2^16. It is ideal for managing dense metadata in concurrent systems.

https://t.co/fM9as65S2a

[Komora @komora_io]

A mysterious data structure has appeared...

https://t.co/AFoLW4LGbo