We published a paper! If you're into computational material science (chemistry of surfaces) u can grab a copy for free until Oct 29. Just use this link 😊🌃✨
https://t.co/IsOmwLUJyM
We've been getting a lot of questions lately about how the Internet Archive digitizes books.
The short answer: page by page, by hand.
You may remember our viral 2021 video of Eliza Zhang scanning a book. That's still how we do it.
Meet Eliza, and learn how we scan books: https://t.co/gt9s5lbPE4
When VS Code already has the Source Control feature, why do developers still use Git commands in the terminal for pushing, committing, or cloning repositories? 👀
Just found a fantastic substitute for cafes or libraries for digital nomads in Taipei. This little ‘library’ is hidden in the former Tobacco Factory as part of the Design Museum. You can get access to it for the whole day for just NTD50. Plenty of sockets and usually not crowded.
theres a jjk artist in palestine who posts updates on her life daily like journal to let everyone know how she's feeling and that she's alive. every single day she's exhausted and fears for her life but she still greets her mutuals and laughs. now im scared she's dead
How Discord Stores Trillions of Messages
In 2015, Discord started using MongoDB to store messages. By November 2015, there were already 100 million messages that did not fit in RAM, causing unpredictable latency.
In 2017, Discord moved to storing billions of messages in a Cassandra database. By 2022, the number of stored messages had ballooned to trillions across 177 nodes.
But Cassandra was plagued by several serious issues.
- Hot partitions occurred when a small number of high traffic channels overwhelmed nodes, cascading latency across the cluster.
- Garbage collection pauses created latency spikes.
- Compactions fell behind, forcing expensive reads to query multiple SSTables.
- Maintenance like node repairs interrupted service.
Discord migrated to ScyllaDB for the following benefits:
- Written in C++ instead of Java, eliminating disruptive garbage collection pauses
- Shard-per-core model provides stronger workload isolation to prevent hot partitions from cascading latency across nodes.
- Reverse query performance optimized to meet Discord's needs
- They reduced nodes to 72 while increasing disk space per node to 9TB.
To further protect ScyllaDB, Discord:
- Built intermediary data services in Rust that limit concurrent traffic spikes
- Data services sit between the API and database, coalescing requests
- Query the database just once even if multiple users request the same data
- Rust provided fast, safe concurrency ideal for this workload
The results? Tail latencies down from 40-125ms to a steady 15ms. Database uptime improved from weekend-long outages to smooth sailing. The system easily handled World Cup traffic spikes, processing events like goals and penalties without breaking a sweat. Discord continues to scale, now reliably storing trillions of messages with ScyllaDB.
–
Subscribe to our weekly newsletter to get a Free System Design PDF (158 pages): https://t.co/kNfv0DVDdf