Duy Do

Dad | Husband | Software Engineer

[Andrej Karpathy @karpathy]

New supply chain attack this time for npm axios, the most popular HTTP client library with 300M weekly downloads.

Scanning my system I found a use imported from googleworkspace/cli from a few days ago when I was experimenting with gmail/gcal cli. The installed version (luckily) resolved to an unaffected 1.13.5, but the project dependency is not pinned, meaning that if I did this earlier today the code would have resolved to latest and I'd be pwned.

It's possible to personally defend against these to some extent with local settings e.g. release-age constraints, or containers or etc, but I think ultimately the defaults of package management projects (pip, npm etc) have to change so that a single infection (usually luckily fairly temporary in nature due to security scanning) does not spread through users at random and at scale via unpinned dependencies.

More comprehensive article:
https://t.co/EJAZbqAPIQ

[Duy Do @duydo]

Vietnamese Analysis Plugin for #Elasticsearch 9.3.0 has been released https://t.co/lyHqXYKEvd

[Duy Do @duydo]

Celery workers freeze, RUNNING but idle. Traced to epoll flaw on RabbitMQ disconnects. Fix: HAProxy for clean RSTs. Issue #3773 still remains https://t.co/mGU3RXOzIZ #Celery #RabbitMQ #DevOps

[Jeff Dean @JeffDean]

Performance Hints

Over the years, my colleague Sanjay Ghemawat and I have done a fair bit of diving into performance tuning of various pieces of code. We wrote an internal Performance Hints document a couple of years ago as a way of identifying some general principles and we've recently published a version of it externally.

We'd love any feedback you might have!

Read the full doc at: https://t.co/jej95g236P

[Shay Banon @kimchy]

Elasticsearch (and Kibana) are Open Source, Again! Soooo excited. Read more in the blog I wrote https://t.co/ANPAoYNfQy

[Elastic @elastic]

Elasticsearch and Lucene vector database report card: up to 8x faster and 32x more efficient searches! Read this blog for recent noteworthy work: https://t.co/tZH3Uu53b7

#elasticsearch #apachelucene #vectordatabase

[Duy Do @duydo]

Backdoor in XZ Utils allows RCE https://t.co/O014psr8JK

[Java @java]

Java 22 is now available! #Java22 #JDK22 #OpenJDK

Download Now: https://t.co/aCramdGK3b
Release notes: https://t.co/UIHZ8nwZNu
API Javadoc: https://t.co/RBVjNl6Qpm
Features: https://t.co/DNvcj2yYbo
Inside Java on JDK22: https://t.co/Wh9qOLeL2N

[Duy Do @duydo]

@FrancoisMassot @CloudNativeFdn Congrats @FrancoisMassot 👏

[Sahn Lam @sahnlam]

How Discord Stores Trillions of Messages

In 2015, Discord started using MongoDB to store messages. By November 2015, there were already 100 million messages that did not fit in RAM, causing unpredictable latency.

In 2017, Discord moved to storing billions of messages in a Cassandra database. By 2022, the number of stored messages had ballooned to trillions across 177 nodes.

But Cassandra was plagued by several serious issues.

- Hot partitions occurred when a small number of high traffic channels overwhelmed nodes, cascading latency across the cluster.

- Garbage collection pauses created latency spikes.

- Compactions fell behind, forcing expensive reads to query multiple SSTables.

- Maintenance like node repairs interrupted service.

Discord migrated to ScyllaDB for the following benefits:

- Written in C++ instead of Java, eliminating disruptive garbage collection pauses

- Shard-per-core model provides stronger workload isolation to prevent hot partitions from cascading latency across nodes.

- Reverse query performance optimized to meet Discord's needs

- They reduced nodes to 72 while increasing disk space per node to 9TB.

To further protect ScyllaDB, Discord:

- Built intermediary data services in Rust that limit concurrent traffic spikes

- Data services sit between the API and database, coalescing requests

- Query the database just once even if multiple users request the same data

- Rust provided fast, safe concurrency ideal for this workload

The results? Tail latencies down from 40-125ms to a steady 15ms. Database uptime improved from weekend-long outages to smooth sailing. The system easily handled World Cup traffic spikes, processing events like goals and penalties without breaking a sweat. Discord continues to scale, now reliably storing trillions of messages with ScyllaDB.

–
Subscribe to our weekly newsletter to get a Free System Design PDF (158 pages): https://t.co/kNfv0DVDdf

[Alex Xu @alexxubyte]

Twitter has enforced very strict rate limiting. Some people cannot even see their own tweets.

Rate limiting is a very important yet often overlooked topic. Let's use this opportunity to take a look at what it is and the most popular algorithms.

A thread.

#RateLimitExceeded

[Architecture Notes @arcnotes]

A deep technical dive into all things Redis. Covering various Redis topologies, data persistence and process forking.

https://t.co/yd92b1uXFv

[Duy Do @duydo]

Thank you @elastic for the gift ❤️

[Duy Do @duydo]

@engineers_feed (24 + 20-10)/2 + 10 = 27

[Victor Stinner 🐍 @VictorStinner]

Announcement: "PEP 594 has been implementated: Python 3.13 removes 20 stdlib modules": https://t.co/IAdKptBznZ I wrote a long message explaining the rationale to coordinate efforts to make this migration (to Python 3.12 and then to Python 3.13) as smooth as possible.

[Duy Do @duydo]

So excited with new feature semantic search in #elasticsearch v8.8 https://t.co/sTdaFS0e9e

[Duy Do @duydo]

The vietnamese-analysis plugin has been upgraded to #elasticsearch 8.7.0
https://t.co/Ya4JPfawL6

[Duy Do @duydo]

Reduce 90% infrastructure costs by moving from a #microservices to #monolith
https://t.co/VxPWUEk27b

[Guido van Rossum @gvanrossum]

Microsoft Semantic Kernel now in Python flavor: https://t.co/1AY2zCFWf3

[Duy Do @duydo]

@MorissaSchwartz Ban Gioc waterfall, Cao Bang, Vietnam
It is the largest waterfall in Asia
https://t.co/O9ayio6RiW