Pelikan Cache

A framework for building the world’s fastest key-value caches. Source:

[Pelikan Cache @pelikan_cache]

Twitter Cache team just open-sourced Pelikan, a cache framework. More details on project website: https://t.co/zjXcjrljC9

[Juncheng Yang @1a1a11a]

I talked about S3-FIFO at #SOSP today. S3-FIFO is a FIFO-based eviction algorithm that is simple, scalable, and efficient. For example, it can reduce the miss ratio of your LRU caches by up to 72% and improve throughput by 6x at 16 threads. S3-FIFO is being tried out at a few companies trying out, e.g., Google, VMWare, and a few startups. Come to talk to us if you are interested.

paper: https://t.co/iN4eRMArCa

video: https://t.co/BhhwASYey4

slides: https://t.co/uGLBtfIR6V

discussion: https://t.co/4YIKyK2neh

[Khawaja Shams @ksshams]

I am still amazed at how easy it was to run @pelikan_cache on Ampere Altra-based Tau T2A VMs on GoogleCloud. @Brianiac was able to make it work instantly and make it fly in the matter of hours! 🙇‍♂️

[Pelikan Cache @pelikan_cache]

Yep, we agree.
https://t.co/tmMEu7tsSS

Corollary:
If you do not know what *exactly* Pelikan Cache is, don't worry. It's by design.

[Open Source Startup Podcast🎙 @OssStartup]

🎙️E49 of the Open Source Startup Podcast is LIVE🎙️

Ft. @dymxzvf, Cofounder of caching platform @MomentoHQ & @thinkingfish from the Twitter team for @pelikan_cache

✨Check it out✨ https://t.co/wvrHx9J7sM

[Pelikan Cache @pelikan_cache]

We know our documentation has a long way to go, so we are spending September, heads-down, to write about the design of Pelikan.

If you want to known about anything in particular, please tell us in the reply.

In October, we'll return with a daily tweet series on Rust + Pelikan.

[Pelikan Cache @pelikan_cache]

Almost no where else in the infrastructure software would the following statement be more true than it is for caching:

"Service delayed is service denied."

Make it fast. Make it predictably fast. Make it consistently fast.

#CachingWisdomOfTheDay

[Pelikan Cache @pelikan_cache]

Friends don't let friends enable dynamic scripting in cache.

All it takes is one bug or a poorly implemented algorithm to completely kill server performance.

Scripting is useful but should be treated as other server code- tested, benchmarked, deployed.

#CachingWisdomOfTheDay

[Pelikan Cache @pelikan_cache]

@ksshams People may not want to, but they do (due to lack of alternatives sometimes), then they have to dedup upon LRANGE type of queries. But this is why data structures with position-based indexing only is less user-friendly than value-based indexing (e.g. sorted map).

[Pelikan Cache @pelikan_cache]

You're probably familiar with read vs. write. But also consider:
- idempotent vs. non-idempotent write: the former can be safely retried, not the latter;
- regular vs. privileged command: e.g. you probably don't want people to run FLUSH_ALL willy-nilly.

#CachingWisdomOfTheDay

[Pelikan Cache @pelikan_cache]

Accessing a large in memory dataset is unlikely to be CPU cache friendly, even if it has a fairly skewed popularity distribution.

What could / should be kept in CPU caches? IO buffers.

How? CPU pinning, flow steering, and keeping storage threads away.

#CachingWisdomOfTheDay

[Pelikan Cache @pelikan_cache]

@tivrfoa Yes, chunking works, this was why Twitter added hybridlist (a linked list of zip list) to its forked Redis in 2012, an idea that got picked up by the OSS Redis and gave us quick list.

[Pelikan Cache @pelikan_cache]

In cache with data structures, writes tend to be incremental. It’s important to keep memory op proportional to update instead of object size.

A DIMM carries ~40GB/s. If updates against 1MB objects involve memcpy whole objects, throughput will be <40K RPS.

#CachingWisdomOfTheDay

[Pelikan Cache @pelikan_cache]

@ksshams Twitter’s Redis fork does that. But what would be better is to offer more data structure in Pelikan (we are working on it)

[Pelikan Cache @pelikan_cache]

Always have a TTL for anything put in cache. To list a few reasons:
- it provides a well defined bound on data inconsistency;
- it gives cache backend a strong hint on how to retain useful data for the right duration;
- it helps you stay GDPR-compliant.

#CachingWisdomOfTheDay

[Pelikan Cache @pelikan_cache]

@ksshams The lack of standardized, representative (of production) benchmark suites allows many to game the numbers.

May we suggest playing some (over 50 distinct workloads, in fact) real-world Twitter traces for a change? https://t.co/ucHsb142Mc

[Pelikan Cache @pelikan_cache]

Cache writes are not always more expensive than reads. Simple read/write can achieve similar RPS because RPC costs dominate.

There are some “tipping points” wrt RPC- e.g. you can expect sudden dip on RPS at MTU, socket buffer, and/or request buffer sizes.

#CachingWisdomOfTheDay

[Pelikan Cache @pelikan_cache]

@ksshams To be fair, if rewriting is handled in place, or allows immediate reuse of memory (e.g. if value sizes don’t change much) then fragmentation is less of a problem.

Worst case scenario is a completely different story, of course.

[Pelikan Cache @pelikan_cache]

@tivrfoa And yes, SLO for Service Level Objective, typically defined as good throughput with a particularly p99/p999 constraint.

[Pelikan Cache @pelikan_cache]

Threading model is especially important for secure cache access.

While modern hardware has made encryption much faster, TLS frames add memory & CPU overhead.

Worse still, TLS handshake is 10x more expensive than TCP– connect is a serious threat to SLO.

#CachingWisdomOfTheDay

[Pelikan Cache @pelikan_cache]

@tivrfoa Mostly yes. TLS is used by most inter-service communication at this point. This was a direct outcome of trying to be GDPR compliant.