Principal software engineer @Cloudflare. Fan of distributed systems, databases, and donuts. Previously at @CockroachDB and @Google on Kubernetes and GKE
Talking to traditional, centralized databases without adding latency is one of the hardest challenges of moving to applications to run in serverless functions. Most of us are all used to co-located DB and compute.
Hyperdrive is our answer.
https://t.co/Eq89T4jn3r
@jadbox@Cloudflare@supabase The cache settings are fully configurable, so please do customize them lower (or disable caching entirely, or use a separate Hyperdrive with caching disabled for specific queries).
As for automatically invalidating cached data when writes occur, we'll be working on it :)
@justinjaffray It's not as sophisticated as you're probably hoping for, but the basic idea of "let me add indexes on a bunch of arbitrary data" is implemented in capnproto's C++ standard library replacement and works fairly well - https://t.co/e1mTgcINkm
Come work with me at Cloudflare! We're hiring a product manager for Durable Objects and a director for all storage-related products under the Workers umbrella. Reach out if you have questions or want to apply!
https://t.co/lLeTU9QcBu
https://t.co/uCN2iEkX32
A very wonky blog post. We noticed many Durable Objects apps had classic storage bugs (race conditions, unconfirmed writes, long waits, etc.). Instead of telling people to fix their code, we fixed the system, so intuitively-written code is now correct. https://t.co/KMUGRJWLfl
@dctanner@jkarneges@KentonVarda@Cloudflare@jgrahamc@eastdakota@pusher@fanout 10k rps to a single Durable Object is definitely not going to work. A single object tends to max out in the hundreds of rps even for simple request handlers today. Like Kenton suggested, to hit that scale on a single counter you'd need some sort of fan-in.
@thescottsb@SergiiShymko@Cloudflare I can assure you the data in KV is persistent :) Do you have any recollection of where you got the impression it might not be? I tried going back through the original announcement blog posts but didn't see anything suggesting otherwise.
@emaxerrno To get decent sync performance on GCP local SSDs, try the nobarrier mount option on your filesystem. See https://t.co/14kDjdoK2P for some discussion and benchmarks of this
@bmatican@ayende@ittaia@arnonrgo I was also surprised to see etcd hasn't enabled it by default yet, given we added it to their raft impl back in 2016 https://t.co/43vz689uQf. I worry it speaks to the lack of resources etcd development gets these days. Imagine how many k8s clusters are out there running like this
@natevanben@ittaia@Cloudflare@tlianza Yeah, it's the right question. Pre-vote still isn't enabled by default in etcd since they released it as an experimental feature in v3.4 in August 2019 and have yet to release a v3.5 with it turned on by default.
@ahmetb Who said anything about global replication? The whole point of breaking your application up into independent objects is that each one can be located/replicated in the region closest to where it's most frequently used.
@DenisTRUFFAUT@KentonVarda Sure, you could store any of those things, and using the Cache API in front can make a lot of sense.
There isn't an API to remove entire objects at the moment, but there will be.
@avaitla16@KentonVarda We do use quorum replication to be able to stay online with no loss of consistency in the face of single node crashes or colo outages.
I'm really excited for the Workers Durable Objects beta - they make it easy to build stateful, coordinated serverless applications that run around the globe close to where they're used without the developer needing to think about where they're running https://t.co/B1jA5erbo3
@jessepmason@Cloudflare Durable Objects are free for the time being during the beta period. See here for more info on what pricing will be like after beta https://t.co/QNyLadpb5C
@joshpowell100@Paul_Reviews@Cloudflare@Azure I'm obviously not Paul, but Workers can make arbitrary outgoing HTTP requests, and Azure table storage conveniently offers a REST API over HTTP: https://t.co/VzBJ1o8OBt