.@knocklabs cut its data streaming costs by 77% (from ~$112K to ~$25K/year) by migrating off Amazon Kinesis to @warpstream_labs.
One engineer (@whatyouhide), 1 month, 5 topics, ~20 consumers. Zero downtime.
Full engineering write-up. https://t.co/jXbVlnN7H1
I put out a GenStage visual explainer a few weeks ago. Only natural follow up is a visual explainer for @elixirlang's Broadway (https://t.co/98GNzUbtGE). 🛝
https://t.co/JWfw9G86wV
There was a bunch of work we had to do in brod itself, the underlying SDK, so you get that on Erlang too!
We're using static membership in a k8s Deployment, using k8s Lease objects to claim each pod's static member ID. Neat stuff
I just released a new version of broadway_kafka (for @elixirlang), adding support for static membership.
That was a blocker for us at @knocklabs for using this: without this, a k8s rolling deploy causes LONG consumption pauses = no bueno.
Give it a try 🙃
@germsvel alas, I think you still need to understand the whole workings of GenStage to get what "resupply" means and whatnot. I don't know that a big ol' rename would particularly be worth it
I just released Xandra v0.20 (@elixirlang driver for @cassandra/@ScyllaDB), which includes SO MANY goodies: token-aware routing, schema agreement, ScyllaDB shard-awareness, and more. Hoping you all will try it out and report issues 😌
https://t.co/KjURHFHBDo
I published a little post about @elixirlang GenStage demand: https://t.co/geiPdw6Avp
It has interactive charts which I totally build line-by-line by hand without any AI. Nah but I’m still really happy to finally be able to put a "visual" explainer like this out there! 🎆
I don't chime in much but this is True™ in my experience too. Sometimes it's blatant and sometimes subtle but I stopped using it because it requires much more scrutiny from me now
I've been heads down into a migration to @warpstream_labs for the past few weeks at @knocklabs. One of the most delightful techs I've worked with. Super excited to share more about it in a bit.
Either I’m doing something wrong, or the type of work I do is so far from this experience. Sure, heavy agentic usage, but nothing remotely close to the luxury of "just smashing Enter"
After working for a month with 20+ concurrent Fables all day, I've realized my job only has a few components to it anymore:
- asking for what I want
- setting up accounts and credentials
- taste-making choices presented to me
- communication with team and customers
@distantprovince@elixirlang End-to-end time was not that much different, if I had to guess. Less mental effort—probably but understanding code you didn't write is still a lot of mental effort 😬
I just shipped Redis Cluster support in Redix (@elixirlang's driver). I have not written any code—it's all agents.
I'm ready to get pushback, but it was a great experiment 🫳
I wrote about the whole experience, too: https://t.co/nV6bJcy9Dg
New version of nimble_lz4 for @elixirlang is out with support for stream encoding/decoding. This is neat and enables production-grade support for many use cases—Kafka, ClickHouse, Cassandra, and more—where all-in-memory is not feasible.
https://t.co/DKrGrqpHRC
Just pushed a big release for the Protobuf @elixirlang library:
https://t.co/qCxulmN4Va
textproto support, tons of bug fixes, lots of little paper cuts that are no more. Please try it out and let us know if anything's off. 🙃
This is what I've been working on for the past ~1 year. All in @elixirlang, mostly powered by the workhorse that is @ClickHouseDB. Big undertaking but one of the most fun things I've ever worked on 🎫
User segmentation is now generally available in Knock.
While this feature is common across customer engagement solutions, we built Knock's version to go further than most. Here's how 👇