Event Sourcing and Message Streaming are NOT the same things. Kafka is NOT suitable for Event Sourcing. Kafka is suitable for Message Streaming.
There are important reasons why these two different things require *different storage mechanics*. I'm going to simplify this by naming the two different kinds of storage:
1. Event Sourcing requires a *database* to serve as an *event journal*. Events stored for Event Sourcing must be written in a way that enables future reads to quickly assemble small(er) streams of events that belong to the single Aggregate that originally emitted them. This requires a random access index.
2. Message Streaming requires storage that is essentially a *flat file* that logs message elements. The message elements are individually written in sequential order and later read in sequential order. This requires one first-to-last sequential index.
Based on these two fundamental points, I now break this down further into details:
3. Now, beyond Aggregate sub-streams, all events of an Event Sourced domain model are generally consumed as a totally-ordered stream of events in the *time* order that they were originally emitted by Aggregates. To do so *also* requires a sequential index. Thus, an Event Sourcing *database* must support *two* kinds of indexes.
4. Kafka is *NOT* suitable for an Event Sourcing *database*. What is the name of the concept under which *messages* are *logged*? It's a *topic*. Kafka is a message log that can have many topics. Kafka has *one single index*, which is the sequence number of the *totally ordered stream of messages*.
5. Therefore, after writing messages into a Kafka topic, they cannot be read randomly because the random access index doesn’t exist. Kafka was not designed for this.
6. With Kafka, if you need to read the small(er) streams of events originally emitted by single Aggregate instances, you would have to scan an entire topic from the first message to the last message to ensure you didn’t miss reading all events in one of the individual Aggregate streams. This would cause O(N) read time--reads become slower as every new message is written to the topic. You have 1 billion total events and you need to read any 5 of those as an individual Aggregate event stream? "Ain't-gonna-happen."
For the really stubborn people who think they can outsmart physics, I'll add a few more points:
7. "I know, I know! I'll use a different topic for each Aggregate instance!" That's okay if Kafka was designed to support millions to billions to trillions of topics under a single broker. It wasn't.
8. "I know, I know! I'll use a K-Table to maintain snapshots of each Aggregate instance so I can read them quickly!" Reconstituting the state of an Aggregate must take priority over consuming a totally ordered stream of all events. If you try this, your K-Table snapshots will be only eventually consistent with the *real current Aggregate state*--the Aggregate can't reliably read it's own state.
The End.
As a side note: Eric Evans believes that Domain-Driven Design skills will get more important as AI takes over programming. We'll be able to scale programming but not scale understanding a domain and modelling it.
Thanks to our talented facilitator @ToF_ and to all the participants in #socratesfr for this excellent 7th edition. It was a blast.
See you next year !
On voit que t'étais pas là en 2000 mon p'tit bonhomme, tu serais surpris de voir à quel point le web tenait avec rien et consommait que dalle. Pas de libs JS, peu de CSS qui prenait à peine. (en version 1 et 2), du JPEG comprimé à la schlag car on n'avait que des modems 56k, …
🗣️Prêts pour Devoxx France 2023 ? Retrouvez du 12 au 14 avril Wemanity au Palais des Congrès ! 🙌
Partagez un café et bien plus avec nos experts au Bar à café ☕️
Notre communauté de technophiles et architectes se feront un plaisir vous accueillir.@DevoxxFR
Friend, maybe you are watching protesters degrade some walls and burn garbage in photos and maybe you think of that as violent, or screaming in protest as violent. To that I would like to gift you one of my favorite quotes.
This is a valid Dockerfile for a NodeJS application. It is also a pile of 💩!
We can improve:
- 🔒 Security
- 🏎️ Build speed
- 👁️ Clarity
Follow along as we go from 💩 to 🥇!
(code in alt text)
This code has gotten me thinking. Many have dissed it, but it's actually not too bad. Compared to assembling a string in a loop, it's way more readable and actually executes a bit faster. Those multiple if statements bug me tho, so I came up with the solution in the next tweet.➡️
SoCraTes is back – in real life! 🙌
We are pleased to announce that this years SoCraTes will happen in Soltau from the 25th-28th of August.
Find all the details and sign up for our newsletter here: https://t.co/7tE0sze0FF
#SoCraTes2022
Mercredi 20 avril : à l'occasion de notre participation à @DevoxxFR, l'un de nos coaches techniques Damien Menanteau interviendra en tant qu'animateur avec Phillippe Bourgau de chez @Murex_Group !
Au programme : #refactoring et #TCR ⚙️💪
+ d'infos sur https://t.co/TGZiHixf6e
Retour sur le BBL animé par l'un de nos Tech Leads Mohamed Hamady, qui nous a présenté les nouveautés apportées par #Java 17 et 18 lors de notre Tech Talk en remote💻
Au menu : Record, Sealed class, Pattern Matching et autres évolutions du langage illustrées en live coding !
Thanks to @Wemanity for being sponsor of #socratesfr for the first time. Wemanity is know as a leader in Agile movement with biggest community in Europe, but also has built alongside a Practice of Tech with strong Craftsmanship and DevOps culture.
I am happy to share a memory of the "Software Craft" mindset, as it was collaboratively distilled with several participants of @SoCraTes_FR, during a session.
Here's the resulting article that we just published: "Things We Wish We Knew When We Started"
https://t.co/SH2BTiXPJW
Each day at @SoCraTes_FR start with a marketplace in order to let the attendees define the agenda.
It has been perfectly facilitated by @rachelcdavies and we end up with great topics as usual. I am prepared to be surprised ! #socratesfr