Rubén Terceño

Born in Cantabria | Living in Madrid | Full time father (x3) | Part time chef | Making technology easy for people

[Rubén Terceño @rubenTerceno]

@Dictatriz @FiQuiPedia @Naturgy Si, si, lo que quería decir es que contratar la TUR con ellos fue muy sencillo (he tenido experiencias peores en el pasado) y que desde que estoy con TUR ninguna tarifa de las que me han ofrecido cuando me llaman era mejor.

[Hernán Rodríguez @rodriguezhernan]

La clave para reducir costes de infraestructura con Apache Kafka está en optimizar el uso de red, lo explica @rubenterceno de @Confluentinc en su última tribuna en @l_e_digital. ¡Un must-read para los CIOs!

🔗 https://t.co/fp81B2BHhn

#ApacheKafka #InfraestructuraTI #Confluent

[Stanislav Kozlovski @kozlovski]

btw, I am launching a weekly newsletter centered around Apache Kafka

It's called 2 Minute Streaming

- main idea is to cap each post at a 2-minute read (476 words). We're all too busy!
- 1 e-mail a week. Everybody can spare that

Subscribe 👉 https://t.co/k9e0fK2ntH

[Stanislav Kozlovski @kozlovski]

Surely you’ve heard of Apache Kafka’s KRaft?

But what is it? An algorithm? A deployment mode?

KRaft (“Kafka Raft”) - is a dialect of Raft, with a few differences. It basically extends Kafka’s existing replication protocol with a few Raft-related features.

Kafka in KRaft mode - deploying Kafka without ZooKeeper (ZK)

But first - why Remove ZooKeeper?

There are a few improvements to be made when ZooKeeper is removed from Kafka:
🔸 simpler deployment and lower cost - it’s easier if the deployment is all within one system (Kafka), and can also be cheaper.
🔸 easier maintenance - different project means different experts. There are not many people in the world that are both ZooKeeper and Kafka experts. Who do you find to triage and solve ZooKeeper bugs?
🔸 single, easier security model - need to secure just one system.
🔸 scaling - loading a lot of data from ZK was expensive and slow. Large Kafka clusters would also have a lot of writes to ZK, and ZK isn’t made for such throughput.
🔸 less unavailability - the slower you load data, the greater the unavailability window when a Kafka controller fails. KRaft is a 15x (!) improvement in some hard-failure cases.

Second - what problem did ZK solve?

ZooKeeper solved the problem of distributed consensus for Kafka - the question of “what is the latest metadata state of the Kafka cluster?”.

Since ZK solved that, Kafka stored all of its metadata there and leveraged it to do the hard task of electing a single Kafka Controller. The Kafka Controller would then be responsible for managing the ZK metadata and acting accordingly to changes there (e.g broker failure).

So... how does Kafka work without ZK?

Four major pillars you should remember: 
🧠 Records all the way down - every metadata change is captured in a plain old Kafka topic (metadata topic) with plain old Kafka records representing the delta. This topic is replicated by all brokers. Naturally, when you read all of the records, you get the latest state (look up stream/table duality).
🧠 Hot Standby and Incremental Updates - since everything is modeled in records, a broker that is replicating the metadata topic has the latest metadata state loaded on disk. On restart, it doesn’t need to fetch everything over the wire - a large portion is stored locally!
🧠 Push → Pull Model - Brokers now pull (fetch) data from the Controller, instead of the Controller sending requests to them. Every broker is a follower of the metadata log, but only a few can become active controllers.
🧠 Separation between Controllers and Brokers - All brokers replicate the metadata topic, but only a few are eligible to become the leader (active controller). As part of deploying in KRaft mode, you configure precisely which brokers are eligible to become controllers. This allows you to isolate and scale them separately.

These changes significantly improve efficiency and pave the way for many further improvements down the road!

Further, this architecture is much more event-streaming native. It makes total sense that Kafka embodies the principles behind event-streaming.

Thanks to all the Kafka contributors who worked on this!

[Jay Kreps @jaykreps]

1. Here’s a summary of my #kafkasummit talk for those who missed the live stream with links and pics of all the things we announced!

[Ángel Abellán @Angelmonteys]

Ayer por fin cerré una etapa de mi vida y me doctoré. Como sabéis, la ciencia tiende a ser fría y hermética, y te puede poner zancadillas para que no te salgas de la línea, para que no intentes hacer algo diferente.

Pues bien, estás son las primeras páginas de mi tesis:

[P. Duchement @PDuchement]

2️⃣0️⃣Por cierto, si crees que el Excm. Señor Alcalde @MiChavarria debería responder a las tres preguntas, un RT a este tuit sería de ayuda, para que entienda que no es una iniciativa de un motivado, sino de un sector de la población que cree en la ciberprotección infantil.

[Arnold Schwarzenegger @Schwarzenegger]

I love the Russian people. That is why I have to tell you the truth. Please watch and share.

[Sergio Durán Vegas @Mr__Duran]

¡Participaré como ponente en el #DevCast de @CodemoMadrid el 27 de Enero hablando de arquitecturas de micros en entornos cloud. Si quieres seguir la charla puedes inscribirte gratuitamente a la serie Cloud aquí: https://t.co/WEb2m47QtO

[P. Duchement @PDuchement]

¿Tienes menores adolescentes activos en redes?

Quizás te gustaría saber que las claves (y hashtags) “Ana” y “Mia” son códigos de comunidades digitales A FAVOR DE LA ANOREXIA Y DE LA BULIMIA.

Siguiendo estas palabras, tus peques pueden descubrir cosas tan educativas como:

[Confluent, an IBM Company @confluentinc]

It was originally designed and built for self-managed infrastructure, deployed on physical servers over numerous datacenters at @LinkedInEng—so how did @apachekafka become cloud native over the last 5 years? Read this technical paper to learn how: https://t.co/jlnkfyY5Uu

[Confluent, an IBM Company @confluentinc]

Looking to share the power of real-time data across your company? Governance for #DataInMotion is key! For a nice overview of our Stream Governance suite, check out this article by @alex_woodie. Advice from @DanRosanova included!

[Confluent, an IBM Company @confluentinc]

Level up your @apachekafka knowledge with our new Premium Education Subscription. You’ll get unlimited access to our huge library of instructor-led courses and self-paced curriculum. Plus, 2 vouchers for Confluent Certification exams! https://t.co/e8BYodiYqJ

[Confluent, an IBM Company @confluentinc]

Get hands-on experience working with streaming data and creating powerful, event-driven applications. Sign up for our new #Developer Learning Path, now accessible through our self-paced subscription! @apachekafka https://t.co/h0WClQCMRp

[Confluent, an IBM Company @confluentinc]

The edge got you on edge?

Join us @AWSreInvent on 12/1 and we can talk you off the cliff by showing you how to run data-in-motion and event-based applications at the edge easier and faster.

You can find more on our #reinvent event page here.

https://t.co/jKgQ3YrKX1

[Confluent, an IBM Company @confluentinc]

We're packing our bags and can't wait to see you in person at @AWSreInvent. Swing by our booth (#220) to say hi and we can show you how we make it easy to connect all your existing data sources to AWS. Let’s set #DataInMotion in Las Vegas! #reInvent

[Confluent, an IBM Company @confluentinc]

✅ Heading to @AWSreInvent
✅ Love @awscloud edge services
☑️ Running full speed at the edge

You know you want to check that box. Join us on 12/1 to learn how to pull it off with Confluent for Kubernetes and Cluster Linking. #reInvent

[Confluent, an IBM Company @confluentinc]

What is stream processing, and how does it work? Join our 3-part series and we’ll cover everything you need to know, starting with basics of stream processing in @apachekafka. Register here for FREE: https://t.co/OuDmzj0wuf

[Confluent, an IBM Company @confluentinc]

Learn how to break down data silos and accelerate time to market for new applications in this upcoming webinar hosted by us and @awscloud. Register here: https://t.co/5vm6GI3lU2