David Vallejo

Mobility developer , DevOps , security , IA , robotics , music & related stuff . Opinions are my own

[Uber Engineering @UberEng]

https://t.co/p4OzJicyw8

[Kube Builders @KubeBuilders]

Radar provides Kubernetes cluster visibility through topology graphs, event timelines, and service traffic visualization running as a single binary that connects directly to the Kubernetes API without cluster-side installation

➤ https://t.co/POAtNJLOxG

[Ayaan 🐧 @twtayaan]

🚨 AWS just killed the most annoying thing about S3.

For years, bucket names had to be globally unique.

If someone on the other side of the world took "prod-logs"...

You were stuck naming yours "prod-logs-company-us-east-1-xyz".

That ends today.

AWS just introduced Account Regional Namespaces.

Now, your bucket name only needs to be unique within your specific account and region.

You can finally just name your buckets:
- logs
- images
- backups

No global naming conflicts. Clean code. Predictable infrastructure.

You can enable it right now in the console or via API across 37 regions.

But here’s the catch.

Your existing buckets are stuck on the old global model.

They cannot be renamed.

[International Cyber Digest @IntCyberDigest]

🚨 CRITICAL OPENSSL SECURITY ALERT 🚨

CVE-2025-15467 affects OpenSSL's processing of CMS/S/MIME messages.

An unauthenticated remote attacker can cause DoS or execute code remotely by crafting a specific message.

We estimate the CVSS score is 9.8.

We developed a working PoC (!) and recommend updating to the latest version ASAP.

[sam henri gold @samhenrigold]

Did you know your MacBook has a sensor that knows the exact angle of the screen hinge?

It’s not exposed as a public API, but I figured out a way to read it and make it sound like an old wooden door.

[Danilo Poccia @danilop]

I have been looking forward to this new capability! It’s now easier to learn and start in the AWS console and then move a workload in production managing the infrastructure as code:

Convert AWS console actions to reusable code with AWS Console-to-Code, now generally available

https://t.co/UjzOtdYrmg

#AWS #DevOps

[Winix @WinixRun]

Hace un tiempo detectamos en uno de los servidores de mi empresa una serie de escaneos de puertos y decidimos montar una trampa divertida. Un Tarpit llamado Endlessh.

https://t.co/S56rTQyIet

[Rohan Paul @rohanpaul_ai]

Interesting Github Repo

An open-source clean & customizable RAG UI for chatting with your documents.

[Kube Architect @K8sArchitect]

Kubernetes Resource Recommender is a CLI tool for optimizing resource allocation in Kubernetes clusters

It gathers pod usage data from Prometheus and recommends requests and limits for CPU and memory

This reduces costs and improves performance

➜ https://t.co/LdHNYG201w

[Denis @elpruvianu]

😡 Ya decía yo que nos apestaba la ropa después de tenderla.

Es la moda del #CHISQUING. Así acosan las comunidades de vecinos en ASTURIAS a los pisos turísticos.

#Turismofobia

[Gunter Rotsaert @mydevlprplanet]

Software Architecture Guide. By Martin Fowler.

https://t.co/AyV3VHiAt0

[Denis @elpruvianu]

😡 Nos han cobrado 153,7€ en la RUTA DEL CARES por no portar MADREÑAS. Nadie nos había informado y las hemos tenido que comprar allí.

¿Esto es lícito? Tengo los pies en carne viva.

[Kube Architect @K8sArchitect]

This article presents a case study of application instability caused by incorrect probe configuration and suggests good probe setup practices

➤ https://t.co/Pg78gdnPQ3

[Kube Architect @K8sArchitect]

One challenge in cloud-distributed systems is keeping network traffic within the same availability zone

Kubernetes introduced Topology Aware Routing to address this issue

This ensures requests between apps remain in the same zone

➤ https://t.co/GjuIuE5Wr9

[Stanislav Kozlovski @kozlovski]

Everyone is using Kafka.
​
But almost no one is using its new Infinite Storage feature. ✨
​
KIP-405 is introducing the ability to store Kafka data in S3.
​
And any other external store for that matter.
​
It’s incredibly needed, because storage is Kafka’s biggest flaw right now.
❌
​
Kafka was not originally developed with elasticity in mind.
​
It’s key limitation is that it co-locates the data with the broker, resulting in many problems:
​
1. Competition for disk IOs ⚔️
​
Historical consumers can decrease write performance by up to 43% (as shown in the tests below) because they force Kafka to read from the disk (as opposed to its page cache) and that causes extra disk strain.
​
This is especially bad for HDDs, which have notoriously improved exponentially in all aspects BUT their IOPS capacity. 👎
​
They have been stuck at roughly 120 IOPS for the last two decades - so you can’t allow that precious IOPS to be used up… without expecting catastrophic latencies that is.
2. Speaking of HDDs - Kafka is practically REQUIRED to use them due to their cost-effectiveness. 💾
​
But they can give you large tail latencies.
​
With tiered storage, you can afford very fast small local SSD storage. ⚡️
​
This also means you can provision less memory because you don’t need to be as reliant on page cache for serving historical reads. 💡
​
Previously, you’d be reliant on it so that you reduce hits to the disk in historical read scenarios which result in prolonged periods of under-replicated partitions, and extra network bandwidth of the cluster being used up.
​
3. Disastrous broker disk failure scenario - in such cases, the broker has to re-fetch everything it had on disk (TBs).
​
This sudden extra historical read traffic at max allowed throughput severely impacts latencies.
​
Such broker start up scenarios can be 12,000% slower and can worsen produce latencies by up to 900%, as shown by tests below. 😱
​
4. Slow broker failure recovery - if a broker simply restarts for whatever reason (e.g host VM died), it has to catch up with a lot of data, proportional to the time it was dead.
​
This again exhausts precious IOPS, causes extra bandwidth and just generally takes slower to resolve under-replicated partitions. 🐌
​
5. Reassigning partitions - partitions that have a lot of data are extremely slow to move.
​
At a decent 100MB/s replication rate, 10TB of data moves at a whopping 27.7 hours! That’s more than one day!
​
In the intermediate state, this means you're replicating 2x as much data for 28 hours. 🤦🏻‍♂️
This means that any actions like expanding or shrinking a cluster will realistically take you days to finish, while also consuming a TON of extra replication bandwidth. It's a real disaster.
​
And finally - you can never effectively rebalance partitions in a reactive way to resolve any problem fast. (since fast isn't measured in hours) 👎
​
6. Impractical to scale storage 😮‍💨
​
If you decide to increase your retention settings across the cluster for whatever reason (e.g GDPR), you either need to:
​
• scale horizontally: add new brokers and waste unnecessary CPU/memory resources)
• scale vertically: do some complex and fragile disk swaps on them
​
7. Your cluster set up ends up impacted by disk requirements 😠
​
You end up with a significantly larger cluster than you would need if disk size wasn’t a concern. i.e you’re buying extra CPU/memory you don’t need. 👎
​
This is because HDDs have limited size - so there are cases where you may need to use extra machines just so you can place more HDDs in there.
​
Not to mention the extra maintenance burden for supporting more nodes.
​
8. High cloud cost ☁️ 💸
​
In the cloud, it’s more expensive to provision larger disk volumes that are attached to the instance.
​
9. Max storage limitation per partition 🛑
​
You’re limited by how much data you can store on a single partition based on the limit of the physical disk on the broker.
​
While admittedly a very niche use case, why couldn’t you have a single partition that consists of terabytes of historical data?
​
Those are a lot of problems...
​
How do we solve all of this? 😥
​
Simple. Put the data in S3 ✨
​
That is what Tiered Storage is - it extends Kafka’s storage beyond the local one by retaining the data in a pluggable external store (HDFS, S3, etc).

Pluggable is a key word here, as it will enable the open-source community to develop different implementations for different external stores in parallel. This can be implemented via the RemoteStorageManager interface.
​
Kafka will end up having TWO tiers of storage placement:
1. a local one (hot) 🥵
2. a remote one (cold) 🥶
​
You will be able to enable this uniquely per topic, with varying local and remote retention settings.
​
This will be done transparently to any clients - they won’t be able to tell when they’re fetching from the remote store as the Kafka API remains the same and simply abstracts it away.
​
😡 Won’t this kill latency?
​
In theory, one should expect slower reads from the remote log store.
​
But. This isn’t a problem practically as historical workloads are usually not performance sensitive. 💡
​
The latency-sensitive workloads usually read from the tail of the log (latest data), and are therefore not impacted by this feature.
​
Performance tests were done nevertheless! (using HDFS as the external system)
​
They focused exclusively on write latency and the impacts there:
​
• The largest produce latency increase in the tests was 21ms → 25ms of p99 produce latency in the steady state.
​
With different scenarios came different results.
​
Get this - when there are historical reads (out of sync consumers), the produce latency was actually improved! 🔥
​
This is because without tiered storage, consumers reading old data compete for IOs on the disk for reading (normal consumers don’t since data is served from pagecache).
​
This reduces the IOs that writes can get and write disk latency increases. ❌
​
The tests showed 42ms of p99 produce with tiered and 60ms of p99 produce without tiered storage in this historical consumer scenario - a 28% latency decrease.
​
And the heavy-hitter final case - rebuilding a broker with an empty disk.
​
For just 12TB of data, recovery took almost 4 hours in their test without tiered storage, and only 2 minutes with tiered storage (a 120x improvement).
​
During this broker recovery, the p99 produce latency was 490ms without tiered and 56ms with tiered - a 9x improvement!

Most importantly?
​
This completely flips the script and enables Kafka to be used as a true long-term store.
​
This is strongly in opposition to its most widely-used use-case today, which is a durable but ultimately ephemeral storage - akin more to a pipe than anything else. 🪈
​
Lakehouses are all the buzz today, but has anybody pondered about what a Streaming Lakehouse might look like? 🤔
​
--
​
Let's be frank... is there any place with better Kafka content on the internet right now?

If you like this and want to see more, help me out in two ways:
​
1. Repost so your network learns too! 💥
2. Follow me here for more high-quality Kafka content - ✅ @kozlovski
​
They take 10 seconds to do - writing this takes me 10+ hours. ✌️
​
#Kafka #ApacheKafka

[Danilo Poccia @danilop]

AWS Secrets Manager announces open source release of Secrets Manager Agent 👉 A language agnostic local HTTP service that you can use in your compute environments to read secrets from Secrets Manager and cache them in memory https://t.co/JIiXqMJYlp #AWA #Security #OpenSource

[Danilo Poccia @danilop]

Today, we’re announcing a public preview of AWS App Studio, a generative AI-powered service that uses natural language to create enterprise-grade applications in minutes, without requiring software development skills. More in @donnieprakoso post:

https://t.co/7ptDsOpNuS

[Kube Architect @K8sArchitect]

Visually presented Kubernetes (VpK) is an application that presents Kubernetes resources and configurations in a visual and graphic fashion

➜ https://t.co/GIGl3HbC2i

[David Vallejo @DaVallejoP]

@RDarrylR The aws-auth ConfigMap is deprecated. The reccomended method to manage access to Kubernetes APIs is Access Entries.
https://t.co/VDiAXel1pa

[Sammy Atman @SammieAtman]

Our project to run DOOM on LLMs won 🥇 place in the finetuning track the @MistralAI hackathon.

Yes, there are better approaches you could use to train a game playing agent, but we just HAD to find out if Mistral-7B could run DOOM.

More details on what we did..