Matthew Hartstonge

💻 Development Growth Lead. ☕ Turns coffee in code. #Go #Backend #DevOps #CloudNative #SoftwareEngineering

[Matthew Hartstonge @MattHartstonge]

New blog looking at why @heroforschools uses #Go - Cross-compilation!

When working with many clients on disparate systems, Go's cross-compilation speaks to why you shouldn't pass up on deciding if #Golang is the right tool for your next project!

https://t.co/ciuxI5ESJr

[Matthew Hartstonge @MattHartstonge]

🦋 https://t.co/ncx3J9pnTW

[Matthew Hartstonge @MattHartstonge]

@andrewmacfnz

[Matthew Hartstonge @MattHartstonge]

@vinnysgreen ✨

[Matthew Hartstonge @MattHartstonge]

.@ThePrimeagen eyeing up his next Neovictim …

[Matthew Hartstonge @MattHartstonge]

It’s pronounced squeal

[WebStorm, a JetBrains IDE @WebStormIDE]

🎉 Just announced! WebStorm is now free for non-commercial use!

Whether you’re learning #JavaScript, working on open-source projects, creating content, or coding as a hobby, you can now do so for free 🥳

🔗 https://t.co/oTcCcGNa8H

[NullVoxPopuli @nullvoxpopuli]

Hey 88% of you not interested or have no opinion on #EmberJS, modern EmberJS is a delight, and a huuuuge breath of fresh air among the insanity of JavaScript.

I've been working on an interactive tutorial here:
https://t.co/g7i7uIMfxo

Let me know if you have questions / complaints / concerns / etc

[Matthew Hartstonge @MattHartstonge]

@LiamLawson30 New Zealand’s proud of you bro! 🥝
You’ll smash it 🤩🤙

[Matthew Hartstonge @MattHartstonge]

@ibuildthecloud Flutter? 👀

[Matthew Hartstonge @MattHartstonge]

@ThePrimeagen Same crew realising their website had been changed without forewarning:

https://t.co/F2RXnkCYMt

[Matthew Hartstonge @MattHartstonge]

@ThePrimeagen Their own con not being able to sign in:

https://t.co/Z3uJt7JqCP

[WordCamp Sydney @wordcampsyd]

We're also asking for clarity on the new login checkbox "I am not affiliated with WP Engine in any way, financially or otherwise". The organising team can't log in to our WCSyd website without clarity.

[Matthew Hartstonge @MattHartstonge]

@nullvoxpopuli Ah yes, JSSugar…
AKA coffeescript, flow, TS etc…

[Matthew Hartstonge @MattHartstonge]

@rameerez You’re still ‘in the cloud’ though?
Or is it more about managed offerings?

[Matthew Hartstonge @MattHartstonge]

DannyRic bringing back this banger on Lap 60 @LandoNorris @danielricciardo

[Matthew Hartstonge @MattHartstonge]

@t3dotgg Crayz given that FL Studio updates are freeee 😭

[Stanislav Kozlovski @kozlovski]

A Kafka in the cloud doing 30MB/s costs more than $110,000 a year.

A $1,000 laptop can do 10x that.

Where did we go wrong? 👇

The Cloud. Namely - its absurd networking charges 👎

Let’s break it down simply:

• AWS charges you $0.01/GB for data crossing AZs (but in the same region).
• They charge you on each GB in and out. Meaning each time a GB passes, you pay twice - for the one who sends it (outgoing) and the one who receives it (incoming)
• For a normal Kafka cluster with replication factor of 3 and a read fanout of 3x, you are going to be charged:
• 2x for 2/3rd of the produce throughput
• 4x for 100% of the produce from replicating it
• 6x of 2/3rd of the produce throughput for consumption.

(but it can get a lot worse - read until the end to see)

Simple example:
• 3-broker cluster, each in a separate AZ
• 3 producers, each in a separate AZ
• 3 consumer groups with 3 consumers each, each group with consumers in a separate AZ

The producers are producing 30MB/s in total to the same leader.

2/3 producers are in a different AZ, so 20MB/s of produce traffic is being charged at cross-zone rates. 👌

It’s charged both on the OUT (producer’s side) and IN (broker’s side).

The leader is replicating the full 30MB/s to both of its replicas.

This is again being charged both on the OUT (leader’s side) and IN (follower’s side), for both replication links. (60MB/s)

Then, each of the 3 consumer groups has 3 consumers.

All consumers read from the leader, with 2/3 in a different zone.

This results in 20MB/s of consume traffic charged at cross-zone rates PER GROUP. (60MB/s total)

Again charged both on the OUT (broker’s side) and IN (consumer’s side).

The total amounts to 140MB/s worth of cross-AZ traffic. Charged both ways.

When one MB is $0.00001/s, this means we’re paying $0.0028/s. 🤔

That’s:

• $241 a day 😕
• $7500 a month 😥
• $88,300 a year 🤯

It all goes down the drain on network traffic ALONE. 🔥

What about the hardware?

Quick napkin math assuming:

• 7 day retention
• all of the data is on EBS (not using tiered storage since it's not GA yet)
• keeping 50% of the disk free for operational purpose (don't ask me what happens if we run out of disk)
• the 3 brokers are running modest r4.xlarge instances (kinda overkill but hey, why not)

We'd pay:

• $19,440/yr for the EBS storage
• $6,990/yr for the EC2 instances

That’s right - you’re paying just $26.4k/yr for the hardware and 88.3k for the network (3.3x the hardware)

For a total of $115k/yr. 💸

I’m not even counting load balancer costs, which could be $12k by some quick napkin math too.

How ridiculous is that? 😂

Want it to get more ridiculous?

This calculation assumes you’re hosting your own Kafka cluster in the same AWS account.

💡If you use a managed Kafka provider that’s not AWS, or otherwise just another AWS account, you’re typically connecting to them through a public endpoint.

AWS then charges all traffic at the cross-AZ $0.01/GB rate internet traffic rate, even if it's in the same AZ.

The end result?

$113,000 a year for network costs. 💀

For 30MB/s. (!!!)

btw - 30 MB/s is absolutely nothing for Kafka... 🤡

It is most often network/disk bounded.

Doing 3GB/s is not hard. 👌

The higher throughput you go, the more absurdly large this discrepancy between network and hardware cost becomes.

For example - this exact setup could probably do 3x the traffic (90MB/s), assuming storage space isn't a concern.

Then you'd have:

• $264,000 a year for the cross-AZ rate. 🥲
• $339,000 a year for the internet rate. 💀

Why is this cost (more than 300k a year) and complexity (this calculation) the case when three laptops can run this practically for free?

Where did we go wrong?

Worth Noting:

There are a few optimizations that can be done here:

• consumers can use fetch from follower, which results in free read traffic (no cross-AZ charges) in the first example. But the second example would still be charged internet costs. 🤝

• you can avoid internet costs by VPC-peering or Private Link-ing the two AWS accounts. This is largely what most cloud providers do, otherwise it becomes prohibitively expensive. It can be super complex to do. 🔧

• AWS can give you large discounts (up to 90%+ afaict) on the quoted prices, depending on your usage. It’s unclear what customer gets what discount. 💰

And perhaps the best example - you can use an ingeniously-designed product like WarpStream that eliminates all of this complexity and cost. ⭐️

It's no wonder they got acquired after just 13 months of operation.

[Matthew Hartstonge @MattHartstonge]

@MarkRober @hinge Maybe @hinge are moving into their “we delete it for you” era ?

[Matthew Hartstonge @MattHartstonge]

Requiring Python to build C++, so that you can run your JavaScript.
The absolute state 😅

[Matthew Hartstonge @MattHartstonge]

@teej_dv @tsoding WHY HAVE YOU DOOMED US ALL TO WRITE C++ ADDONS 😭

[Matthew Hartstonge @MattHartstonge]

But not when you have two managers installed… 🙄

I have @LastPass for work and @Bitwarden for personal and browsers don’t appear to support routing of where to store the key…

[Matthew Hartstonge @MattHartstonge]

DHH with another banger. 💥

Passkeys work well though, ironically, if you use a password manager.

[DHH @dhh]

"Bottom line, I'm disappointed to report that passkeys don't appear worth the complexity of implementation (which is substantial!) nor the complexity and gotchas of the user experience. So we're sticking to passwords and emails." https://t.co/sx4pQ0FT0Q