@GergelyOrosz@ShopifyEng It's 66M messages per second. We use Kafka's monitoring: kafka.server:type=BrokerTopicMetrics,name=MessagesInPerSec,topic=([-.\w]+)
https://t.co/EOGA9RxwOZ
A few more fun #apacheKafka peak stats during Black Friday for @ShopifyEng:
- 120% growth from 2023 with 66.3 million messages per second
- 4.89 PB of data
- 6.3 ms latency
Yesterday was an incredible day for our merchants and for Shopify:
* Merchants drove a record-breaking $5 billion Black Friday in sales (GMV)
* Peak sales / minute for yesterday = $4.6 million
* Shopify's egress processed more than 173B requests
* Peak requests per minute was at 284 million on edge, with more than 80 million on app servers pushing 12TB a minute
* 45 million peak database queries per second, peak database writes was 7.6 million per second
* We rely on Kafka. At peak we hit 66 million messages per second
Let's keep going 🔥
A few more fun #apacheKafka average stats during Black Friday Cyber Monday at Shopify:
- 1.5 trillion messages per day
- 2.9 PB of data
- 4.8 ms latency
Our #ApacheKafka streaming infrastructure served 29 million messages per second at peak. That’s 45% growth from 2022 – and we did it more efficiently this year with 14% fewer brokers.
I enjoyed discussing the different solutions to improve #apachekafka resiliency at #current23 the last couple of days!
The recording to my talk should be available next week. In the meantime, here's a link to my slides: https://t.co/dsXGLUHFsP
I am once again a speaker this year at #Current23! I'll be talking about how to improve the resiliency of #ApacheKafka clusters with DNS traffic management and client wrappers. Thanks for having me back and I look forward to seeing you next week!
👋🏼 The streaming platform team @ShopifyEng had an absolute blast participating in #Current22 the past couple of days! With the event wrapped up, here are some of our talks that you should definitely check out when the recordings are available 👇🏼🧵:
Neisi Dajomes of #ECU wins #Gold in the women’s weightlifting 76 kg!
It’s the first time Ecuador have won two medals at the same Olympic Games and Dajomes becomes the first Ecuadorian woman to win an Olympic gold medal!
@ IWFNet #StrongerTogether | #Tokyo2020 | #Weightlifting
I wonder what Space Aliens would say, if told that Earthlings shift clocks by an hour in the summertime to fool themselves into thinking there's more sunlight in the day.