Top Tweets for #Messagequeue

Message queues are indispensable for building robust large-scale systems Decoupling - Message queues decouple senders and receivers by enabling asynchronous communication through a queued buffer. - Senders simply dispatch messages to the queue without waiting for a response from receivers. This prevents senders from being blocked while waiting for receivers to process messages. - Senders and receivers are not tightly coupled through direct synchronous calls. This prevents them from blocking each other while exchanging messages. - Decoupling allows senders and receivers to scale independently as traffic grows. It also prevents bottlenecks during spikes in traffic. Fault Tolerance - Message queues provide fault tolerance by buffering messages even in the event of a receiver failure. - The messages remain safely in the queue until the failed component recovers. - This prevents message loss and allows smooth recovery from failures. Scalability - Multiple receivers can process queued messages in parallel, increasing throughput. - Additional receivers can be added to handle more load, dynamically scaling to demand. - This makes the overall system easily scalable to handle increasing traffic. Message queues are indispensable for building systems ready for the scale and unreliable conditions of the real world. Have you ever seen an epic system crash that message queues could have saved? Tell us your war stories! – Subscribe to our weekly newsletter to get a Free System Design PDF (158 pages): https://t.co/kNfv0DVDdf

IBM MQ -> RabbitMQ -> Kafka ->Pulsar, How do message queue architectures evolve? 🔹 IBM MQ IBM MQ was launched in 1993. It was originally called MQSeries and was renamed WebSphere MQ in 2002. It was renamed to IBM MQ in 2014. IBM MQ is a very successful product widely used in the financial sector. Its revenue still reached 1 billion dollars in 2020. 🔹 RabbitMQ RabbitMQ architecture differs from IBM MQ and is more similar to Kafka concepts. The producer publishes a message to an exchange with a specified exchange type. It can be direct, topic, or fanout. The exchange then routes the message into the queues based on different message attributes and the exchange type. The consumers pick up the message accordingly. 🔹 Kafka In early 2011, LinkedIn open sourced Kafka, which is a distributed event streaming platform. It was named after Franz Kafka. As the name suggested, Kafka is optimized for writing. It offers a high-throughput, low-latency platform for handling real-time data feeds. It provides a unified event log to enable event streaming and is widely used in internet companies. Kafka defines producer, broker, topic, partition, and consumer. Its simplicity and fault tolerance allow it to replace previous products like AMQP-based message queues. 🔹 Pulsar Pulsar, developed originally by Yahoo, is an all-in-one messaging and streaming platform. Compared with Kafka, Pulsar incorporates many useful features from other products and supports a wide range of capabilities. Also, Pulsar architecture is more cloud-native, providing better support for cluster scaling and partition migration, etc. There are two layers in Pulsar architecture: the serving layer and the persistent layer. Pulsar natively supports tiered storage, where we can leverage cheaper object storage like AWS S3 to persist messages for a longer term. Over to you: which message queues have you used? – Subscribe to our weekly newsletter to get a Free System Design PDF (158 pages): https://t.co/uc5M7CdXXC