Hi there👋

This is mingyan. Welcome to my blog. I am a system lover and will document what I learn in my blog.

Kafka High-Throughput Architecture: Demystifying the Reactor Pattern (Part 2)

This is part 2 of the Kafka series. In this part, we will dive into the Reactor pattern and how Kafka implements it. In the next following series, we will dive into each component of the Reactor pattern. The source code referenced in this article uses Kafka’s trunk branch. I’ve pushed it to my personal repo for version alignment with this article: https://github.com/cyrilwongmy/kafka-src-reading What is the Reactor Pattern? (Readers familiar with the Reactor pattern can skip to the Kafka’s Reactor Implementation section) ...

Kafka’s TCP Unpacking: A Production-Grade Code Walkthrough (Part 1)

This is part 1 of the Kafka series. The Kafka series is a series of blog posts that will introduce the design and implementation details of Kafka. The source code referenced in this article uses Kafka’s trunk branch. I’ve pushed it to my personal repo for version alignment with this article: https://github.com/cyrilwongmy/kafka-src-reading TCP and Packet Fragmentation Why Does TCP Require Packet Fragmentation? Kafka uses TCP as its underlying transport layer protocol. TCP operates as a continuous byte stream without inherent message boundaries. Therefore, any application using TCP must implement its own mechanism to delineate message boundaries. ...