Apache Kafka is a common choice for inter-service communication. Kafka facilitates the parallel processing of messages and is a good choice for log aggregation. Kafka also declares to be low latency, high throughput. However, is Kafka fast enough for many microservices applications?
When I wrote Open Source Chronicle Queue my aim was to develop a messaging framework with microsecond latencies. In this article, I will describe why Kafka does not scale in terms of throughput as easily as Chronicle Queue for microservices applications.
Visualizing Delay as a Distance
In order to illustrate the difference, let me start with an analogy. Light travels through optic fiber and copper at about two-thirds the speed of light in a vacuum, so to appreciate very short delays, they can be visualized as the distance a signal can travel in the time. This can really matter when you have machines in different data centers.