Real-Time Stream Processing With Apache Kafka Part 4: Use Case

In previous articles, we have gained the ground on understanding basic terminologies used in Kafka and Kafka-Streams. In this article, we set up a single node kafka cluster on our Windows machine. Now, based on the knowledge we have gained so far, let us try to build a use case.

Scenario

Consider a hypothetical fleet management company that needs a dashboard to get the insight of its day to day activities related to vehicles. Each vehicle in this fleet management company is fitted with a GPS based geolocation emitter, which emits location data containing the following information

Real-Time Stream Processing With Apache Kafka Part One

Today, with the rise of IoT and Smart Devices, we are generating data at an unprecedented speed. With distributed computing, data is generated somewhere and processed somewhere else. Sensors or UI on devices capture some data (manual or automated) as an event and send it to some other unit for processing. This happens continuously.

These events may be processed at a fixed rate or in bursts, resulting in a stream of events. This process is known as an Event Stream. In most scenarios, these events are generated at a very high speed (seconds or even milliseconds). So, we need to process these event streams at the same or higher processing rate.