Posted on

FIFO vs. LIFO: Which Queueing Strategy Is Better for Availability and Latency?

As an engineer, you probably know that server performance under heavy load is crucial for maintaining the availability and responsiveness of your services. But what happens when traffic bursts overwhelm your system? Queueing requests is a common solution, but what's the best approach: FIFO or LIFO? In this post, we'll explore both strategies through a simple simulation in Colab, allowing you to see the impact of changing parameters on system performance. Comparing the pros and cons of each approach helps to build an understanding of the trade-offs and make better calls about queueing strategies, improving your engineering skills in the process. After all, as the saying goes: "I hear and I forget, I see and I remember, I do and I understand."

Model

To compare the performance of FIFO (which processes requests in the order they are received) and LIFO (which prioritizes the most recent requests) queueing strategies, we'll build a simple model using a Client that generates requests and a Server that handles them. You can find the details in this Colab. The essential characteristics of the model are:

Posted on

Azure Service Bus Dead-letter Queues

What is the Azure Service Bus Queue?

Queues are a very common piece of infrastructure. It offers First In, First Out (FIFO) message delivery to one or more competing consumers. As the queue grows the last item added to the queue typically must wait longer to be processed. Just like someone waiting in line to order food, the first one in line gets to order first. Microsoft Azure Service Bus (ASB) is Microsoft’s premier enterprise-level messaging technology that uses this principle of FIFO.

Types of Queues:

  • Primary queue called main queue or active queue.
  • Secondary sub-queue called a dead-letter queue (DLQ). 

Main Queue

ASB queues always have two parties involved-a producer and a consumer. The producer pushes the messages into the queue, while the consumer periodically polls for messages and consumes them. The main queue holds the messages until it is consumed or moved to the dead-letter queue.

Posted on

An Overview of the Priority Queue

The priority queue is a somewhat similar data structure to the queue. The difference lies in how the elements are being processed:

  • A standard queue strictly follows the FIFO (First-In-Last-Out) principle.
  • A priority queue does not follow the FIFO principle.

In a priority queue, the elements are being removed from the queue based on their priority. This translates to the requirement that: