s01e02: Last-In, First-Served - When a Single LIFO Queue Starved the VIP User
Execution Contexts: Navigating the LIFO logic behind your code.
LIFO Stacks: From Code Execution Efficiency to A Restaurant Disaster
Cody, bartender: “After the fiasco we had on our lunch Soft Opening, Chef Julia learned that if she’s not careful with her Base Case, her Recursive Lasagna blows up the kitchen. But… ‘Lessons learned’, said our boss, Nina Glamour.
Now it was time for the Grand Opening evening. Julia had refined her lasagna recipe and was ready for action, and Nina was awaiting the arrival of the man who could eat us all in a single byte! A tycoon with a wallet as deep as a black hole! A man that you don’t serve, but you survive…
And again… it all went wrong…”
Deep Dive: When LIFO Ruins the Party (Stack vs. Queue).
In episode 1 we understood how the Stack works and why it uses LIFO principle (Last-in, First-out stack). We explained briefly why FIFO Stack is lightning-fast compared to the Heap memory. But in today’s comic, LIFO principle failed our kitchen. Jack Falcone was suffering from Resource Starvation. In LIFO system, the newest items get all the attention, while Jack’s order sitting at the bottom, was getting dust. If the stack never clears, the VIP orders might never get processed. “Problem-o?!”
What LIFO is made for?
LIFO is very good for code flow execution as programs by themselves are practically a set of nested instructions (functions) and the execution context navigates deeply through these functions until they return, before it continue with the “parent” context all the way back to the main() block. In other words - the program is like a lasagna eaten layer by layer at runtime.
But there are also other practical examples of LIFO serving you nice dishes, and when it doesn’t and FIFO queues would be better. And there are also situations where they both will make your seafood risotto stink.
🎁 Follows a «FREE PREVIEW» of PAID sections of our 8bytes episodes!
The “Undo” Use Case
LIFO fits great if you need to develop “Undo” functionality for your text editor app, managing the text history in a push/pop style. (Substack guys, are you doing this like that?). Caveat - the REDO functionality will require own separate stack.
The Depth-First Search (DFS) Use Case
LIFO is good when you need to navigate the maze! Many AI solutions still use this technique to search through tree structures. Think of DFS as a reckless explorer in a dark forest. Instead of looking at every path nearby, you pick one direction and start digging deeply into the tree structure. But you aren’t lost as you are maintaining a LIFO stack of “path” addresses by dropping breadcrumbs along the road (remember Hansel & Gretel fairy tale?!).
Each time you hit a fork on the road, you push the alternative path you take onto your stack. When reaching a dead end (a leaf node on the tree structure), you simply start to pop down the stack using the “breadcrumb” pointers and lookup the next unexplored path to continue with. Eventually you will get out of the maze or find whatever data you were searching for, without ultimately reading the whole data tree.
The “Latest Value Matters the Most” Use Case
In high-performance trading (HPT) software, the fresh data is the most important one. Imagine that at noon the stock price bids are flooding you system at 10,000 updates per second. If your engine lags and it uses FIFO queue, then it would process “old news” prices (stale data) from a second ago And that is a lifetime in trading! By using LIFO stack approach, it will prioritize the most recent prices, while the older data will still remain at the bottom of the stack for historical analysis to be done once the system load returns to normal and the engine catches up with cleaning the stack. Also, this mechanism allows you to restore to previous price (undo) in case the latest push is deemed faulty for some reason.
When FIFO (First-in, First-out queue) excels?
Like in our restaurant, FIFO would excel in managing table orders, unless you have a very specific use case of processing last orders with priority. Actually, even in a restaurant where priority matters, FIFO still is the right solution most of the time. You just need to have priority queues, like separate queues for VIP and regular tables. Ollie was on the right track to write down VIP orders on golden tickets and regular orders on white tickets, we just needed to have a ticket rails system (queue) instead of a ticket spike (stack) and we needed two rails to manage priority queues.
In programming, if LIFO is the bread, FIFO is the butter of data structures.
Message Brokering Use Case
In distributed systems, FIFO is the standard way to handle communication between microservices. Decoupling microservices from each other usually is achieved via asynchronous communication via message brokers like RabbitQM, Kafka, Amazan SQS, etc., which all ensure ordered processing.
Burst of Traffic Use Case
Now and then you might be in a situation when your system can experience a sudden burst of traffic. And that’s where FIFO queues come to the rescue. All WebServers for example (IIS, Tomcat, Undertow, etc.) have a limited number of worker threads that process incoming web requests - think of these like restaurant waiters. In case the load is too high and the processing is slow enough your web server will quickly run out of free threads. To not lose user requests immediately, web servers pushes these requests on a FIFO queue where they will wait for a thread to get free. Of course, to keep the memory healthy, this also has its limitation because after all we have a limited number of tables in our restaurant and at 11PM we will be closing… So if the queue size grows to big, the web server will start returning 503 - Service Not Available error immediately to any other request. (This topic I will definitely deep dive in one of my next 8bytes seasons).
Event Loops Use Case
Modern environments like Node.js or browser engines and most significantly mobile app environments for Android and iOS use an Event Loop. Events (like mouse clicks or API responses) are placed on a FIFO queue and are processed one by one. For mobile apps this is the core mechanism that ensures the UI remains responsive while handling user interactions and system events. (Also an interesting topic to go through in next seasons).
In real-world, queues are practically everywhere - in the stores checkout, in airports; in front of restaurants, night clubs, doctor's offices, etc. To craft your metaphors for explaining technology to the CEO you should just look around! Software architecture patterns are everywhere! You only need the right mental model to actually see them!
LIFO vs. FIFO: When - What?
In case you have a pile of “tickets” to process and you must decide Stack vs. Queue, you shall choose a FIFO to ensure deterministic behavior and “fairness”, ensuring that the oldest tasks are prioritized without the risk of newer tasks causing starvation. LIFO, from the other hand, is all about context and relevancy. It is the preferred solution when you value the immediate context most and when you need backtracking or state reversal.
Indexed Structures: And what if you need to eat the lasagna from the middle layers first?
In software, as in life, you eventually you are in a situation where you don’t want to read data neither top-down, nor down-to-top. Sometimes you need to taste the lasagna from the middle layers without destroying the whole dish.
The Emergency Room (Triage vs. Queue)
In the typical emergency room there is a FIFO queue - a couple of guys with broken fingers waiting for their turn. But when an ambulance brings a stroke patient and another one - a child from a car crash, the queue “disappears”. And priority queues (non-critical and critical queue) is also not that simple, as doctors will need instant access to data on every patient to perform a Triage and descide who is the most critical case.
In programming, this is where linear structures like FIFO and LIFO fail, and Indexed Structures (Arrays, HashMaps, Dictionaties) take over. They allow random access.
The Kitchen Spice Rack
Think of it like the Spice Rack in Julia’s kitchen. The spices are all lined up, but she doesn’t use the salt because it was the last thing in the stack, nor does she use the oregano first just because it’s at the front of the shelf. No! She needs to have access to any spice at any given moment, regardless of its position and after usage, the jar will be returned back on the Rack, staying there as long as someone come to clean things up.
Famous failures
Apollo 11 (1969): The “Priority Queue” vs. The Stack Overflow
During the lunar descent, the Eagle’s onboard computer (AGC) began triggering “1202” and “1201” alarms, signaling that the system was being overwhelmed with data. A faulty radar switch was flooding the computer with unnecessary data at a high rate. The computer had a limited amount of memory “sets” (essentially its execution stack). As the radar data filled up these slots, the system ran out of room to process the critical landing tasks. However, unlike Julia’s kitchen, the AGC was designed with a “priority scheduler.” It recognized the overflow, automatically killed lower-priority tasks, and cleared the stack to focus solely on the landing.
Lesson learned: In high-stakes environments, you must have a mechanism to “shed load” or prioritize your queue, otherwise, a data overflow will crash the entire mission.
Southwest Airlines (2022): The “SkyView” FIFO Meltdown
During the 2022 holiday season, Southwest Airlines collapsed, canceling 16,700+ flights. The culprit? Their legacy crew-scheduling software, SkyView. The system used a strict FIFO (First-In, First-Out) queue. When a massive storm hit, the queue was flooded with reassignments. A few complex requests at the front required manual intervention, creating a “Head-of-Line” block. Thousands of simpler, automated tasks were trapped behind them. The backlog grew until the system effectively froze, unable to “skip” the hard tasks to reach the quick wins.
Lesson learned: Strict FIFO is dangerous with variable task complexity. Without a way to prioritize or use a Dead Letter Queue, one heavy process can paralyze a global operation.
Read more: https://www.npr.org/2022/12/30/1146377342/5-things-to-know-about-southwests-disastrous-meltdown
The Architect’s CEO translation guide
Stack (LIFO): A pile of plates. You always wash the one on top unless you want to break the whole set. In programming, it's the fastest way to remember exactly where you came from.
Queue (FIFO): A line outside a club. First come, first served. Fair, but slow if there are 100 people ahead of you.
Transaction Rollback (Undo): The fine art of saying, "This never happened." Using a Stack to dump wrong moves and return to the last "clean" state.
Priority Queue: When someone pays more or is named Falcone, they skip the line. This is achieved by separate stack “rail” for each priority.
Indexed Structures: When the priority is not obvious without doing “triage” or when the items order is not strict, you should use a spice rack, that you can “see”end to end and be able to grab any item at any time.
Close Time: The Bourbon Reflection
The lights are dim in The Software Restaurant. Another day of failure was over. Cody, is wiping the mahogany bar…
Meanwhile, in the shadows of the VIP booth, a customer was still waiting for his Lasagna. There he sat - Jack Falcone! A man the whole city learned to fear before they learned to code. In the dim silence of the restaurant, the glowing tip of his cigar was the only source of warmth...
What is that all about?! You will learn in next episode when Julia will try to fix it all by implementing CACHE…
Bonus content: download the episode comic:
TBD…
☕Or buy me coffee (one-time tip)
No tip today?! No worries, just sharing my story will be most appreciated. Thanks!
Illustration credits: Comic scenes conceptualized by 8bytes and rendered by Nano Banana.