s01e02: The LIFO Logic That Starved the System

📺 When LIFO works, when FIFO is better, and when both will fail.

Jan 09, 2026

♨Last-In, First-Served

Comic ‣ Deep-Dive ‣ Real Failures ‣ Takeaways ‣ Downloads

Cody, bartender: “After the fiasco we had on our lunch Soft Opening, Chef Julia learned that if she’s not careful with her Base Case, her Recursive Lasagna blows up the kitchen. But… ‘Lessons learned’, said our boss, Nina Glamour.
Now it was time for the Grand Opening evening. Julia had refined her lasagna recipe and was ready for action, and Nina was awaiting the arrival of the man who could eat us all in a single byte! A tycoon with a wallet as deep as a black hole! A man that you don’t serve, but you survive…
And again… it all went wrong…”

Steal These Now (Before Going Deep)

For Engineers: LIFO = starvation risk. If your queue never empties, bottom items wait forever. Check queue depth metrics - if oldest item age > 10min, you have a Jack Falcone problem.
🡆 More for you? Deep Dive section awaits below.

For Tech Leads: When reviewing queue design, ask two questions: “Can old tasks starve?” (LIFO problem) and “What happens if one task fails 50 times?” (Poison Pill). No answers = no merge. Southwest paid $800M for missing the second one.
🡆 More for you? Design Review Questions in the middle.

For Managers: When the CTO says “resource starvation in LIFO queue”, translate: “VIP orders sit at the bottom while new orders keep cutting in line. Jack waited 2 hours because the kitchen serves newest tickets first.”
🡆 More for you? CEO Recipe near the end.

(Jack’s golden ticket disaster + Julia’s hollandaise explain why this matters...)

What Just Happened with Jack’s Order?

LIFO looked perfect for program execution on paper.

But in restaurant service? Jack’s VIP order sat at the bottom while newer tickets kept piling on top. Resource Starvation!

Welcome to Episode 2 of Season 1 “Memory Management”

8 episodes exploring Stack, Heap, Garbage Collection, Memory Leaks and other hilarious software disasters. New episodes drop bi-weekly:

✅ E01: Stack Overflow & Recursion → [Read now]
✅ E02: LIFO vs FIFO Starvation → You are here
✅ E03: Cache & Stale Data → [Read now]
✅ E04: Heap Fragmentation → [Read now]
✅ E05: Garbage Collection - “Stop the World” → [Read now]
⏳ E06: Memory Leaks (drops Mar 06)
📅 E07: Race Conditions (drops Mar 20)
📅 E08: Deadlocks (drops Apr 03)

Deep Dive: When LIFO works, when FIFO is better, and when both will fail the system

In episode 1 we understood how the Stack works and why it uses LIFO principle (Last-in, First-out stack). We explained briefly why the Stack is lightning-fast compared to the Heap memory. But in today’s comic, LIFO principle failed our kitchen. Jack Falcone was suffering from Resource Starvation (when a task waits forever because newer tasks keep cutting in line). In LIFO system, the newest items get all the attention, while Jack’s order sitting at the bottom, was getting dust. If the stack never clears, the VIP orders might never get processed. “Problem-o?!”

What LIFO is made for?

LIFO is very good for code flow execution as programs by themselves are practically a set of nested instructions (functions) and the execution context navigates deeply through these functions until they return, before it continue with the “parent” context all the way back to the main() block. In other words - the program is like a lasagna eaten layer by layer at runtime.

But there are also other practical examples of LIFO serving you nice dishes, and when it doesn’t and FIFO queues would be better.

And there are also situations where both LIFO & FIFO will make your seafood risotto stink.

The “Undo” Use Case

LIFO fits great if you need to develop “Undo” functionality for your text editor app, managing the text history in a push/pop style. (Substack guys, are you doing this like that?). Caveat - the REDO functionality will require own separate stack.

UNDO/REDO feature implementation via Stacks (PSEUDO CODE). — UNDO/REDO pseudo code (empty stack checks omitted for simplicity).

The Depth-First Search (DFS) Use Case

LIFO is perfect for navigating mazes. Many AI solutions still use this technique to search through tree structures. Think of DFS as exploring a dark forest. You don’t check every nearby path - you pick one direction and go DEEP.

But you’re not lost. You’re dropping breadcrumbs (remember Hansel & Gretel?). Each breadcrumb is a pointer in your LIFO stack - marking every fork where you chose left instead of right.

Hit a dead end? Pop your stack. Follow the breadcrumbs backward to the last fork. Try the path you didn’t take. Repeat until you find what you’re looking for - or exhaust all possibilities.

The “Latest Value Matters the Most” Use Case - The Stock Ticker Board

Imagine Nina installed a stock ticker in the restaurant (she’s fancy like that). Stock prices update constantly - dozens of times per second. If Cody used FIFO to display them, he’d show prices from 10 seconds ago - ancient history in trading terms! By using LIFO, he always shows the freshest price at the top. The old prices? Still in the stack for anyone who wants to see the trend later.

When FIFO (First-in, First-out queue) excels?

Like in our restaurant, FIFO would excel in managing table orders, unless you have a very specific use case of processing last orders with priority. Actually, even in a restaurant where priority matters, FIFO still is the right solution most of the time. You just need to have priority queues, like separate queues for VIP and regular tables. Ollie was on the right track to write down VIP orders on golden tickets and regular orders on white tickets, we just needed to have a ticket rails system (queue) instead of a ticket spike (stack) and we needed two rails to manage priority queues.

In programming, if LIFO is the bread, FIFO is the butter of data structures.

Message Brokering Use Case

In distributed systems, FIFO is how microservices talk to each other without getting tangled. Message brokers like RabbitMQ, Kafka, AWS SQS ensure ordered processing - first message in, first message out. Simple, reliable, scalable.

Burst of Traffic Use Case

Web servers have a limited number of worker threads that process incoming requests - think of them as restaurant waiters. If you have 10 waiters, you can handle 10 tables at the same time, no more!

Sudden traffic spike? All threads busy processing? New requests get pushed onto a FIFO queue where they wait for a thread to free up. Fair and orderly - first come, first served.

But the queue has limits. You can’t hold 1000 people waiting in line (API requests towards your system) - you don’t have infinite space, and at 11PM you’re closing anyway. If the queue overflows, your web server starts returning 503 - Service Unavailable to new requests. Better than crashing entirely, but your users still get rejected.

(Deep dive on thread pools and load handling coming in a future season!)

Event Loops Use Case

Node.js, browsers, mobile apps - they all use Event Loops. User clicks a button? That’s an event in the FIFO queue. API response arrives? Another event. They get processed one by one, keeping the UI responsive.

(Also future season material - it’s fascinating!)

So FIFO sounds perfect, right? Fair, orderly, predictable. Yeah, BUT …

The Poison Pill: When One Bad Task Kills the Whole Queue

FIFO seems fair and deterministic, right? First come, first served. But there’s a dark side to this fairness: the Poison Pill.

The Hollandaise Disaster

Imagine Jack Falcone orders Eggs Benedict, which requires a perfect hollandaise sauce. Julia starts making it, but the eggs curdle. She tries again. They curdle again. And again. But the eggs are not fresh and that’s a problem for making a good hollandaise. So, Julia’s stuck in a loop, desperately trying to complete this ONE order.

Meanwhile, behind Jack’s golden ticket on the rail, there might be 47 regular orders - simple pasta dishes, salads, things that would take 2 minutes each. But Julia can’t move on! The FIFO queue demands she finish the current task before touching the next one.

This is the Poison Pill - a single task that fails repeatedly, blocking every other task in the queue from being processed. In strict FIFO systems without proper handling, this one bad request can paralyze your entire system.

In software, poison pills show up as:

A corrupted message in a message queue that causes the parser to crash
A database query that times out every time, blocking all queries behind it
An API call to a dead external service that keeps retrying
An email with malformed encoding that crashes the mail processor

The tragedy? While Julia battles her hollandaise, she could have served 47 happy customers. The poison pill doesn’t just fail - it starves everyone behind it.

The Dead Letter Queue (DLQ): The Quarantine Solution

This is where the Dead Letter Queue comes to the rescue. Think of it as Nina Glamour's surprisingly brilliant solution:

Nina: “Julia! If you try an order 3 times and it still fails, STOP. Put that golden ticket in the Problem Orders Box (DLQ) and move to the next customer. We’ll figure out Jack’s hollandaise later when we’re not drowning.”

How DLQ works

The Benefits

The queue keeps flowing - Julia moves on to serve the 47 people waiting.
The problem is preserved - Jack’s order isn’t lost, just quarantined.
You can debug later - Engineers can analyze what went wrong without time pressure.
Manual intervention is possible - Someone can fix the issue (get fresh eggs!) and retry.

DLQ in Practice

Most modern message brokers (RabbitMQ, AWS SQS, Azure Service Bus) implement DLQ automatically:

Set a max retry count (e.g., 3 attempts)
Set a visibility timeout (how long before retrying)
Failed messages automatically move to DLQ
Alerts notify engineers: “Check the DLQ!”

Without DLQ: One bad message = entire queue blocked = system down With DLQ: One bad message = isolated = system keeps running.

LIFO vs. FIFO: When - What?

In case you have a pile of “tickets” to process and you must decide Stack vs. Queue, you shall choose a FIFO to ensure deterministic behavior and “fairness”, ensuring that the oldest tasks are prioritized without the risk of newer tasks causing starvation. LIFO, from the other hand, is all about context and relevancy. It is the preferred solution when you value the immediate context most and when you need backtracking or state reversal.

Indexed Structures: When You Need Random Access

Sometimes you can’t process top-down or bottom-up. Sometimes you need to grab lasagna bite from the middle without destroying the whole structure.

The Emergency Room (Triage vs. Queue)

In the typical emergency room there is a FIFO queue - a couple of guys with broken fingers waiting for their turn. But when an ambulance brings a stroke patient and another one - a child from a car crash, the queue “disappears”. And priority queues (non-critical and critical queue) is also not that simple, as doctors will need instant access to data on every patient to perform a Triage and decide who is the most critical case.

In programming, this is where linear structures like FIFO and LIFO fail, and Indexed Structures (Arrays, HashMaps, Dictionaries) take over. They allow random access.

The Kitchen Spice Rack

Think of it like the Spice Rack in Julia’s kitchen. The spices are all lined up, but she doesn’t use the salt because it was the last thing in the stack, nor does she use the oregano first just because it’s at the front of the shelf. No! She needs to have access to any spice at any given moment, regardless of its position and after usage, the jar will be returned back on the Rack, staying there as long as someone come to clean things up.

Quick tips:

🔧 Production Pattern: Monitor queue age per message. Alert when oldest item > 5min (LIFO starvation warning). Set retry limit = 3 and implement DLQ for all FIFO queues. RabbitMQ, SQS, and Kafka support this out of the box - use it.

💼 Business Decision Alert: Queue choice isn’t “developer preference” - it’s business risk. LIFO without priority = VIP customers wait forever. FIFO without DLQ = one broken request kills the system. Both scenarios = revenue loss.

Famous Failures

Apollo 11 (1969): The “Priority Queue” vs. The Stack Overflow

During the lunar descent, alarms started screaming: “1202” and “1201” - the Eagle’s computer was drowning in data.

The culprit? A faulty radar switch flooding the system with garbage data at high speed. The computer had limited memory “sets” (its execution stack), and the radar spam was eating them all. No room left for the critical task: landing on the damn moon.

But unlike Julia’s kitchen, the AGC was designed with a priority scheduler. It recognized the overflow, killed the low-priority junk, and cleared the stack. Only the landing tasks survived.

Result? Touchdown. Armstrong walked. Julia should’ve had one of those.

Lesson learned: In high-stakes environments, you need a way to shed load or prioritize ruthlessly. Otherwise, one data flood crashes the entire mission.

Southwest Airlines (2022): The “SkyView” Poison Pill Meltdown

During the 2022 holiday season, Southwest Airlines collapsed, canceling 16,700+ flights. The culprit? Their legacy crew-scheduling software, SkyView, and a textbook Poison Pill scenario.

The system used a strict FIFO queue for crew reassignments. When a massive storm hit, the queue was flooded. A few complex requests at the front required manual intervention - like Julia’s curdling hollandaise. These poison pill tasks kept failing and retrying.

Thousands of simpler, automated tasks were trapped behind them. The system had no Dead Letter Queue - no way to say “skip this nightmare and serve the easy ones.” The backlog grew until the system effectively froze.

Lesson learned: Always implement retry limits and quarantine mechanisms. Strict FIFO without DLQ is a ticking time bomb. One complex task can paralyze a global operation.

Quick tips:

👨‍💼 For Your Next Board Meeting: “Apollo 11 had priority queues and survived overflow. Southwest had FIFO without DLQ and lost $800M. Our queues need both: priority handling for critical tasks AND poison pill quarantine. Non-negotiable.”

👨‍🎓 Teaching Moment: Use Jack’s golden ticket story. “LIFO = newest on top, VIP waits at bottom. FIFO = fair line, but one broken order blocks everyone. Solution? Priority FIFO queues + Dead Letter Queue for failures.”

Key Takeaways

The CEO-Digestible Recipe

When the CTO says “FIFO queue blocked by poison pill” or “LIFO causing resource starvation”, translate it like this:

♨ LIFO (Last-In, First-Out) - Stack where newest item processed first. Great for code execution, terrible for fair service.

♨ FIFO (First-In, First-Out) - Queue where oldest item processed first. Fair and orderly. Perfect for message brokers, web requests.

♨ RESOURCE STARVATION - Task waits forever because newer tasks keep cutting in line. Jack’s VIP order at bottom of LIFO spike.

♨ POISON PILL - Single task that fails repeatedly, blocking everything behind it in FIFO queue. Julia’s hollandaise loop.

♨ DEAD LETTER QUEUE (DLQ) - Quarantine box for failed messages. After 3 retries, move to DLQ and keep processing.

♨ PRIORITY QUEUE - Separate queues for different priority levels. Apollo 11 had it, survived overflow.

♨ INDEXED STRUCTURES - Arrays, HashMaps for random access. The spice rack - grab what you need, when needed.

♨ BACKTRACKING - LIFO technique for exploring paths. Go deep, hit dead end, pop stack, try another path.

The Three Rules of Queue Management (Remember These)

1. Never use LIFO for user-facing requests.
Bottom items starve. Always use FIFO or priority queues for fairness.

2. Always implement Dead Letter Queue with retry limits.
Set max retry = 3. One poison pill shouldn’t kill entire system (Southwest: $800M lesson).

3. FIFO + priority lanes + DLQ covers 90% of scenarios.
LIFO is for execution stacks and undo buffers, not service queues.

For Engineers: Queue Failure Detection Patterns

Watch for these patterns in production:

Pattern 1: Silent Starvation

Symptom: Some requests never complete despite system appearing healthy

Diagnosis: LIFO queue with constant new arrivals, old items never reached

Fix: Switch to FIFO or add age-based priority (promote items older than 10 minutes)

Pattern 2: The Poison Pill Freeze

Symptom: Queue processing stops completely, backlog grows

Diagnosis: Single failing message blocking FIFO queue, retrying forever

Fix: Implement DLQ with maxRetries=3, visibilityTimeout=30s

Pattern 3: Thundering Herd on Priority

Symptom: All high-priority items arrive simultaneously, starving regular queue

Diagnosis: Priority queue without fair scheduling (Apollo 11 almost hit this)

Fix: Weighted round-robin between priorities (process 3 high, then 1 normal)

Metrics to monitor:

Queue depth per priority level (track separately)
Oldest item age (alert at 5 min warning, 10 min critical)
DLQ message count (should be near zero)
Retry count distribution (spike at max = poison pills)

For Tech Leads: Design Review Questions

Before approving queue architecture, require answers to:

1. Is this queue user-facing?
User-facing = must be FIFO. Internal execution context = LIFO acceptable.

2. Do we have DLQ configured?
Require: maxRetries value, DLQ destination, monitoring/alerting setup.

3. Can one message failure block others?
If yes, you have poison pill vulnerability. Need isolation strategy.

4. Do we monitor oldest item age?
Starvation is silent. Age-based alerts catch it before customer complaints.

5. What happens during traffic spike?
Does queue overflow reject new items (good) or crash entire system (bad)?

Team Practice: Quarterly “Queue Failure Drills”

Simulate:

Poison pill message that fails every retry attempt
Traffic spike that fills queue to capacity
Priority inversion (low priority blocks high priority)

Measure recovery time and customer impact.

For Managers: Investment Priorities

What queue failures look like in business terms:

Southwest Airlines (2022): 16,700+ flights canceled. FIFO crew scheduling without DLQ. Poison pill tasks froze the system.

Apollo 11 (1969): Alarms during moon landing. Priority scheduler saved mission by shedding low-priority tasks.

Common pattern: Queue design determines whether systems survive or fail under stress.

Questions to ask your CTO:

Do we use LIFO for any customer-facing queues?
Is DLQ implemented on all FIFO queues?
What’s our oldest queue item age right now?
How do we handle poison pill messages?
When did we last test queue overflow scenarios?

Budget Allocation:

Invest in queue reliability:

Queue monitoring - Age, depth, DLQ metrics across all services
DLQ infrastructure - Automated setup, monitoring, alerting
Load testing - Traffic spike scenarios, poison pill injection
Priority scheduling - Weighted round-robin, starvation prevention

The decision: Southwest lost $800M. Apollo 11 survived with priority queues. Proper queue design prevents disasters.

Bonus content: download the episode comic:

8bytes S01e02 (full Comic Format)

5.88MB ∙ PDF file

Download

8bytes S01e02 (short Comic Format)

4.05MB ∙ PDF file

Download

8bytes S01e02 (slideshow Format)

1.35MB ∙ PDF file

Download

Perfect for: Presentations, onboarding docs, tech talks

Using it? Reply and tell me!

Close Time: The Bourbon Reflection

The lights are dim in The Software Restaurant. Another day of failure was over. Cody, is wiping the mahogany bar…

Episode 3 Drops in Two Weeks

When Fast Isn’t Fast Enough: Cache, Speed, and the Stale Data Trap

Julia is NOT happy with her perforamance. She has a brilliant idea of precooking a lot of lasagna and keep it warm for the dinner. But a news announcement will turn it all to Stale Data…

First time here? Start from the beginning: