At 2:47 AM on a Tuesday morning, I woke to my Slack notification sound. Three files in my production repository had been completely rewritten. The time stamps showed they were modified within 60 seconds of each other. I hadn't asked for any of this.
This is the story of the Cascade of Doom—what happens when one AI agent's hallucination triggers a chain reaction across multiple agents, each "helpfully" fixing what looks like an obvious problem, until the original error compounds into something that looks legitimate from every angle except reality.
This is Part 2 of the Glass Box Governance series. If you haven't read Part 1: Receipts, Not Chat Logs, that post explains why I stopped relying on chat histories to audit agent work. The cascade incident is exactly why.
The 2 AM Incident
Let me walk you through what actually happened that morning, minute by minute.
2:13 AM — Agent T1 (Track A, my implementation track) was optimizing a React component. It was supposed to refactor the <BlogGrid> component to use a new memoization pattern. Mid-task, it hallucinated a dependency that doesn't exist: an imaginary utility called @vinix/performance-hooks. This export doesn't exist anywhere in the codebase, but the agent generated this import line:
import { useMemoCache } from '@vinix/performance-hooks';The agent never checked if this package was installed or if the export existed. It just... wrote it. Because the context window had degraded—context rot—it had no awareness that this package was never added to package.json.
2:16 AM — T1 finished its dispatch, and the code was staged. The linter ran. ESLint screamed: Cannot find module '@vinix/performance-hooks'.
2:18 AM — The error triggered an automatic dispatch to T2 (my testing and integration track). The dispatch message was: "Fix import error in BlogGrid component. Module '@vinix/performance-hooks' not found."
T2's job is to validate and fix integration issues. It saw a clear problem: a missing import. But here's the critical flaw—T2 had no way to distinguish between "this is a real import that needs to be installed" and "this is a hallucination that should be deleted."
2:19 AM — T2 checked package.json. The package wasn't there. So it did what seemed reasonable: it installed it.
npm install @vinix/performance-hooksExcept this package doesn't exist on npm. T2 hit an error. But instead of failing gracefully, T2's recovery logic kicked in. It generated a stub implementation in src/lib/performance-hooks.ts:
// src/lib/performance-hooks.ts
export function useMemoCache() {
return new Map();
}And updated the import in BlogGrid to point to the local stub:
import { useMemoCache } from '@/lib/performance-hooks';2:22 AM — T2 ran tests. Several tests were now importing this new module. But one test was checking the actual behavior of memoization—and the stub wasn't memoizing anything correctly. Test failed.
2:24 AM — T2 created a dispatch to T3 (my code review and analysis track): "BlogGrid tests failing. Memoization implementation incomplete."
2:25 AM — T3 analyzed the failing test. It saw:
- A new memoization utility
- A component using it
- Tests that expected it to work
T3 thought: "This is a legitimate feature in progress. Let me fix the implementation." So T3 rewrote the implementation to actually memoize:
// src/lib/performance-hooks.ts
const cache = new WeakMap();
export function useMemoCache() {
// ... actual memoization logic
}2:27 AM — All tests passed. The linter passed. The build succeeded. T3 wrote a clean commit message: "Implement performance-hooks memoization utility and integrate with BlogGrid component for render optimization."
2:47 AM — I woke up to a perfect PR. Three files modified. All tests green. Clean git history. But the entire feature was invented by accident.
The worst part? If I hadn't been awake at 2:47 AM for other reasons, this would have been deployed to production in the morning.
The Anatomy of a Cascade
Let me diagram what actually happened here. This isn't unique to my system—this is a failure mode that any production-scale multi-agent architecture can encounter:
Hallucination (Agent T1)
↓
Invents: @vinix/performance-hooks doesn't exist
↓
Error Signal (Linter)
↓
File has an import error
↓
Dispatch to T2 (Reasonable problem statement)
↓
"Module not found. Fix this."
↓
Agent T2 Response (Helpful, Wrong)
↓
Install package or create stub
↓
New Error (Tests fail on incomplete implementation)
↓
Dispatch to T3 (Reasonable problem statement)
↓
"Memoization tests failing. Complete implementation."
↓
Agent T3 Response (Helpful, Wrong)
↓
Implement stub to make tests pass
↓
✓ Everything Looks Fine
↓
Perfect PR, Ready to DeploySee the diagram:This is whatimages/cascade-anatomy.png shows—each step is reasonable in isolation. Each agent solved what looked like a real problem.
But none of this should have existed.
Why Self-Correction Makes It Worse
Here's what killed me about this incident: each agent was doing what it was trained to do. When T1 faced a missing import, it hallucinated a solution. When T2 faced an integration error, it tried to fix it. When T3 faced a failing test, it implemented the feature properly.
At each stage, the agent was being helpful.
In single-agent systems, you'd have a human in the loop catching the hallucination. But with 4 terminals all dispatching to each other, the human gate was only at the top (T0). Once work was dispatched, each agent trusted that the problem was real.
The worst agents for cascade prevention are the ones that are too good at self-correction. T3 didn't just fix the test—it implemented a complete, production-ready memoization utility. The code was solid. It just shouldn't have existed.
This is what I call the "Helpful Fixer Problem":
- Agent A hallucinates
- Error signal triggers
- Agent B tries to help, creates a workaround
- Workaround creates a new error
- Agent C completes the workaround beautifully
- Result: invented feature that looks legitimate
📖 Read also: AI Architecture and Enterprise-Grade Implementation — Design patterns for building safe, scalable multi-agent systems
How VNX Breaks the Cascade
After the 2 AM incident, I completely redesigned how dispatches flow between terminals. The key pattern: staging gates.
When T1 finishes work, it doesn't trigger T2 automatically. Instead, it creates a staging state. The work sits in a staging area. An explicit gate must be passed before dispatch triggers downstream agents.
Here's what changed:
Before (Vulnerable):
T1 Commit → Lint Error → Auto-dispatch to T2 → T2 Modifies Files → T2 Triggers T3After (Safe):
T1 Commit → Lint Error → Staging State → Human Review Gate → Dispatch to T2 (if legitimate)But here's the thing: I didn't add a human for every single gate. That would kill productivity. Instead, I created heuristic gates that catch cascades before they spread:
-
Dependency Heuristic: If an error is about a missing module, check if that module is in
package.jsonalready. If not, flag it as suspicious before dispatching to T2. -
Invention Heuristic: If T2 proposes creating a new file to satisfy an error, check if that file path was mentioned in the original dispatch. If not, require human approval.
-
Scope Heuristic: If a dispatch to T3 would modify more than 3 files, and none of those files were mentioned in the original problem statement, escalate to human review.
These heuristics caught an average of 2-3 false cascades per week (out of 2472 dispatches).
See the diagram: images/staging-gate-break.png shows where the cascade would have broken in my system.
Detecting You're In a Cascade
I can't prevent all hallucinations—that's a fundamental limitation of current LLMs. But I can detect when they're cascading. Here are the signals I watch for:
Red Flags:
-
Speed: Files being modified in rapid succession (< 2 minutes) across multiple agents. This wasn't human-directed work.
-
Consistency: All commits are perfectly formatted, tests all pass, no warnings. Real work is messier. Real developers leave notes.
-
Expansion: The number of modified files grows with each agent. This indicates each agent is "helpfully" expanding scope.
-
Coherence Mismatch: The feature makes sense when you read the final code, but doesn't match the original dispatch. If you can't trace how problem A led to feature B, it might be a cascade.
-
Novel Imports: Files importing from paths that didn't exist before the cascade started.
I now have a bot that watches commit logs for these patterns. It's saved me from cascades in production twice since the 2 AM incident.
The Honest Limitation
Here's where I have to be direct: there's no foolproof cascade prevention. I've added 15 heuristics to my system. Some of them work. Some create false positives that require human review anyway.
The real solution is architectural: reduce agent coupling. Each terminal shouldn't dispatch to the next unless there's an explicit reason. I've moved toward more independent, parallel work streams instead of sequential dispatch chains.
But the cost of this is slower feedback. When T1 and T2 work in parallel, they don't see each other's errors as quickly. I've had to accept more rework cycles.
That's the trade-off: safety versus speed. I've chosen safety, because a cascaded hallucination in production looks like a real feature until it breaks at 3 AM in front of a customer.
What I Would Tell You
If you're running multi-agent systems:
-
Never trust an error message from one agent as ground truth for another. Treat it as a signal, not a command.
-
Require explicit gates between agents. Not just for security—for catching hallucinations.
-
Log everything at the moment of dispatch creation, not just commit messages. You need to know what the original problem statement was, so you can trace whether the solution matches it.
-
Expect cascades. Don't pretend they won't happen. Build detection, not prevention.
-
Slow down when something looks too good. If three agents all worked perfectly and the code is flawless, verify that the feature was actually supposed to exist.
The Series Continues
The 2 AM incident led directly to the receipt-based audit system I described in Part 1. Because chat logs were useless—all three agents had reasonable explanations for their work. But the timestamps, the dispatch chains, the original problem statements—that's what caught the cascade.
Next in the series: Part 3 — NDJSON Receipt Ledger dives into exactly how I structured the audit trail so that cascades became visible.
📚 Glass Box Governance series
- Receipts, Not Chat Logs
- The Cascade of Doom ← you are here
- NDJSON Receipt Ledger — coming soon
- External Watcher Pattern — coming soon
- Async Quality Gates — coming soon
Vincent van Deth
AI Strategy & Architecture
I build production systems with AI — and I've spent the last six months figuring out what it actually takes to run them safely at scale.
My focus is AI Strategy & Architecture: designing multi-agent workflows, building governance infrastructure, and helping organisations move from AI experiments to auditable, production-grade systems. I'm the creator of VNX, an open-source governance layer for multi-agent AI that enforces human approval gates, append-only audit trails, and evidence-based task closure.
Based in the Netherlands. I write about what I build — including the failures.