My orchestrator session does not wait for Claude Code's auto-compact. It rotates first.
At 500,000 tokens a guard hook stops new work and asks for a handover. A fresh session picks up from that file.
I built rotation because context compaction kept losing the nuance I needed.
On 5 October a paper from Microsoft Research and collaborators measured something close to the same idea, at a much larger scale.
Their agents compacted their history periodically and still got stuck. Fresh branches with a cleared chat and the same files got them out.
The headline number is 66% better at equal compute. Neither the clearing nor the branching delivers that on its own.
What the paper measured
The paper is Fork-and-Flush: Escaping Idea Basins in Autoresearch Agents.
The agents ran open-ended optimization tasks with an automatic score, for hours and sometimes more than a hundred hours.
The authors started with a simple observation.
Eight independent runs of the same agent on the same task ended between 68.3 and 100.0, and the gaps stayed wide with more compute.
Each run settled into a small region of the solution space and kept circling it.
They call that an idea basin. More time in the same basin produced variations on the same idea.
The fix they tested is mechanical. At 33% and 66% of the budget the agent splits into four branches.
Each branch gets a copy of the workspace and an empty chat history. After a fixed horizon the best-scoring branch continues and the rest are dropped.
On 13 tasks the normalized score went from 0.47 for a single run to 0.78 for Fork-and-Flush.
That is the 66%. Against best-of-four independent runs (0.54) the gain was 44%.
Context compaction is not a reset
This is the part that matters for anyone running long agent sessions.
The agents in the experiment were not running with raw, ever-growing context. They compacted.
The paper says it in one line: "this localization persists despite periodic compaction of the agents' conversational histories."
A summary carries the agent's earlier reasoning forward. That includes the approach that is not working.
The authors put it plainly: a branch that kept the history "would carry the parent's reasoning that tends to re-enter the same local basin."
Two other results point the same way.
The Complexity Trap from JetBrains Research found that LLM summarization made SWE-agent runs about 15% longer than simply masking old tool output: 52 turns against 44 with Gemini 2.5 Flash.
Their explanation: summaries "act as a reinforcing signal, encouraging the agent to keep going." They smooth over the failure signals that would otherwise end a dead attempt.
And The Illusion of Diminishing Returns showed that models become more likely to make mistakes when their own earlier mistakes sit in the context.
Scaling the model did not remove that effect.
Anthropic wrote the conclusion down last year in its guide to long-running agent harnesses: "compaction isn't sufficient." Their fix was a fresh session per step with a progress file and git history.
Neither the fork nor the flush does it alone
The ablation is where the headline needs care. On 11 tasks the scores were:
| Variant | Normalized score |
|---|---|
| Single run, periodic compaction | 0.45 |
| Flush only, no branches | 0.55 |
| Fork only, branches keep history | 0.57 |
| Fork and flush | 0.71 |
Clearing the chat on its own gave roughly 22% over the single run, and it scored below the single run on 5 of the 11 tasks.
Branching on its own gave a little more. The two together gave the most. The authors say it themselves: "neither mechanism alone accounts for the full effect."
That has two consequences. First, if you only rotate a single session, you get part of the gain at best, and on some tasks none.
Second, the forking needs an automatic score to pick the winner. A feature branch or a refactor without a benchmark has no such score.
The paper also does not run a head-to-head arm of summarization against clearing. Every arm compacted.
What it shows is that compaction did not prevent the basins and that clearing helped. It does not show by how much clearing beats a better summary.
And it did not help everywhere. On the LWE-rec task the single run beat Fork-and-Flush on both maximum and mean.
What a fresh branch still gets
A flushed branch does not start from nothing.
It gets the task prompt, the full workspace with every file the agent wrote, and a short notice saying what the best submission so far scored and where it lives.
That is a handover. A short one, structurally the same as what my rotation writes before a clear: the state lives in files, the reasoning does not come along.
One detail surprised me. In each round two of the four branches got an explicit instruction to try a fundamentally different approach.
Those branches won 81 of 178 rounds, against half expected by chance (p = 0.26). The instruction did not measurably change which branch won.
What this means for how I run agents
My orchestrator rotates. A guard hook measures the context from the session transcript.
At 500,000 tokens it refuses new work and asks for a handover. A fresh session then starts in a new tmux window and continues from that file.
The first version, from February, did this for every agent at 65% of the window. Claude Code's auto-compact fired around 80% back then. Current versions compact closer to the limit.
If auto-compaction wins the race, the nuance I want to keep is gone. That first pipeline is in how I fixed context rot in Claude Code.
Since late September only the orchestrator rotates. The other agents do not.
In a five-day autonomous run from 7 to 12 October on another project, the agent's context was compacted automatically 13 times.
What got lost in those 13 summaries, I did not measure.
That is the gap in my own setup, and the paper sharpens it. One of my systems resets with a handover.
The other summarizes in place. The research says the second one is the more likely to keep circling the same idea.
There is a cost argument too. The Claude Code docs note that /compact reads the whole conversation it summarizes.
With a warm prompt cache that read is cheaper, but it is never free.
/clear costs nothing. A fresh session does have to re-read what it needs, but that cost shows up in the tokens you can see.
The counterpoint deserves a place. Cognition argued last year for sharing full agent traces, because decisions made in conversation carry implicit choices.
Clear the chat and those choices are gone, unless they were written into the workspace first.
That is the rule I take from it. Before anything clears, every decision has to live in a file.
How that works across many parallel agents is in context rotation at scale.
📖 Read also: Autonomous agent patterns: five production-tested approaches for agents that run without you
A practical rule set
- Write decisions to the workspace as you go. Not only at handover time. If it is not in a file, a reset loses it.
- Rotate at a threshold instead of compacting. Clear the chat, keep the files, reload a short handover.
- Keep the handover small. Current state, best result so far, open decisions. Not the reasoning that led there.
- Branch when you have a score. If the task has an automatic metric, run a few fresh branches from the same workspace and keep the best.
- Measure what compaction drops. I have not done this yet for my own runs. It is the next measurement on my list.
Why a long session drifts in the first place is covered in context drift in a headless orchestrator.
Frequently Asked Questions
Vincent van Deth
AI Strategy & Architecture
I build production systems with AI — and I've spent the last six months figuring out what it actually takes to run them safely at scale.
My focus is AI Strategy & Architecture: designing multi-agent workflows, building governance infrastructure, and helping organisations move from AI experiments to auditable, production-grade systems. I'm the creator of VNX, an open-source governance layer for multi-agent AI that enforces human approval gates, append-only audit trails, and evidence-based task closure.
Based in the Netherlands. I write about what I build — including the failures.