On 14 September I found out that the model I was calling no longer existed. deepseek-v4-flash still returned HTTP 200.
DeepSeek had retired DeepSeek V4 Flash four days earlier and routed the name to V4.1 Flash.
My own model registry still listed it as a 4.0 model. Nothing failed.
Every request succeeded, at a different generation than I thought, and nothing in my stack noticed.
That is the whole problem in one line. A model name is a pointer.
What sits behind it can change: the weights, the price, the host, the serving path. And the response code stays 200.
What DeepSeek did with the DeepSeek V4 Flash name
DeepSeek documents its changes, so the timeline is public. The API changelog shows the same names being re-pointed several times in a year.
On 31 July the name deepseek-v4-flash moved to a re-post-trained 0731 checkpoint.
The changelog's advice was to keep calling the same name "to use the latest version."
On 10 September the previous generation was retired.
The names deepseek-v4-flash and deepseek-v4-flash-vision-exp are now "temporarily routed to V4.1 Flash." The name for the current model is deepseek-flash.
The pricing page says the legacy names are "still accepted, but the corresponding models have been retired." Requests go to DeepSeek-V4.1-Flash and are billed at the Flash price.
None of this is hidden. It is just not visible at the place where your code lives. The API takes the old name and answers.
When I sent a name that does not exist at all, the error message listed exactly two valid names: deepseek-flash and deepseek-v4-pro.
The name I had been using was not one of them, and it still worked.
Three places the old name survived in my own stack
The registry was the first. The second was a cost ledger in one of my own dashboards.
It hard-codes deepseek-v4-flash at $0.14 per million input tokens and $0.28 per million output tokens.
What I measured on 14 September for deepseek-flash: $0.15 in and $0.60 out off-peak, $0.30 and $1.20 during peak hours.
Output tokens are booked two to four times too low. The ledger never errors. It just reports a cheaper world.
The third one surprised me most. On 29 September I tested DeepSeek's own agent harness, dsh 0.2.0-rc.2, a developer preview.
Its ACP profile defaults to deepseek-v4-flash. Its headless profile defaults to deepseek-flash.
Same release, two profiles, two default models. One of them on the retired name.
If the vendor's own tool carries both names, plenty of downstream configs will too.
Same name, different serving path
On 10 October a paying user of the Cline coding agent opened issue #15003. The model id was deepseek-v4.1-flash.
The complaint was that the agent stopped calling tools and started inventing their results.
According to the reporter, the response metadata changed between 2 October 19:17 UTC and 3 October 05:35 UTC.
The final provider went from deepseek to openai-compatible-private. Same model id, no announcement. The reporter found out by reading the responses.
The numbers in the issue are self-reported, so treat them as one account.
Leaked tool-call markup went from zero in roughly 6,000 messages between 1 and 8 October to 4.1% on 9 October, and to every step in the tail of one session on 10 October.
The interesting part is the reporter's own control. On MATH-500 the new pool scored 80.8% against 80.0% on DeepSeek's official API.
The weights look fine. What broke is the serving path: tool calls written as text instead of structured calls.
A second issue on a private pool under the same provider label, #13348, has been open since 18 August with the same class of defect.
Neither has a public reply from a maintainer at the time of writing.
Same weights, different host
This is not specific to one reseller. On routers the default is to spread traffic over whichever providers are up and cheap.
OpenRouter load balances "to maximize uptime" and allow_fallbacks is on by default.
The same open weights then behave differently per host.
One OpenRouter user published the per-provider board for DeepSeek V4 Flash 0731 in September: 81.3% on TAU-Bench Airline through DeepSeek's own API, 58.4% through another host.
Moonshot built a vendor verifier for exactly this reason.
For K2-thinking it measured 100% tool-call schema accuracy on its own API and 83% to 86% at three third-party hosts.
And a CISPA paper from May found that with weights, decoding parameters and hardware fixed, the inference backend alone can shift benchmark scores by up to 16.6 percentage points.
I ran into the reliability side of this when I built a cost-aware router across twelve lanes.
Capability and launch reliability turned out to be different axes. That story is in from benchmark to router.
📖 Read also: The real cost of AI agents in production: where the token bill of an agent setup actually comes from
Even a pinned ID moves
Pinning helps. It does not finish the job.
Anthropic's model documentation says each model ID identifies a pinned version, and then adds that "the serving infrastructure around the model can change over time."
If behaviour shifts on a previously stable ID, Anthropic names an infrastructure update as the most likely cause.
The weights are fixed. The router, the safety classifiers and the sampling logic around them are not.
OpenAI's aliases have moved before too. In October 2024 the gpt-4o alias moved to a newer dated snapshot.
OpenAI announced it in advance, and the dated snapshots exist so you can opt out of exactly that.
What I check now
- Use the current name, and a snapshot where one exists. For DeepSeek that is
deepseek-flashnow. Where a provider offers a dated snapshot, pin to it for anything with an eval baseline. - Log what served each response. The provider and the model field from the response, not the name you sent. On OpenRouter the provider is in the response.
- Turn fallbacks off where reproducibility matters. For evals and benchmarks, set the provider order and disable fallbacks. For production, keep fallbacks and measure quality per provider.
- Run a tool-call smoke test per route. A short daily check that a structured tool call comes back as a structured tool call. The failure in the Cline report would have shown up the day the leaks started.
- Keep prices in config, not in code. A price that lives in code is a price nobody updates.
- Check the balance, not the door. On the same 14 September a
/v1/modelsprobe said the API was fine. The real dispatch failed eleven seconds later on402 Insufficient Balance. A 200 on a list endpoint tells you nothing about whether work can run.
How I run several model families through one harness, and what broke along the way, is in one harness, five model families.
Why a fallback is not the same as escalating on quality is in quality escalation is not a fallback.
Frequently Asked Questions
Vincent van Deth
AI Strategy & Architecture
I build production systems with AI — and I've spent the last six months figuring out what it actually takes to run them safely at scale.
My focus is AI Strategy & Architecture: designing multi-agent workflows, building governance infrastructure, and helping organisations move from AI experiments to auditable, production-grade systems. I'm the creator of VNX, an open-source governance layer for multi-agent AI that enforces human approval gates, append-only audit trails, and evidence-based task closure.
Based in the Netherlands. I write about what I build — including the failures.