What is model deprecation risk?
An email says the model you built on retires in six months. Nobody on the team can say what changes when you swap it.
Model deprecation risk
Model retirement riskModel sunset risk
Model deprecation risk is the exposure a product carries because the AI model it runs on will eventually be retired or superseded by the vendor. The risk is not the switch itself but the behaviour change: a prompt tuned for one model version can produce different output on the next.
Every major provider publishes deprecation notices and a date after which a named model stops serving requests. That part is orderly. The disorderly part is what happens inside your product when the replacement arrives.
Two things move at once. Output changes, because a newer model words things differently and sometimes reasons differently. Cost and speed move too, which can quietly break a feature built around a response time.
The migration is a testing problem, not a config change
Swapping a model is one line of code. Proving the product still works is the actual project, and teams routinely budget for the first and not the second.
What breaks is rarely dramatic. A model that used to return clean JSON starts adding a sentence before it. A classifier that sat at 94 percent lands at 89 on the categories you care about. Output that was reliably terse becomes chatty, and a downstream parser that assumed short strings starts failing.
Vendors give you one useful lever. Name a specific model version rather than a moving alias, and your build stops changing under you. You own the upgrade date instead of receiving it.
- 01Pin a named model version in production, so a vendor release never changes behaviour without your say-so.
- 02Keep the model name in one place in the codebase. Hunting it through twelve files is how a migration overruns.
- 03Record the model version in your logs, or you cannot explain why last quarter's answers differ.
- NoticeVendor names the retirement date.
- SwapOne line. The easy part.
- ScoreRun the eval set on both.
- RepairFix the prompts that moved.
- WatchTraces, for the first weeks.
A team with no eval set cannot run the middle station, so it substitutes reading a few outputs and hoping.
Common questions
01How much notice do vendors give before retiring a model?
Enough to plan, if somebody is watching for it. Major vendors publish deprecation pages listing affected models and their shutdown dates. Put one person in charge of checking those pages. Relying on an email reaching the right inbox is how a date gets missed, and the announcement is the start of a project rather than a diary note.
02Does using an open-weight model remove this risk?
It trades the risk rather than removing it. Host the weights yourself and nobody can retire them out from under you. You take on the hosting, the upgrades and the security patching instead. The upgrade pressure returns anyway, because staying three generations behind has its own cost.
03Will a framework protect us from a model change?
Only from the plumbing part. A framework that offers one interface across providers keeps your code from caring which vendor answers, which is real value. Behaviour still shifts when the model does, and only your own test cases show you where.
04Should we upgrade as soon as a new model appears?
Not automatically. A newer model is usually better on average. It can still be worse on your specific task, which is exactly what an eval set is for. Upgrade when the score says so, or when the retirement date decides for you.
