What is a shadow deployment?
A test environment cannot recreate the messy, real questions your users actually ask. A shadow deployment lets a new model see those exact questions before you trust it with the reply.
Shadow deployment
shadow mode
Shadow deployment is a release method that sends a copy of live traffic to a new model, in parallel with the version already live. The new version processes every request and logs its answer. Only the current version's answer ever reaches the user.
Picture a support system built on a new model. Every incoming ticket still gets answered by the current model, exactly as before. A duplicate of that same ticket is also sent to the new model, at the same time, with the same context. The new model writes a full answer. Nobody sends it. It goes into a log next to what the current model said.
That log is the entire point. Run it for a week, or a month. You end up with the new model's answer to every real ticket, side by side with the answer that actually went out. No user was ever exposed to the new version. If it had failed badly on ticket four hundred, only the log would know.
This differs from a canary deployment, which does route some real traffic to the new version and accepts the risk on that slice. Shadow mode accepts no risk at all. The cost is that it takes longer to trust the result, because you are comparing logged answers instead of watching live outcomes.
You cannot script the traffic that breaks a model
A staging environment runs the test cases someone thought to write. Real users ask things nobody wrote a test case for. A question in a second language. A request tangled up with three earlier messages. A typo that flips the meaning. A model that passes every staged test can still fail the first Tuesday it meets your actual traffic.
Shadow deployment closes that gap without betting the outcome on it. The new version sees the same volume, the same edge cases and the same odd phrasing as the version already live. It is fed identical requests. You get a real answer to "how does this actually behave" instead of a guess based on a curated test set.
We reach for this when a wrong answer has a real cost. A system that approves a transaction. A case that touches a refund. A flagged compliance exception. In those cases a canary's small live slice is still too much exposure. The shadow log has to prove the new model out first.
There is a trade-off in time and infrastructure. Every shadowed request runs twice, so compute cost roughly doubles for the test's duration. Someone still has to build the pipeline that scores the new answer against the old one at volume. A shadow deployment nobody scores is just an expensive log file.
- Request arrivesOne real request from one user.
- Copy to new versionSent in parallel, same input, same context.
- Current version repliesThis is the only answer the user ever sees.
- New version logs its answerRecorded, never returned to the user.
- Score the pairCompare against the eval set, off the live path.
Both versions see the same request. Only one of them is ever allowed to answer it.
Common questions
01What is the difference between shadow deployment and canary deployment?
Shadow deployment sends a copy of every request to the new version but never lets its answer reach a user, so the risk is zero. Canary deployment routes a small slice of real traffic to the new version and lets its answer go out. That accepts a limited amount of real risk. Shadow mode answers "what would this version have done". Canary answers "how does this version perform when it actually has to."
02How long should a shadow deployment run before you trust it?
Long enough to see the traffic patterns that matter for your product: a full weekly cycle at minimum. Add a full monthly cycle if volume varies by time of month. The right length is set by coverage of your actual request variety, not a fixed calendar number.
03Does shadow deployment work for a full application, or only a single model call?
It works at either scope. You can shadow one model call inside a larger pipeline and compare just that step's output. Or shadow an entire new service end to end and compare final results. Narrower scope isolates one change; full-pipeline shadowing catches interactions a single-call test would miss.
Related terms
- Canary deployment →The next step after shadow mode: routing a small, real slice of live traffic instead of a logged copy.
- MLOps consulting →Where we build the shadow pipeline, the scoring rule and the rollout that follows it.
- How do you know if an AI agent is production ready? →The wider checklist a shadow deployment feeds into before an agent takes over a real workflow.
