What is a system prompt?
A support bot answers a refund question correctly in testing. Then it invents a return policy for a real customer, and nobody can point to the line that let it happen.
System prompt
system message
A system prompt is the set of instructions given to a language model before a conversation begins. It fixes the model's role, its rules and what it should refuse to do, and the user never sees it.
Most model APIs accept a system role alongside the user's messages. Anthropic's Messages API and OpenAI's Chat Completions API both take a separate system field, sent once and applied to every turn in the conversation. It sits above the user's own words, so a well-written one holds even when the user tries to talk the model out of it.
A typical system prompt sets four things. Who the model is speaking as, what it knows or can look up, what format the answer must take, and what it must never do. A support agent's prompt might name the company and list its three allowed actions. It would also state one rule: never quote a refund amount without checking the order record first.
It behaves like config, so treat it like config
A demo's system prompt lives in a chat window or a scratch file, edited by whoever is testing that day. Nobody notices, because a demo has one user and no memory of yesterday's wording.
A production system loses that safety net. Ten people touch the same prompt over a year, each fixing the complaint in front of them. A line added to stop the model from apologising too much quietly loosens the rule that stopped it from quoting prices. Nobody sees the connection, because the two edits happened months apart.
Version it like source code, review changes like a pull request, and run a saved set of test conversations against every edit before it ships. That is what stops a prompt from drifting one small, reasonable-looking change at a time.
- System promptRole, rules, boundaries. Set once.
- ToolsWhat the model is allowed to call.
- User messageThe actual question, turn by turn.
- Model replyShaped by all three above it.
- Eval runEvery change scored before it ships.
The system prompt is set once per session and outranks the user's own words. That is exactly why an unreviewed change to it reaches every conversation at once.
Related questions
01What is the difference between a system prompt and a user prompt?
A system prompt is set once by whoever builds the application and applies to the whole conversation. A user prompt is the message a person types in that turn. The system prompt outranks the user prompt when the two conflict, which is what stops a user from talking the model out of its rules.
02Can a user see or change the system prompt?
Not by default, and most applications hide it deliberately. A determined user can sometimes get a model to repeat or paraphrase parts of it. So a system prompt should set behaviour. It should never hold secrets like API keys or unpublished pricing.
03How long should a system prompt be?
As short as the task allows. Length itself is not the problem, but every added rule is a chance to contradict one written earlier. Test any change against a saved set of real conversations rather than judging it by how thorough it reads.
