Forward deployed AI engineer: what changes when the product is a model?
A platform behaves the same on your data as on the vendor's. A model doesn't. That one fact rewrites the job.
Answered in short
4 things that decide this
- 01A forward deployed AI engineer does the same job as any forward deployed engineer, embedded in the customer's systems, with one difference: the product's behaviour isn't fixed. A model answers differently on your data than on the vendor's demo, so the engineer's first deliverable is a way to measure it, not a feature.
- 02Four things change when the product is a model. Evals replace unit tests as the definition of done. Guardrails decide what the model is allowed to do, not just what it can do. Data plumbing (retrieval, permissions, freshness) becomes most of the code. And every workflow needs a designed handoff to a person.
- 03The role exists because putting a model into a business is ambiguous work. OpenAI's Colin Jarvis says its FDEs 'typically work with more ambiguity' than solutions architects and 'write code directly on customer infrastructure.' Ambiguity is the normal state of an AI deployment, not a sign it's going badly.
- 04A business without a lab's FDE team gets the same work from an embedded AI engineer. Hashlogics' AI agent builds ship with a scored eval set the client owns, guardrails on what the agent may do, and a human handoff for the cases the model shouldn't decide alone.
Evals: the definition of done moves
On a platform, done means the feature works. On a model, done means it answers your questions correctly often enough, and you can prove it. So the engineer's first week goes on that proof: a set of real questions from your operation, with the right answers, scored every time anything changes. Prompt edits, model upgrades and new documents all get re-scored against it.
That set is the most valuable thing the engagement produces, and it should be yours. In the agent systems we build, you own the eval set and can run it without us. Without one, nobody can say whether the upgrade the vendor pushed on Tuesday made your answers better or worse, and the argument about it goes on for months.
Guardrails and handoffs: what the model may do, and when it stops
A model can do more than you want it to. Guardrails are the rules that narrow it: which tools it may call, which records it may read, which answers it must refuse, and what it does when it isn't sure. Those rules are code, written on your systems, and they're where most of an AI engineer's judgement goes.
Handoffs are the other half. Every workflow needs a designed point where a person takes over, with the context the model gathered attached. When we built an AI receptionist and parts ordering for ZhoopZhoop's auto-repair branches, the design work went into the flows: inbound answer to booking, outbound supplier call to quote comparison, and the dashboard where staff see every call and transcript. The phrasing was the easy part.
Data plumbing: where most of the code goes
A model is only as good as what it can see at the moment it answers. Retrieval, permissions and freshness are the plumbing: which documents it can read, whether it respects who's asking, and whether it's looking at last month's version. This is ordinary integration work. It's where the embedded engineer earns their place, because it can only be done inside your environment.
For PremiumAudit, the work was AI automation over insurance premium audits, which lives or dies on what the model can see in the audit file. TrialTriage matches oncology patients to clinical trials, so it needed retrieval over trial criteria that change. Both are plumbing problems first and prompting problems a distant second.
- BaselineRun the model on your real data; score it.
- PlumbRetrieval, permissions, freshness.
- GuardWhat it may do; what it must refuse.
- Hand offWhich cases go to a person, with context.
- MonitorDrift, cost, failure rate in production.
- Re-scoreEvals rerun on every change.
Only one of the six steps is about the model. The other five are why the engineer is on site.
Related questions
01Do we need a forward deployed AI engineer if we're using a vendor's agent product?+
Usually yes, or someone doing that job under another title. A vendor's agent product still has to reach your data, respect your permissions, hand off to your people and be measured on your questions. The vendor's FDE does that for its largest accounts; for everyone else it's an embedded engineer, whether from the vendor's partner network or a firm like ours.
02What's the difference between a forward deployed AI engineer and an AI engineer?+
Where they sit. An AI engineer builds the model-facing parts of a product from inside the product company. A forward deployed AI engineer builds inside the customer's environment, on the customer's data and infrastructure, and carries what they learn back to the product. The skills overlap heavily; the location and the loyalty differ.
03Who owns the eval set and the prompts?+
You should, and in our engagements you do, from the first commit. The eval set is the record of what 'correct' means for your business, and the prompts and guardrails encode your rules. If a vendor keeps those, you can't switch models, can't audit a change, and can't tell whether an upgrade helped. Ask before signing.
Related
- What is a forward deployed engineer? →The definition, the Palantir origin and the three names for one role.
- What does a forward deployed engineer actually do? →The week as it happens: data, operators, code, feedback.
- Can a small or mid-size business get a forward deployed engineer? →Yes, through an embedded senior engineer. What the engagement looks like.
- Hire forward deployed engineers →A senior engineer embedded in your operations, from a free scoping call.
- AI agents and automation →The build this page describes: evals, guardrails, handoffs, maintained after launch.
- Eval-driven development →Why the eval set comes before the prompt.
- Guardrails →What the model may do, as code.
- Agent handoff →Designing the point where a person takes over.
- Hire AI engineers →The engineers who do this work.
