AI that still works in month six
Senior engineers who have taken agents, retrieval and automation past the demo, and who stay on to keep them running.
What you get
4 things that decide this
- 01Senior engineers only. Every one has an AI system in production today, not a course certificate.
- 02You interview each engineer before they join, and you can decline any of them.
- 03They work in your repository, your standups and your review process. Your team approves their pull requests.
- 04They stay after launch under an agreed service level, or hand over with the evals and the runbook and train your people.
What an AI engineer actually does here
Almost none of it is prompt writing. The job is deciding what the model must never be trusted to do alone, then building the structure that holds it. Retrieval that returns the right passage. Tools it cannot misuse. Tests that catch a bad change before a customer does.
Go4Gr8 is a fair example. Executives get an AI sparring partner for hard decisions, and the interesting engineering is not the chat. It is commitment tracking over MCP, multi-tenant onboarding, and a real-time layer on FastAPI and React that has to behave when two people talk at once.
The second half of the job starts at launch. Models get deprecated, documents change format without warning, and costs move. Someone has to notice.
- 01Retrieval design, so the model reads the right thing rather than more things.
- 02Tool and agent design, including what the model is not allowed to call.
- 03Evaluation harnesses, so a prompt change becomes a measurable event.
- 04Cost and latency control: caching, model routing, and knowing when a smaller model is correct.
The work these engineers take on
Agents that act
Systems that call tools and change state. The design question is what happens when step four fails after step three already wrote to your database.
Retrieval over your own data
Answers grounded in your documents, with permissions applied per user, so nobody retrieves a file they cannot open.
Document and workflow automation
Extraction and validation from messy real files, where the failure mode is confident nonsense rather than an error message.
The layer around all of it
Evals, tracing, cost monitoring and rollback. The part teams skip, and the reason month six goes badly.
- RetrieveThe right passage, measured.
- ConstrainTools it may and may not call.
- EvaluateCases drawn from real failures.
- ObserveCost, latency, what it read.
- RecoverRollback, and a deprecation plan.
A demo needs the first box. A production system needs all five, and the last two are what nobody budgets for.
What happens when the model is deprecated
Every model behind a production system gets retired eventually. Prompts behave differently across versions, so the swap is not a config change. Ask any candidate how they would handle it, and listen for whether they have lived through one.
A good answer names an evaluation suite: real cases, run against the new model, with a pass mark agreed in advance. A weak answer talks about testing in general terms. This single question separates people who have shipped AI from people who have demoed it.
The stack
Models
Retrieval
Runtime
Quality
AI these engineers put in production
“They will treat your vision like their own and build it that way.”
Ron Klabunde · Founder, SmartREI ↗
From first call to first commit
- 01
Scoping call
Free. We work out what you are building and whether we have the right person. If your problem does not need AI, we say so.
- 02
You interview them
You meet the engineer who would do the work, and you decide. Nobody joins your project sight unseen.
- 03
They embed
Your repository, your standups, your definition of done. Reporting goes to your lead, not to an account manager.
- 04
Nobody disappears at launch
You pick an agreed service level, or a handover with the evals and runbook written and your team trained on them.
Questions, answered
01How does this compare to a marketplace like Toptal or Upwork?
You get a senior engineer backed by a team and a written standard. Freelance work stops when the freelancer moves on. Our engineers work to the same production standard on every project, so if one becomes unavailable another picks it up with the documentation already there.
02Do we interview them ourselves?
Yes, and we would worry about a vendor who said otherwise. You meet the specific engineer before they start and you can decline any of them for any reason.
03What happens if we want to change the engineer mid-project?
Tell us and we replace them. The interview makes it unlikely, but a bad match is our problem to fix rather than a contract you are stuck inside.
04Which time zones do they work?
At least four hours of overlap with your working day, agreed before anyone starts. Most clients take a longer overlap so standups and code review happen live.
05Who owns the code, the prompts and the models?
You do, from the first commit. Source code, prompts, fine-tuned weights and data pipelines are yours. Nothing is licensed back to us and nothing needs us to keep running.
06Can they work with our existing engineers?
That is the normal arrangement. They join your repository and your review process, and your engineers review their pull requests. Embedding beats a black box that returns a finished build nobody in your team can maintain.
07Which models do you build on?
Whichever fits the problem. Most of our production AI work uses the Claude API or the OpenAI API, and several systems use both. Model choice is an engineering decision, not a loyalty we bring with us.
08What does it cost?
Scoping is free and you get the scope in writing before you commit. Where we have to work inside an existing codebase first, a paid two-week diagnostic sets a fixed price for what follows.
