Human-in-the-Loop Is a Feature, Not an Apology
Teams design the approval step out as soon as they can, as if it were a stopgap. In a high-stakes flow, it is the reason the system is allowed to act.
The short version
5 things that decide this
- 01A human approval step is not evidence an AI system is unfinished. In a high-stakes flow, it is what makes automation acceptable at all.
- 02Trading CoPilot sends every trade alert to the trader over messaging and waits for a yes before it executes anything through the broker.
- 03TrialTriage ranks clinical trial matches with an LLM, and a nurse reviews and finalizes every match before it reaches a patient.
- 04Both systems keep the human step after launch. It was not a training-wheels phase removed once the model proved itself.
- 05The design problem worth solving is not whether to ask a human. It is making the ask fast enough that people keep answering it.
Teams apologize for the approval step
A common pitch: the AI will handle everything eventually. The human review step is there for now, until the model earns enough trust to run alone. That framing treats the approval gate as a limitation to design out.
In a flow where a wrong call costs money or health, that framing is backwards. The gate is not standing in for a smarter model. It is standing in for accountability a model cannot carry. Removing it does not make the system more finished. It makes the system unable to say who is responsible when it is wrong.
Two systems that kept the gate after launch
Trading CoPilot turns TradingView alerts into trade decisions for forex traders. The AI reads an alert and messages the trader with its read on it. Nothing executes until the trader replies yes or no. Only then does the system place the order through a broker connection and log the result.
That approval step is also where the system earns its keep. Filtering rules prioritize the alerts most likely to get a yes. Weak alerts expire on their own, so the trader is not buried in noise when a real setup arrives. Gate and filtering were designed together, not bolted on after.
TrialTriage matches oncology patients to clinical trials. Large language models rank candidate trials against NCCN guidelines and drug data. A nurse reviews every ranked list, adjusts it, and finalizes the match before anything reaches a patient or an insurer. The system launched with a 23-action audit trail and TOTP multi-factor login. Every AI recommendation and every nurse decision is on the record from day one.
Neither team scoped the human step as temporary. It is load-bearing in the design. The AI is trusted to rank because ranking is not the same as deciding.
- 01Trading CoPilot: alert, human yes or no, then execution through the broker.
- 02TrialTriage: an LLM ranks trial matches; a nurse reviews and finalizes each one.
Make the ask fast, or people stop answering it
A human-in-the-loop step fails a different way than a missing one: it gets ignored. If approving takes a login, a form, and three clicks, people rubber-stamp it or let it pile up. Either way, the review stops meaning anything.
Trading CoPilot puts the decision inside a messaging app the trader already has open, so a yes or no is one reply. TrialTriage puts the ranked list in the nurse's own review screen, with adjust and finalize as the only two actions that matter. Neither system asks the human to leave their workflow to supervise the AI.
That is the actual engineering problem in human-in-the-loop design. Deciding a human should approve something is the easy half. Building the approval so a busy person keeps doing it correctly is the half that determines whether the gate holds or quietly gets skipped.
Questions this raises
01When should an AI system require human approval before acting?
Require approval wherever a wrong action costs money, health, or something else that cannot be undone by a second try. Trading CoPilot gates every trade on a trader's yes or no. TrialTriage gates every clinical trial match on a nurse's sign-off. The pattern is the same: the AI ranks or recommends, and a person with accountability decides.
02Does adding a human approval step mean the AI is not production-ready?
No. In a high-stakes flow, the approval step is what makes the AI production-ready, not what it is waiting on. Trading CoPilot and TrialTriage both launched with the human gate in place and kept it. The gate is what lets the automation run at all in a domain where mistakes are expensive.
03How do you design an approval workflow people will actually use?
Put the decision where the person already works, and reduce it to one action. Trading CoPilot sends the decision as a messaging alert with a yes or no reply. TrialTriage puts adjust and finalize in the nurse's existing review screen. An approval step that requires a separate login or a multi-click form gets rubber-stamped or ignored, which defeats the reason it exists.
