AI trial matching that a clinician will sign
Any model can rank trials. A production system has to survive a nurse disagreeing with it, a privacy officer auditing it, and a payer batching ten thousand patients through it.
Clinical trial matching is a search problem wearing a compliance costume. The ranking model is the part everyone demos. Three other things decide whether it reaches patients. Strip identity before the model call. Give a nurse the final say. Keep a record showing why each trial appeared.
What this class of system demands
4 things that decide this
- 01Strip identity before the model call, never after. Age ranges, ZIP prefixes and fake patient IDs keep health data out of the request.
- 02The model ranks and a nurse decides. Trial eligibility is a medical call, so automate the search and leave the judgement alone.
- 03Log every action. TrialTriage tracks 23 action types, so anyone can show why a trial was suggested months later.
- 04Insurers work in batches, not one patient at a time. Queues belong in the first build, not a later tidy-up.
- IntakeStructured patient facts.
- De-identifyBefore anything leaves.
- RetrieveTrials, guidelines, drug data.
- RankModel proposes, with reasons.
- ReviewA nurse adjusts and signs.
- AuditEvery step reconstructable.
Steps two and five are what separate a production system from a demo. Both are cheap to skip and expensive to retrofit.
The four problems every build hits
These show up on every trial-matching project regardless of cancer type, sponsor or country.
Eligibility criteria are prose, not data
Trial protocols are written for humans. Turning free-text inclusion and exclusion criteria into something checkable is most of the work, and it is where accuracy is won or lost.
The corpus moves under you
Trials open, close and amend constantly. A system that indexes once and never reconciles will confidently recommend a study that stopped recruiting in March.
Absence is not exclusion
A missing biomarker result is not a negative one. Treating unknown as ineligible quietly removes patients who qualify, and nobody notices because the output still looks reasonable.
Explanation is a requirement
A clinician will not sign a ranking they cannot interrogate. Each recommendation has to carry the criteria it matched and the source it came from, or review becomes rubber-stamping.
Human in the loop is a design, not a disclaimer
Most systems claiming clinician oversight present a ranked list and a confirm button. That is not review. It is consent theatre, and the reviewer's attention drifts within a fortnight.
Real review means the nurse can reorder, reject with a reason, and see which criterion drove each match. Those rejections are the most valuable data your system will ever produce, so capture them from day one.
- Let the reviewer disagree in a structured way. A free-text note is unusable feedback.
- Show the matched criterion beside the rank, not behind a click.
- Measure how often the reviewer changes the order. A rate near zero means nobody is really reading.
Systems where AI ranks and a human decides
A matching demo against a production system
Both return a ranked list of trials. Only one of them can be deployed.
| Criterion | The pilot | What production requires |
|---|---|---|
| Patient data | Sent to the model as it arrives. | De-identified before the call is made. |
| The decision | The ranking is the answer. | A clinician reviews, adjusts and finalises. |
| Trial freshness | Indexed once at build time. | Reconciled on a schedule, with closures honoured. |
| Explaining a result | The model said so. | Matched criteria and source shown per trial. |
| Volume | One patient at a time. | Batch upload with queued parallel processing. |
| Six months later | Nobody can reconstruct a recommendation. | Audit trail covers every tracked action. |
The stack this work runs on
Retrieval and models
Platform
Safeguards
What sponsors and providers ask us
01Does patient data have to reach the model at all?
No, and it should not. Structured facts can be reduced to age ranges, ZIP prefixes and a pseudo-identifier before any inference call, which is how TrialTriage was built. The model needs the clinical shape of the patient, never their identity.
02How accurate is AI trial matching?
Accuracy depends on how the eligibility criteria were encoded, not on which model you picked. Protocols are written as prose, so the parsing of inclusion and exclusion rules sets the ceiling. That is where evaluation effort belongs before launch.
03Who is accountable when a match is wrong?
The clinician who signed it, which is exactly why the review step is designed rather than bolted on. A system that lets a recommendation reach a patient without a named human approving it has moved liability somewhere nobody agreed to.
04Can insurers process eligibility in bulk?
Yes, through batch upload with queued parallel processing and progress reported in real time. TrialTriage handles insurer CSV batches this way. Designing for batch late usually means rebuilding the core, since single-patient assumptions spread everywhere.
05What happens when a trial closes recruitment?
The index has to reconcile against the source on a schedule, and closures must win over cached data. A stale recommendation is worse than no recommendation, because it costs a clinician time and costs your platform credibility on the first occurrence.
Go deeper
- TrialTriage →The full build: four user roles, 23 audit action types, insurer batches.
- RAG development →Grounding answers in your own corpus, with sources.
- How do you build HIPAA-compliant AI? →Where protected health information can and cannot go.
- Human in the loop →What the term has to mean to be worth anything.
