AI at a trial site: judge one workflow at a time

The mistake: buying a platform before naming a problem

Most disappointing technology purchases at research sites follow the same pattern. A team sees a polished demonstration and signs for a platform promising help with recruitment, documentation, scheduling, and data quality all at once. Six months later, nobody can say whether anything improved. The tool is used a little bit everywhere and owned by no one, and the site cannot tell whether the software failed, the training failed, or the problem was never defined well enough to solve.

The cost is not just the license. Coordinators spend hours learning the system, and the work it was meant to relieve continues in parallel because staff do not yet trust the output. A tool tested on one workflow creates a contained question: did this task get faster, more accurate, or both, without new risk? If you cannot answer that for one workflow, adding five more only makes the evaluation harder and the exit costlier.

Start with one workflow, and only one

Pick a workflow that is frequent, repetitive, and measurable, and where an error is detectable before it reaches a participant or sponsor. Three candidates fit at most sites:

• Pre-screening chart review: a first pass through the electronic health record (EHR) to flag charts that may match basic protocol criteria, so a coordinator reviews likely candidates first.
• Visit follow-up reminders: drafting routine reminders for upcoming visits, so coordinators stop spending prime morning hours on repetitive outreach.
• Coordinator documentation drafts: turning a coordinator's notes into a first draft of a visit note or follow-up summary, which the coordinator reviews, corrects, and owns.

Each has a clear start and finish, produces an output a qualified person can check, and already consumes hours you can count. None, done properly, asks the tool to make a clinical judgment. Resist piloting two or three at once because the vendor bundles them: a pilot that tests everything tests nothing.

Measure the baseline before you touch the tool

This is the step sites skip, and skipping it makes every later claim worthless. Before the pilot, spend two to four weeks measuring the workflow as it runs today. Record the staff hours it consumes per week. Record the error rate, using a definition written down in advance — for chart review, for example, the share of flagged charts that were never plausible candidates, or the share of eligible charts the current process misses. Record the cycle time.

Date those numbers and keep them. Without a baseline, a vendor can claim credit for an improvement already under way, and your team can blame a tool for a problem that predates it. With one, the conversation becomes short and factual: either hours fell or they did not; either the error rate improved or it did not; either saved time went somewhere useful, or it was absorbed by fixing the tool's output — which is not savings at all.

Run a bounded pilot, not an open-ended trial period

A pilot needs four boundaries set in advance. First, a duration: six to eight weeks usually covers normal variation in clinic volume without drifting into permanence. Second, a scope: one workflow, one study or a small set of studies, a named group of staff. Third, an owner: one person who runs the pilot and reports the numbers, not a committee that shares responsibility so loosely no one holds it. Fourth, a decision date, on which the site will keep, adjust, or drop the tool based on the scorecard below.

Document the pilot in a standard operating procedure (SOP) addendum or a short pilot plan, even if it is only two pages: what the tool may do, what it may not do, who reviews its output, and how errors are logged. That document is the difference between a controlled evaluation and an informal experiment conducted with live study work. Bounded also means reversible: if you cannot turn the tool off at the end without breaking the workflow, you have not run a pilot — you have made a purchase and postponed admitting it.

Keep eligibility and medical decisions human

Here is the line I would not move. A tool may gather, sort, summarize, and draft. A qualified human being decides. Final eligibility determinations, medical assessments, adverse event judgments, and any decision affecting a participant's care or a protocol's integrity stay with the investigator and trained study team — every time, without exception for speed.

This is how accountability works in a regulated trial. When a sponsor or regulator asks who decided a participant was eligible, the answer must be a person who can explain the reasoning, not a system no one at the site can reconstruct. So build review into the workflow: the tool produces a draft or a flag, a named staff member checks it against the source, and that review is documented. Human-in-the-loop is not a slogan in a sales deck. It is a specific person reviewing specific output before anything is acted on — if your pilot design cannot name that person, the workflow is not ready.

Check validation and privacy before participant data is involved

Any tool that touches participant data carries obligations a general office product does not. Before the pilot, get plain answers in writing. Where does the data go, and where is it stored? Is protected health information (PHI) used to train the vendor's models, and can you rule that out contractually? Who at the vendor can access the data, and how is access logged? What happens to the data if you end the contract?

Then check your own side. Does the tool require updates to your SOPs, data management practices, or validation records? Does your institutional review board (IRB) need to be informed, particularly if the tool contacts participants directly, as reminder workflows may? Do sponsor contracts restrict where study data may be processed, or which sub-processors may handle it? These questions are far cheaper before a pilot than after an auditor asks them. A vendor that treats them as friction has told you something — believe it.

The pilot scorecard

Judge the pilot on the workflow alone, using your baseline:

Question	Baseline	Pilot result	Verdict
Staff hours spent on this workflow per week
Error rate, as defined before the pilot
Cycle time from start to finish
Hours spent reviewing and correcting tool output
Errors that reached a participant, sponsor, or source document
Staff confidence: would the team keep using it?

Two rows deserve emphasis. The review-and-correction row is where inflated savings go to die: if the tool saves four hours of drafting and creates three hours of fixing, the honest saving is one hour. The errors-that-escaped row is a hard gate, not an average — a workflow can be faster and still fail if its errors travel further than the old process allowed.

Expand to a second workflow only after the first one earns it

If the first workflow passes — real time saved, error rate equal or better, no escaped errors, a team that wants to keep it — you have earned the right to test a second, with the same discipline. If it fails, you have bought valuable information cheaply: one contained disappointment instead of a platform-wide commitment.

I write this as the CEO of Amavita Research Services in Miami, a site with zero Food and Drug Administration (FDA) Form 483 findings, and as CEO of bioaccess, a first-in-human contract research organization (CRO). In both roles I see the same truth: sites need a fair test, honestly scored — not more technology promises. Pick one workflow, measure it before and during, and let the numbers decide.