Hospital AI beyond the benchmark — why live use needs different monitoring

TL;DR. A public benchmark is someone else's exam. Inside a hospital, clinicians ask unpredictable questions under real liability and workflow pressure. Recent peer-reviewed deployment lessons keep rhyming: monitoring must follow live use — not only the scoreboard.

Two different questions

Question	What it answers	What it misses
“Did the model score well on a shared test?”	Rough capacity vs peers	Your SOPs, languages, permissions, refusal rules
“Is this safe and useful here, this week?”	Live acceptance, rejection, unsupported claims	Almost everything a static suite never saw

Buyers often hear the first answer and assume they bought the second.

What changes when clinicians drive the prompts

In a real medical centre, the “user” is not a tidy benchmark prompt:

• Questions arrive mid-ward-round, mid-note, mid-handover.
• The same string can mean different jobs by role and department.
• Feedback is sparse — a silent reject may be the only signal.
• Liability sits with people and institutions, not with a leaderboard rank.

So a system can look strong on paper and still produce unsupported claims or unhelpful answers in the wild. That is not a reason to panic about AI; it is a reason to watch the deployment, not only the marketing PDF.

A calm buyer checklist

Clarify	Good enough answer sounds like…
Workflow	“We evaluated on these clinician tasks, not only a public suite.”
Monitoring	“We log rejects, escalations, and unsupported-claim patterns after go-live.”
Who can ask what	“Access is role-scoped; the eval used the same scopes.”
When we stop	“If monitoring trips, we throttle, abstain, or roll back — here is the playbook.”
Honesty	“Here are failure modes we already saw — we are not hiding week one.”

Soft OrchAI bridge (one short beat)

OrchAI Eval is the suite's quality-inspector idea: check AI surfaces on agreed suites before a change reaches users. Shipping status is still Partial (monitoring / CI-gate framing) — so this post is literacy, not a feature brochure. The useful habit is older than any product name: judge the system in the workflow you actually run.

What this post is not

• Not a summary of any single hospital's confidential results.
• Not a claim that AiHPC evaluated ChatEHR or any named clinical stack.
• Not a Hong Kong Hospital Authority / Department of Health programme.
• Not medical advice.

See also

• Strategy note: Stanford ChatEHR deployment eval · W34 digest
• Sibling literacy: Leaderboard vs your work · Same API name ≠ same model
• Series home: 科普 direction