Research · August 2026
TriageBench: Benchmarking Sequential Clinical Triage Reasoning
TriageBench is a proposed benchmark for sequential UK clinical triage. It tests each decision made before a complete case is known. A model must ask useful questions, stop at the right time, recognise red flags and recommend a safe level of care. The protocol is public; final performance claims will follow only after clinician review and a frozen benchmark run.
01 · The problem
The next safe action is often a question.
Most medical benchmarks reveal the whole case and ask for an answer. Patient-facing triage begins earlier. The system sees an incomplete description and must decide whether to ask, act, escalate or hand the case to a clinician.
Answering too early can miss an emergency. Asking indefinitely delays care. Sending every patient to emergency services may avoid some under-triage, but it is not useful triage. The real capability is knowing when enough evidence has been gathered for the next decision.
“Does the model know enough to act?”The question TriageBench is designed to test
Prior work points to this gap. MediQ studies proactive question asking. CARE-Bench evaluates action under partial disclosure. HealthBench shows the value of clinician-authored criteria for open-ended medical responses. TriageBench narrows the problem to sequential UK care routing grounded in reviewed pathway logic.
02 · Capability
Ask. Stop. Escalate. Route.
The benchmark separates four behaviours that a final-answer score can hide. Each one is measured at a specific point in a progressively disclosed case.
Ask
Request the missing fact that most changes the next safe action.
Stop
Recognise when enough evidence exists and avoid unnecessary delay.
Escalate
Respond immediately when a reviewed safety threshold is crossed.
Route
Recommend the minimum safe care level without inventing a diagnosis.
03 · Method
The complete case stays hidden.
Every task begins with a reviewed patient state and a limited visible history. If the model asks a valid question, a deterministic simulator reveals the corresponding fact. Acting ends the trajectory and triggers scoring.
The evaluator does not use a free-form model to improvise a patient. That would make the test difficult to reproduce. Facts, valid questions, red flags, care levels and stopping points come from the reviewed case graph.
Six complementary tracks
- Choose the correct current action from a partial case.
- Select the highest-value next question when asking is required.
- Stop questioning and select the minimum safe care level.
- React correctly when one decisive red-flag fact changes.
- Complete an interactive trajectory within a question budget.
- Write a clear patient-facing response after the structured decision.
04 · Scoring
Safety cannot disappear inside an average.
A single score can make a model look strong while hiding missed emergencies. TriageBench therefore publishes safety outcomes beside capability outcomes. Unsafe under-triage and critical red-flag recall remain visible in every headline result.
How often the model recommends care below the reviewed minimum safe level.
Whether critical warning signs cause the required safe action, not merely a mention.
Whether a question targets a required, high-priority fact without needless delay.
Whether the model avoids both premature answers and redundant questions.
Models that fail frozen safety thresholds can still be reported, but they are not eligible for an overall rank. Fluency cannot compensate for an unsafe structured decision.
05 · Failure analysis
A correct-looking answer can still be wrong.
The benchmark reports trajectory-level failures, not only a score. Important categories include premature disposition, missed red flags, delayed escalation, irrelevant or repeated questions, failure to stop, indiscriminate escalation and an unsupported diagnosis.
A patient reports a painful mouth ulcer. Duration and relevant history have not yet been disclosed.
- Unsafe response
- Assume a minor ulcer and recommend self-care.
- What is missing
- Duration can change the appropriate care route.
- Capability tested
- Ask the decisive question before routing.
Counterfactual cases make this diagnostic. Two cases can differ by one reviewed fact while every other detail remains fixed. The model should change its action only when that fact changes the safe route.
06 · Release design
Public method. Protected final test.
The planned release publishes the task definitions, schema, representative development cases, deterministic scorer and baseline configurations. Most final cases, safety-critical counterfactuals and adjudications remain private to reduce leakage and preserve the value of future evaluations.
Cases from one pathway family stay in one split. Public examples support integration and prompt development; a private holdout supports comparable final evaluation.
07 · Limits
A benchmark is evidence about a model, not approval for care.
TriageBench is UK-specific, bounded by its reviewed pathways and limited to written interaction. It cannot test physical examination, local service availability or clinical benefit. A strong score does not establish that a model is suitable for autonomous patient care.