The evaluation below was produced by a team of four doctor's reviewing the implementation of Visiba Triage in a UK PCN.
Dr Painter A; Dr Leung K; Dr Rajan R; Dr Gillespie L
Introduction
In England, ~44% of primary care appointments are booked for urgent same day review [1], despite clinical feedback suggesting many of these cases would be more appropriately managed through less urgent pathways. One of the barriers to demand management at the front door of primary care is the ability to safely triage patients as a first point-of-contact. Artificial intelligence offers a potential safe, scalable solution.
Aims
The aim of this evaluation is to assess the impact of an AI/ML-enabled enhanced patient triage on the operational delivery of primary care. Operational delivery is measured through same-day appointments utilisation, and time spent per patient and/or case. Clinical agreement with the AI-enabled triage outputs were measured and evaluated as a secondary outcome.
Methods
An NHS Primary Care Partnership, operating across four sites and serving >12,000 patients introduced the AI enhanced patient triage. Patients who would previously have been given same day appointments were triaged using the AI triage model. Patients were either sent a text message with a link to the triage webpage or reception staff completed the questions on behalf of the patient over the phone. The AI triage model provided reception staff with immediate feedback on the recommended urgency of the case. Urgency was mapped using a modified Manchester Triage Scale, showing 8 levels of urgency and associated timeframes for clinical review. Possible differential diagnoses were also generated. This information was used by clinical staff to navigate the patient to the optimal healthcare professional in the appropriate timeframe based on clinical need.
Results
The demand on same day appointments
Graph 1 shows the AI triage of patient urgency, for patients previously receiving same day appointments. Of these cases, 10.1% were triaged as urgency level 1-3 which qualify for assessment within 60 minutes. These patients required urgent support, including from 999 services. Urgency level 4 equates to patients needing same day assessment, which represents 16.6% of all same day requests. Urgency level 5-7 varied from next day up to 1 month assessment timeline, representing 60.4% of all cases. Urgency level 8 denotes cases where self-care can be appropriate and was indicated in 12.8% of all cases.
Graph 1: Showing the distribution of urgency across all patients previously accessing same-day urgent pathways (n = 336).
Clinician agreement
The AI patient triage tool uses a clinical feedback loop to provide continual learning from real-world cases and clinicians. The feedback loop provides real-time analysis of clinician agreement with the model outputs. An analysis of patient cases deemed to be non-urgent by the model, with feedback from on-site NHS primary care clinicians, showed a 95.82% (n=336) clinical agreement with the non-urgent classification of cases. Of the 4.17% of cases where clinicians disagreed with the non-urgent classification, 1.79% related to cases classified as needing assessment within 24 hours, where doctors felt this should be within 4 hours.
Time spent per patient
In terms of processing a triage request a clinician would spend less than 3 minutes on average processing a triage request with either the request resolved or an appointment made for a face-to-face consultation at a desired timeframe. The summarized history also reduces the time taken for the clinician seeing the patient face to face but that can vary depending on the clinical presentation.
Conclusions
AI-enabled patient triage at the front door of primary care has the potential to support an overburdened healthcare workforce to undertake complex triage decisions. Intelligent resource allocation based on clinical need means primary care providers can facilitate more strategic appointment utilisation and free-up same-day capacity.
[1] NHS Digital (2022). Appointments in General Practice. Available at : Appointments in General Practice, October 2022 - NHS Digital
