Introduction
Stroke is the second leading cause of death worldwide (1, 2). Due to advances in cardiovascular medicine and in recanalization therapies, including intravenous thrombolysis (IVT) and especially mechanical thrombectomy (MT, synonymous: endovascular thrombectomy, EVT) for large vessel occlusion (LVO), the number of strokes and stroke survivors has increased over the past 2 ½ decades (3–5).
Despite the aforementioned advances, acute ischemic stroke therapy remains a time-critical disease, and comprehensive stroke centers (CSC) capable of EVT are not evenly distributed, particularly in rural areas (6, 7). While telestroke networks enable a framework for IVT, patients often need to be transferred for EVT, and in some instances, interventionalists are even transferred for MT by car, helicopter or even plane (8–10). Yet, secondary transport of patients for MT is time-consuming and can worsen the patient’s prognosis (11, 12).
LVO detection in the prehospital setting and direct transfer to CSCs can lead to faster symptom-to-recanalization times. In the Maryland stroke study, preclinical triage strategies for LVO detection indeed led to faster transport to a CSC which translated to improved outcomes by avoiding delayed transport to a closer primary stroke center (PSC) (i.e., “drip and ship”), however, as only patients with LVO were analyzed and not the whole stroke population including IVT only a selection bias exists (13, 14). This was done without disproportionately delaying IVT, yet the effect of overtriage for the CSCs by false-positive LVO patients may vary regionally (4, 7, 8, 14, 15).
Although many clinical stroke scores have been published to identify LVO (16–24), very few of them have been validated “in the field” by paramedics, and even fewer amongst patients with acute ischemic stroke eligible for intravenous thrombolysis (IVT), hemorrhagic stroke, or stroke mimics (18, 25–27). The sensitivity of these scores is significantly inferior to the specificity, with acceptable to good results (28). No scale has both high sensitivity and specificity for determining large vessel occlusion, so that despite a negative test, a LVO may still be present with a probability of ≥10% (29).
However, in the absence of mobile CT or other brain imaging tools, all guidelines worldwide recommend the use of stroke scales, though none endorse a specific scale as having clearly superior performance (30). When prehospital scores alone are used to diagnose LVO and decide on a target hospital, 50% of patients are inappropriately diverted to a CSC, causing potential harm by delaying IVT (13). Therefore, a method with higher sensitivity would be optimal as an addition to the established scales.
Several strategies are currently applied or in the research phase: mobile stroke units, stroke serum biomarkers, electroencephalography (EEG) and near infrared spectro-scopy, amongst others. Neurovascular ultrasound such as transcranial Doppler (TCD) or transcranial color-coded sonography (TCCS) is uniquely suited for the prehospital setting, offering a mobile, non-invasive, relatively inexpensive solution that can be guided by telemedicine and is aimed directly at diagnosing LVO. In the Regensburg stroke mobile project, stroke neurologists joined the emergency team at the patient’s site and applied TCCS in conjunction with echo-enhancing agents (EEA).
The combination of TCCS with neurological stroke knowledge had a sensitivity of 90% and specificity of 98% for middle cerebral artery occlusion (MCAO) (31, 32). In contrast to CT, TCCS is pathophysiologically oriented; for example, in a case of left sided hemiparesis, the investigation starts at the contralateral skull for right MCAO detection.
A novel mobile sonography device was recently developed, specifically designed to investigate hemispheric bilateral brain perfusion (SONAS™) as a point-of-care (POC) diagnostic. A phase II-III study on brain perfusion ultrasound (BPU) in pre-selected acute ischemic stroke patients who underwent CT-perfusion indicated a sensitivity of 93.3% for LVO detection (33). The advantage of SONAS BPU lies in the simplicity of simultaneous hemispheric perfusion measurements after bolus injection of EEA, which may be carried out by paramedics with telemedical support.
The aim of the present study was to analyze the SONAS study patient dataset to compare the sensitivity and specificity of various prehospital stroke scores and the SONAS device for LVO detection, and to assess whether combining BPU with prehospital scores enhances diagnostic confidence in identifying acute LVO.
Methods
This study was a secondary analysis of data from the initial SONAS study, with the addition of calculated stroke scales from the initial NIHSS and ultrasound data derived from the patient chart (33). The SONAS study was approved by the local ethics committee of the university clinic Regensburg (21-2,586-104), registered in the European union clinical trials register (EudraCT Number: 2018-001279-19) and the final patient was enrolled December 19th 2019. No new data was acquired for this study.
In brief, the SONAS study investigated whether a novel non-imaging ultrasound device was able to detect hemispheric perfusion differences in patients with acute ischemic stroke. The SONAS® device is a battery-powered, non-imaging ultrasound system weighing only 2.2 kg, consisting of a transducer-headset that is placed over the patient’s temporal sound windows like headphones, and an operator unit. After injection of a FDA-approved ultrasound contrast agent (UCA, Sonovue®), an increased backscatter leading to hemispheric time intensity curves that correlate to brain perfusion, as previously demonstrated by Kilic et al.
(34) with perfusion MRI. The time between injection and maximum signal is referred to as the ‘time to peak’ (TTP). Any changes in the hemispheric TTP are displayed as a numerical value, e.g., LVO of the middle cerebral artery would result in delayed or absent hemispheric contrast inflow.
The results (LVO/non-LVO) of the SONAS measurement of the 20 study patients performed immediately after admission were compared with CT perfusion imaging (32). Briefly, inclusion criteria were age ≥ 18 years, with a clinical diagnosis of ischemic stroke (NIHSS > 6 and exclusion of intracerebral hemorrhage (ICH)), onset of symptoms <24 h and assessment of brain perfusion using perfusion cCT. Main exclusion criteria included existing contraindications to the SonoVue®, severe cardiac or pulmonary disease, acute endocarditis and artificial heart valve, amongst others.
However, none of these occurred in the study population. Fourteen of the 20 patients included were female (70%). All patients were of Caucasian origin.
The patients ranged in age from 44 to 91 years, with an average age of 72.8 years. For each patient included, NIHSS certified doctors acquired the NIHSS within 30 min after admission. MCAO was present in 17 (85%) of the 20 patients according to the reference standard.
The SONAS assessment could not be performed in two subjects due to low data quality. Of the 18 evaluable datasets, 14 subjects were correctly identified with existing MCAO by the SONAS assessment and 2 were correctly identified with no MCAO. However, a single patient was identified as having a perfusion deficit by the SONAS assessment that was not detected as stroke by the reference standard.
An additional subject was identified as not having a perfusion deficit by the SONAS assessment but was confirmed as having stroke as detected by the reference standard (33). Since SONAS was a single center trial in our clinic, we had access to all patient source data.
Prehospital stroke scales
The acquired NIHSS data of the SONAS study participants were used to retrospectively calculate 16 different prehospital scores, of which 10 of them had NIHSS equivalences described in the validation studies: RACE, CPSSS, CPSS and d-CPSS, VAN, PASS, FAST-ED, CG-FAST, G-FAST, and sNIHSS-EMS (see Table 1, see Supplementary material). In addition, we investigated the following stroke scales with items that were not directly transferable from the NIHSS: LAMS, LARIO, 3I-SS, ELVO, VES, and M-DIRECT (see Table 2). Of these scores, G-FAST, CG-Fast, RACE and NIHSS perform best according to a recent prospective study by Duvekot in 2021 (28).
The SONAS trial excluded patients with ICH, the main differential diagnosis in severe acute ischemic stroke, as its primary aim was to evaluate accuracy in detecting perfusion asymmetry. In this secondary analysis, we performed an exploratory analysis using the prehospital intracerebral hemorrage (pH-ICH) score that was validated in the mobile stroke unit project in Berlin in ICH analysis only in order to compare this to the BPU results (35).
In all scores we used the published cutoff values that yielded the highest sensitivity and specificity (see Supplementary material).
Statistical analysis
The various prehospital stroke scores, SONAS results and the combination of prehospital stroke scores with SONAS BPU results were applied to the 20 study patients and their sensitivity, specificity, positive and negative predictive value (PPV and NPV) were calculated with regard to their recognition of LVO.
Results
Comparison of the performance of different prehospital scales
In the SONAS study population, patients were classified into two groups according to the initial CT results (which included CT-Angiography and CT-Perfusion): “large vessel occlusion present” (n = 17) and “large vessel occlusion absent” (n = 3). The results of the cerebral imaging were used to define the “ground truth,” on which all further analyses are based on.
We focused our primary analysis on the scores found to perform best: G-FAST, CG-Fast, RACE, and NIHSS, while the results of the other analyzed scores are shown in the Supplementary material (Figure 1; see also Supplementary material). Overall, these four scores showed good results: With a cut-off between 2 and 3, G-FAST as well as CG-FAST would have missed an LVO in a single patient (#20), while correctly classifying all other patients. This corresponds to a sensitivity of 94% (16/17) and a specificity of 100% (3/3).
The PPV would be 100% (16/16) and the NPV would be 75% (3/4). Though the performance of the two scores was identical regarding sensitivity, specificity, PPV and NPV, CG-FAST should be preferred based on our results as it seems to be able to classify patients better (the “C”-aspect of the score was only added to patients with LVO and thus the group means have a larger distance) (22).
With regard to the RACE score, the cut-off value was set between 4 and 5. This classification would have missed two patients with LVO (#16 and #20) and would have classified all other patients correctly. This corresponds to a sensitivity of 88% (15/17), a specificity of 100% (3/3), PPV of 100% (15/15), and an NPV of 60% (3/5).
While G-FAST, CG-FAST and RACE are common prehospital scores, the NIHSS is the most common hospital score used to assess stroke severity, including in patients with LVO. If the cut-off value would have been set between 7 and 8, it would have been able to classify all of our patients within the cohort correctly. This would correspond to a sensitivity, specificity, PPV and NPV of 100% each.
Performance of SONAS
The SONAS-BPU device was unable to demonstrate time-intensity curves in 2 out of 20 patients (#6 and #17) in the SONAS study (33). The device indicated low data quality, due to either high attenuation of the ultrasound from inadequate temporal bone windows, decreased signal-to-noise ratio or non-optimal headset placement. In the remaining 18 patients, the data showed a very good concordance with the CT results as illustrated in the contingency table (Table 3).
This corresponds to a sensitivity of 93% (14/15), a specificity of 66% (2/3), a PPV of 93% (14/15), and an NPV of 66% (2/3).
Combination of BPU-SONAS with prehospital scales
To investigate whether or not the combination of the two best permorming prehospital scales (C(G)-FAST, RACE) and BPU improved LVO detection, we analyzed the following scenarios: If both examinations showed negative results, the patient was classified as “no large vessel occlusion”. If both showed positive results, the patient was classified as “large vessel occlusion”. For contradicting analyses, there were two possibilities: (A) classification as “large vessel occlusion” or (B) classification as “no large vessel occlusion” (Figure 2).
For these contradicting analyses, Scenario (A), or empiric classification as “LVO,” should be chosen if an increase in sensitivity is wished for at the cost of specificity, while Scenario (B), or determination of “no LVO,” would lead to an increase in specificity and a decrease in sensitivity. Scenario (A) would favor a good outcome in patients with real LVO at the cost of patients with smaller vessel occlusion or stroke mimics (as the “unclear” patients with contradicting predictions are transferred to the next hospital capable of mechanical thrombectomy), Scenario (B) would favor patients with small vessel occlusions and stroke mimics that need urgent medical treatment (as the “unclear” patients are transferred to the nearest hospital). To perform these analyses, the two patients that did not have useable data from SONAS were excluded, reducing the total number of patients to 18.
Scenario A
(C) G-Fast
The sensitivity of the combined method was 100% (15/15), the specificity 66% (2/3), the PPV 94% (15/16), and the NPV 100% (2/2). Compared to the results of the score alone, the sensitivity would have increased (score alone 94%) at the cost of specificity (score alone 100%). The PPV would have also decreased (score alone 100%) with an increase in negative predictive value (score: 75%).
Overall, a shift of the prediction values would have been achieved without clear improvement or deterioration.
Race
The same applies to the RACE score in scenario A. The sensitivity would increase from 88 to 100% at the cost of specificity with a decrease from 100 to 33%. The PPV would decrease from 100 to 88% and the NPV would have increased from 60 to 100%.
Scenario B
(C) G-Fast
The sensitivity of the combined method would be 86% (13/15), specificity 100% (3/3), PPV 100%, and the NPV 60% (3/5). Compared to the results of the prehospital score alone, the values remained almost identical.
Race
The sensitivity of the combined method would be 86% (13/15), specificity 100% (3/3), PPV 100%, and the NPV 60% (3/5). Again, compared to the results of the prehospital score alone were almost identical.
Scenario C
Scenario C examines the situation in patients that are LVO-positive by SONAS, but not by the prehospital score, or vice versa. In this scenario, patients are grouped into three clusters: “clear LVO,” “clearly no LVO” and “unclear.” This scenario is shown in Figure 3 and makes maximal use of both the advantages of the SONAS prediction (high sensitivity) and the score prediction (high specificity).
This combination of the scores would have classified either 1 patient (by RACE score) or 2 patients (by CG-FAST and G-FAST scores) as “no LVO,” respectively. Both classifications would have been correct. The same applies to 13 patients in which both BPU-SONAS as well as the three prehospital stroke scores classified the patients into “clear LVO” – again, all classifications would have been correct.
Thus, this classification system increases the predictive values.
The classification of “unclear” cases, in which an LVO might be present but contradicting predictions have emerged, would have been recognized reliably. This would have occurred in 4 patients using RACE (with 2 having a real LVO) and 3 patients using (C)G-FAST, of which 2 had an LVO. Most importantly, this group included ALL patients with false predictions by either the prehospital score or BPU-SONAS, demonstrating this classification system as highly effective in recognizing unclear or falsely classified cases.
In an exploratory analysis we applied the only validated prehospital intracerebral hemorrhage score (pH-ICH) to our patient population with cerebral ischemia only (35). A pH-ICH score of <3 indicates that ICH is more likely than cerebral ischemic stroke, and this constellation was found in 3 of our 20 patients (false positive results). In all of these 3 patients, a perfusion deficit could be detected in both SONAS-BPU and perfusion CT.
Discussion
In this study, we investigated the performance of 16 different prehospital stroke scales on their ability to detect LVO in a pre-selected population of acute ischemic stroke patients with and without LVO enrolled in the SONAS-Trial (33). The two most promising stroke scores, (C)G-FAST and RACE, were also analyzed in conjunction with the results of brain perfusion ultrasound by the SONAS device. In summary, the simple combination of SONAS with prehospital stroke scores does not add to the predictive values themselves but enables adjustment for preference of sensitivity over specificity, PPV over NPV, and vice versa.
Accurate diagnosis of either LVO or ICH is critical for site-specific transfer and appropriate stroke treatment, balancing the benefit of early reperfusion in LVO patients with the risk of potentially delayed IVT in non-LVO patients or delayed initiation of antihypertensives and reversal agents in ICH patients.
Since the expansion of recanalization therapies for the treatment of acute ischemic stroke, prehospital identification and separation of stroke mimics has been the primary goal of clinical prehospital scores (36). The FAST-MAG trial was the first prehospital interventional trial making use of a prehospital stroke identification score – FAST (37). In the study population, acute ischemic stroke comprised 73.3% of patients, while ICH was present in 22.8%, and a stroke-mimicking condition in 3.9%.
Since then, a variety of prehospital stroke scales have been designed and tested with the primary goal of clinically identifying LVOs in the field. Yet, many were tested in selected populations, i.e., in known ischemic stroke patients, excluding those with ICH, by medical doctors in the emergency ward or stroke unit, or in registries with IVT and MT patients only (22, 27). The RACE score is one of the few scores to have been tested prior the clinical RACECAT trial (15, 18).
However, according to the first study, 20% of patients in the RACECAT were re-directed to CSCs for MT actually suffered from ICH. Interestingly, in the RACECAT study, patient transport to the nearest stroke unit with suspected LVO led to better chances of functional independence at 90 days for patients with ICH (38). This positive effect may be attributed to earlier hospital arrival of patients with earlier CT diagnosis of ICH, initiation of early blood pressure management, prevention of aspiration, control of seizures, and with hemostatic normalization in patients with anticoagulation.
In summary, bypassing PSCs for suspected LVO has the potential of backfiring in patients with ICH, and calls for earlier prehospital stroke diagnostics. Even the addition of machine learning algorithms to prehospital stroke scales was found to be imperfect due to a lack of external validation in randomized studies (39). Therefore, if a single clinical score that can detect LVO preclinically with both high sensitivity and high specificity is not available, a scoring system to identify ICH may help (40, 41).
A recent meta-analysis by Almubayyidh et al. (40, 41) evaluated the performance of prehospital models in distinguishing ICH from other causes of suspected stroke, identifying 9 prediction studies, five of which were designed to differentiate between stroke subtypes and three to distinguish between ICH and acute ischemic stroke-- including the study by Geisler et al. defining pH-ICH, which was specifically designed to identify ICH and therefore used in our study to test for false-positive results (35, 41). In the meta-analysis, all studies were assessed as having a high risk of bias, particularly in the analysis domain.
The performance of the models varied, with the area under the receiver operating characteristic curve ranging from 0.73 to 0.91.
The current study, which used the highly selected population of the SONAS trial lacking patients with ICH (as the SONAS trial was primarily designed to test patients with relevant hemispheric perfusion differences), identified the (C)G-FAST and RACE scores as having high sensitivity and specificity to detect LVO patients. In addition to being a feasibility study with only 20 patients, the main limitations of this study include the lack of randomization and blinding, application by trained stroke neurologists not paramedics or emergency physicians, as well as the exclusion of patients with stroke mimics, including those with ICH or subarachnoid hemorrhage. Application of the ph-ICH score the SONAS study cohort incorrectly classified 15% of patients (3 of 20 patients) as having ICH over ischemic stroke similar to the misclassification in the RACECAT study of ICH as LVO by the application of the RACE score (18, 38).
Thus, our results are very much in line with previous comparative studies on prehospital stroke scales, supporting the overall validity of the results (15, 42). However, in all 3 both perfusion CT and BPU correctly identified significant perfusion deficits. Indeed, little data exist on hemispheric perfusion abnormalities in ICH, and none have been published with the SONAS™-device yet.
Few studies examining perfusion changes in patients with spontaneous ICH focused on local perfusion changes round the ICH (43). However, the observed changes are minor compared to LVO: In a recent study in ICH patients without midline shift the mean perfusion deficit on single photon emission tomography (SPECT) was 15% only perilesionally, attributed to compression of the neighboring brain tissue by perihematoma oedema (44).
In order to improve prehospital stroke diagnosis and diagnostic confidence, especially in severe stroke such as ICH and LVO, additional diagnostic testing seems mandatory to further identify, characterize and stratify stroke-call patients in the prehospital phase that are unselected and investigation should be performed by trained paramedics or emergency doctors perhaps with telemedical support by experienced stroke neurologists. This ancillary testing may be classified as blood-serum based or imaging-neurophysiological based, and with or without the aid of telemedicine. Blood-serum biomarkers may include NSE, MMP-8, S-100B, D-Dimer, GFAP, amongst others, and technical devices for cerebral imaging and functional assessment may be microwave-based, mobile MRI-units, mobile electroencephalography and evoked potentials, and neurosonography, amongst others (45–50).
Measurement of glial fibrillary acidic protein may be the serum biomarker best characterized in the prehospital field and with the greatest potential to identify ICH, though its use limited to the first hours after symptom onset (51–53). Mobile stroke units are able to combine a variety of these aforementioned techniques, but their limited mobility range may restrict use to more densely populated areas (54–56). Our own group had prehospital experience with conventional portable transcranial color-coded ultrasound, and experienced neurologists achieved high sensitivity and specificity of >90% in identification of MCA occlusions (31, 32).
However, deployment of stroke neurologists for pre-hospital stroke diagnostics is inefficient and requires great individual enthusiasm and funding.
Outlook
If diagnostic confidence in LVO identification is sufficiently high, it could pave the way for prehospital thrombolysis. Recently, tenecteplase has been approved as an alternative and potentially superior thrombolytic agent than alteplase in acute ischemic stroke, and especially in patients with LVO (57, 58). Given its ease of administration, tenecteplase would be the ideal agent to perform prehospital thrombolysis in patients for whom there is high diagnostic confidence of acute ischemic stroke from SONAS-BPU and prehospital stroke scales, possibly with exclusion of ICH using serum GFAP and confirmatory support by stroke specialists via telemedicine (41).
Another consequence of this combined use of SONAS-BPU with RACE or (C)G-FAST may be the proper adjustment of blood-pressure and careful revereal of ongoing anticoagulation in the prehospital setting. The importance of this was suggested by the recent INTERACT-4 trial, in which blood-pressure lowering may be beneficial in ICH but harmful in ischemic stroke (59). Also, accurate prehospital LVO identification may further aid concept such as “direct to angio-suite” approaches (4, 60).
Technical refinement of the SONAS™ device for detection of posterior circulation ischemic stroke (especially acute posterior cerebral artery and basilar artery occlusion), presently underway, are urgently needed, as this population was absent in the current study and pose additional challenges in diagnosis and prognosis.
As stroke is a time critical disease, any time delay using POC diagnostics including stroke scales, use of MSU and SONAS™ should be as brief as possible. Thus, comparative studies between various methods of POC equipment for stroke diagnostics should include timelines, sensitivity/specificity, overall costs, simplicity and robustness, and should be performed prehospital arrival in an unselected stroke cohort, including patients ICH, LVO, lacunar stroke, subdural hematoma and stroke mimics, amongst other (61).
Conclusion
In the present retrospective study, we compared the sensitivity, specificity, and predictive values of various prehospital stroke scales with the novel brain perfusion ultrasound device SONAS in detecting LVO and ICH. While the stroke scales alone demonstrated effectiveness in identifying LVO, our results suggest that combining SONAS-BPU with the highest-performing scales, such as CG-FAST, may contribute to a more efficient framework to guide site-specific transfer strategies for acute stroke patients. However, these findings require validation in larger, more heterogeneous cohorts, including real-world prehospital application by paramedics of both CG-FAST and SONAS-BPU, ideally supported by telemedical support from experienced stroke neurologists.