Elsevier

Journal of Clinical Epidemiology

Volume 128, December 2020, Pages 66-73
Journal of Clinical Epidemiology

Original Article
The transfer of clinical prediction models for early trauma care had uncertain effects on mistriage

https://doi.org/10.1016/j.jclinepi.2020.08.014Get rights and content

Highlights

  • This study suggests that the transfer of clinical prediction models for early trauma care might lead to decreased or increased mistriage rate depending on setting and clinical context.

  • Models developed in samples from centres with high volumes of trauma patients generally performed better after transfer, both in validation and after transfer. However, these transfers also lead to increased undertriage.

  • Overtriage is more affected by model transfer than undertriage. Some transfers leading to improved overtriage also lead to worse performance in terms of undertriage.

Abstract

Objectives

This study aimed to assess how transfers of clinical prediction models for early trauma care between different care contexts within a single health system affected mistriage rates.

Study Design and Setting

Patients aged 15 years or older, registered between 2011 and 2016 in the Swedish national trauma registry, SweTrau, were included. Three data set groups were created: high- and low-volume centers, metropolitan and nonmetropolitan centers, and multicenters and single centers. Clinical prediction models were developed using logistic regression in each data set group and transferred between data sets within groups. Model performance was evaluated using mistriage rate, undertriage rate, and overtriage rate. Multiple imputation using chained equations was used to handle missing data. Model performance was reported as medians with 95% confidence intervals (CIs).

Results

A total of 26,965 patients were included. Changes in mistriage rates after transfer ranged from −0.25 (95% CI −0.21 to 0.04) to 0.29 (95% CI 0.13–0.39). Both overtriage and undertriage rates were affected.

Conclusions

Transferring clinical prediction models for early trauma care is associated with substantial uncertainty in regards to the effect on model performance. Depending on the care context, model transfer led to either increased or decreased mistriage. Overtriage was more affected by model transfer than undertriage.

Introduction

Trauma, defined as physical injuries to a host by outside objects [1], accounts for approximately 458.5 million hospital visits annually across the globe [2], and around four million deaths [3,4]. Each year, 9% of global deaths are the result of trauma, with the leading causes being road traffic accidents, suicide, and homicide. Predictions indicate that the incidence of trauma due to these causes is likely to increase by 2030 [5].

In a typical high-resource setting, the initial management of trauma is performed on the scene by emergency medical services. Patient data and vital signs are transferred to the receiving hospital. This information is then evaluated using a system to determine the level of trauma, prepare adequate resources [6], and dictate whether a full or limited trauma team is activated [7].

Systems that determine the level of trauma during early trauma care can be based on clinical prediction models. Models differ in quality and characteristics but generally perform well at predicting survival [8]. Many models are developed in a single, standardized context, such as a major trauma center, and are then implemented in different contexts [9].

What is not fully understood is how this transfer affects model performance. Previous research has shown that model performance in terms of calibration can be adversely affected [9]. However, that study assessed model transfers between substantially different settings (India and the United States) and did not assess more clinically relevant performance measures, such as misclassification.

In trauma, misclassification is often referred to as mistriage. Triage refers to the classification of trauma severity as minor or major. Mistriage can be subdivided into overtriage, which is the incorrect classification of a patient with minor trauma as one with major trauma, or undertriage, which is the incorrect classification of a patient with major trauma as one with minor trauma. Mistriage can ultimately lead to decreased patient survival and is also detrimental to patient care and the distribution of resources [7].

The effects of model transfers between care contexts within a single health care system, as well as the effect of such transfers on mistriage, have not been studied and represent substantial knowledge gaps. The aim of this study was to assess how transfers of clinical prediction models for early trauma care between different care contexts within a single health system affect mistriage rates.

Section snippets

Design

A registry-based cohort study was conducted using SweTrau data to create clinical prediction models, which were then transferred between different data sets to study the effects of model transfer on mistriage. The study and analysis plans were made publicly available before the research was undertaken [10].

Source of data

Sweden has a nationally encompassing trauma registry, SweTrau. At the time of the study, the register consisted of 55,000 trauma cases, recorded from 52 of Sweden's 55 hospitals [11].

Participants

The

Results

We analyzed data from 26,965 trauma patients (Table 1) after excluding 78 patients with missing date and time of trauma. The total number of missing observations across all variables was 9,984 in the entire study cohort. The data set with the highest percentage of missing observations was the nonmetropolitan data set, with 48% incomplete observations. The variable with the highest number of missing values was RR, with 8,296 missing values, or 31% of the total values for this variable. The

Discussion

This study aimed to assess how transfers of clinical prediction models for early trauma care between different contexts within a single health system affect mistriage rates.

The most notable effect on model performance after model transfer was observed after transferring the single center model. This transfer resulted in an increased mistriage rate of 0.29. Mainly contributing to this was an increase in overtriage. By contrast, the transfer of the multicenter model to the single center

Conclusion

Depending on the care context, model transfer led to either increased or decreased mistriage. Both overtriage and undertriage were affected by model transfer, and the effects on mistriage were in all cases primarily due to changes in overtriage. Data sets with a high number of patients had the lowest mistriage both during validation, and after transfer to the validation sample in the other data set. However, the transfer of these models (while improving mistriage) also led to increased

CRediT authorship contribution statement

Martin Henriksson: Software, Formal analysis, Data curation, Writing - review & editing. Dell D. Saulnier: Conceptualization, Methodology, Formal analysis, Data curation, Writing - review & editing. Johanna Berg: Formal analysis, Data curation, Writing - review & editing. Martin Gerdin Wärnberg: Conceptualization, Methodology, Software, Formal analysis, Data curation, Writing - review & editing.

Acknowledgments

The analyses were performed on resources provided by the Swedish National Infrastructure for Computing (SNIC) at LUNARC. The data were provided by the Swedish trauma registry (SweTrau).

References (29)

  • M. Naghavi et al.

    Global, regional, and national age-sex specific mortality for 264 causes of death, 1980-2016: a systematic analysis for the global burden of disease study 2016

    Lancet

    (2017)
  • Injuries and violence: The facts. Report

    (2014)
  • Advanced Trauma Life Support (ATLS)

    (2018)
  • A. Granstrom et al.

    A criteria-directed protocol for in-hospital triage of trauma patients

    Eur J Emerg Med

    (2018)
  • Cited by (0)

    Trial registration: The protocol for this study was uploaded to github.com/613Martin/transfer-effect-mistriage before the research was undertaken.

    Conflict of interest: The authors declare no conflict of interest.

    Source of funding: This work was supported by the Swedish National Board of Health and Welfare, Sweden (grant numbers 22464/2017, 23745/2016 and 22289/2015-3). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

    Ethics committee approval: This study has been approved by the Regional Ethical Review Board in Stockholm. Ethical review numbers 2015/426-31 and 2016/461-32.

    View full text