An empirical study of using radiology reports and images to improve ICU mortality prediction. (arXiv:2307.07513v1 [cs.AI])

Background: The predictive Intensive Care Unit (ICU) scoring system plays an
important role in ICU management because it predicts important outcomes,
especially mortality. Many scoring systems have been developed and used in the
ICU. These scoring systems are primarily based on the structured clinical data
in the electronic health record (EHR), which may suffer the loss of important
clinical information in the narratives and images. Methods: In this work, we
build a deep learning based survival prediction model with multi-modality data
to predict ICU mortality. Four sets of features are investigated: (1)
physiological measurements of Simplified Acute Physiology Score (SAPS) II, (2)
common thorax diseases pre-defined by radiologists, (3) BERT-based text
representations, and (4) chest X-ray image features. We use the Medical
Information Mart for Intensive Care IV (MIMIC-IV) dataset to evaluate the
proposed model. Results: Our model achieves the average C-index of 0.7829 (95%
confidence interval, 0.7620-0.8038), which substantially exceeds that of the
baseline with SAPS-II features (0.7470 (0.7263-0.7676)). Ablation studies
further demonstrate the contributions of pre-defined labels (2.00%), text
features (2.44%), and image features (2.82%).