Login / Signup

A new accuracy metric under three classes when subclasses are involved and its confidence interval estimation.

Nan NanLili Tian
Published in: Statistics in medicine (2023)
"Compound multi-class classification" refers to the setting where three or more main classes are involved and at least one of the main classes have multiple subclasses. A common practice in evaluating biomarker performance under "compound multi-class classification" is "subclasses pooling." In this article, we first explore the downsides of accuracy metrics based on pooled data. Then we propose a new accuracy measure proper for "compound multi-class classification" with three ordinal main classes, namely "volume under compound R O C $$ ROC $$ surface ( V U S C $$ VU{S}_C $$ )." The proposed V U S C $$ VU{S}_C $$ evaluates the accuracy of a biomarker appropriately by identifying main classes without requiring specification of an ordering for marker values of subclasses within each main class. For confidence interval estimation of V U S C $$ VU{S}_C $$ , both parametric and nonparametric methods are studied, and simulation studies are carried out to assess coverage probabilities. A subset of Alzheimer's Disease Neuroimaging Initiative study dataset is analyzed.
Keyphrases
  • machine learning
  • deep learning
  • healthcare
  • primary care
  • quality improvement
  • randomized controlled trial
  • cognitive decline
  • health insurance