A prevalence-robust measure of diagnostic test performance

The sensitivity and specificity of diagnostic tests are generally assumed to be independent of the prevalence of the disease as opposed to the predictive values, which depend on it. Nevertheless, daily practice shows that this is not the case, and the independence hypothesis appears to be only a coarse simplification of the real world, where the disease mechanisms and the limitations involved in the experiments may let the prevalence influence the sensitivity and specificity measures. Independence can be affected by clinical factors, such as spectrum bias, reader expectations, and referral filters, or impacted by the design of the diagnostic experiments, such as the selection of patients, verification bias, and imperfect reference standards. The information ratio was recently introduced as a metric for diagnostic test performance. It models the flow of information from the disease state to the reader as a “diagnostic channel” and measures it as a single value in the interval [0.0,1.0]. The information ratio was defined to be prevalence-independent. This work extends the analysis of these metrics and shows that the information ratio is also robust with respect to both clinical and experiment-design-related perturbations.