The goal of any radiological diagnostic process is to gain information about the patient’s status. However, the mathematical notion of information is usually not adopted to measure the performance of a diagnostic test or the agreement among readers in providing a certain diagnosis. Indeed, commonly used metrics for assessing diagnostic accuracy (e.g., sensitivity and specificity) or inter-reader agreement (Cohen κ statistics) use confusion matrices containing the number of true- and false positives/negatives results of a test, or the number of concordant/discordant categorizations, respectively, thus lacking proper information content. We present a methodological paradigm, based on Shannon’s information theory, aiming to measure both accuracy and agreement in diagnostic radiology. This approach models the information flow as a “diagnostic channel” connecting the state of the patient’s disease and the radiologist or, in the case of agreement analysis, as an “agreement channel” linking two or more radiologists evaluating the same set of images. For both cases, we proposed some measures, derived from Shannon’s mutual information, which can represent an alternative way to express diagnostic accuracy and agreement in radiology. Key points • Diagnostic processes can be modeled with information theory (IT). • IT metrics of diagnostic accuracy are independent from disease prevalence. • IT metrics of inter-reader agreements can overcome Cohen κ pitfalls.

An information-oriented paradigm in evaluating accuracy and agreement in radiology

Casagrande A.;Girometti R.
2023-01-01

Abstract

The goal of any radiological diagnostic process is to gain information about the patient’s status. However, the mathematical notion of information is usually not adopted to measure the performance of a diagnostic test or the agreement among readers in providing a certain diagnosis. Indeed, commonly used metrics for assessing diagnostic accuracy (e.g., sensitivity and specificity) or inter-reader agreement (Cohen κ statistics) use confusion matrices containing the number of true- and false positives/negatives results of a test, or the number of concordant/discordant categorizations, respectively, thus lacking proper information content. We present a methodological paradigm, based on Shannon’s information theory, aiming to measure both accuracy and agreement in diagnostic radiology. This approach models the information flow as a “diagnostic channel” connecting the state of the patient’s disease and the radiologist or, in the case of agreement analysis, as an “agreement channel” linking two or more radiologists evaluating the same set of images. For both cases, we proposed some measures, derived from Shannon’s mutual information, which can represent an alternative way to express diagnostic accuracy and agreement in radiology. Key points • Diagnostic processes can be modeled with information theory (IT). • IT metrics of diagnostic accuracy are independent from disease prevalence. • IT metrics of inter-reader agreements can overcome Cohen κ pitfalls.
File in questo prodotto:
File Dimensione Formato  
s41747-023-00327-y.pdf

accesso aperto

Tipologia: Versione Editoriale (PDF)
Licenza: Creative commons
Dimensione 1.28 MB
Formato Adobe PDF
1.28 MB Adobe PDF Visualizza/Apri

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11390/1245444
Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus 0
  • ???jsp.display-item.citation.isi??? ND
social impact