Performance comparison of CGM systems: MARD values are not always a reliable indicator of CGM system accuracy

Harald Kirchsteiger, Lutz Heinemann, Guido Freckmann, Volker Lodwig, Günther Schmelzeisen-Redeker, Michael Schoemaker, Luigi Del Re

Research output: Contribution to journalArticlepeer-review

35 Citations (Scopus)

Abstract

Background: The ongoing progress of continuous glucose monitoring (CGM) systems results in an increasing interest in comparing their performance, in particular in terms of accuracy, that is, matching CGM readings with reference values measured at the same time. Most often accuracy is evaluated by the mean absolute relative difference (MARD). It is frequently overseen that MARD does not only reflect accuracy, but also the study protocol and evaluation procedure, making a cross-study comparison problematic. Methods: We evaluate the effect of several factors on the MARD statistical properties: number of paired reference and CGM values, distribution of the paired values, accuracy of the reference measurement device itself and the time delay between data pairs. All analysis is done using clinical data from 12 patients wearing 6 sensors each. Results: We have found that a few paired points can have a potentially high impact on MARD. Leaving out those points for evaluation thus reduces the MARD. Similarly, accuracy of the reference measurements greatly affects the MARD as numerical and graphical data show. Results also show that a log-normal distribution of the paired references provides a significantly different MARD than, for example, a uniform distribution. Conclusions: MARD is a reasonable parameter to characterize the performance of CGM systems when keeping its limitations in mind. To support clinicians and patients in selecting which CGM system to use in a clinical setting, care should be taken to make MARD more comparable by employing a standardized evaluation procedure.

Original languageEnglish
Pages (from-to)1030-1040
Number of pages11
JournalJournal of diabetes science and technology
Volume9
Issue number5
DOIs
Publication statusPublished - Sept 2015
Externally publishedYes

Keywords

  • Accuracy
  • CGM
  • Continuous glucose monitoring
  • MARD
  • Performance comparison
  • Performance evaluation
  • Precision
  • Reproducibility of Results
  • Blood Glucose Self-Monitoring/instrumentation
  • Biosensing Techniques/instrumentation
  • Humans
  • Materials Testing
  • Diabetes Mellitus, Type 1/blood
  • Blood Glucose/analysis

Fingerprint

Dive into the research topics of 'Performance comparison of CGM systems: MARD values are not always a reliable indicator of CGM system accuracy'. Together they form a unique fingerprint.

Cite this