TY - JOUR
T1 - Performance comparison of CGM systems
T2 - MARD values are not always a reliable indicator of CGM system accuracy
AU - Kirchsteiger, Harald
AU - Heinemann, Lutz
AU - Freckmann, Guido
AU - Lodwig, Volker
AU - Schmelzeisen-Redeker, Günther
AU - Schoemaker, Michael
AU - Del Re, Luigi
N1 - Publisher Copyright:
© 2015 Diabetes Technology Society.
Copyright:
Copyright 2021 Elsevier B.V., All rights reserved.
PY - 2015/9
Y1 - 2015/9
N2 - Background: The ongoing progress of continuous glucose monitoring (CGM) systems results in an increasing interest in comparing their performance, in particular in terms of accuracy, that is, matching CGM readings with reference values measured at the same time. Most often accuracy is evaluated by the mean absolute relative difference (MARD). It is frequently overseen that MARD does not only reflect accuracy, but also the study protocol and evaluation procedure, making a cross-study comparison problematic. Methods: We evaluate the effect of several factors on the MARD statistical properties: number of paired reference and CGM values, distribution of the paired values, accuracy of the reference measurement device itself and the time delay between data pairs. All analysis is done using clinical data from 12 patients wearing 6 sensors each. Results: We have found that a few paired points can have a potentially high impact on MARD. Leaving out those points for evaluation thus reduces the MARD. Similarly, accuracy of the reference measurements greatly affects the MARD as numerical and graphical data show. Results also show that a log-normal distribution of the paired references provides a significantly different MARD than, for example, a uniform distribution. Conclusions: MARD is a reasonable parameter to characterize the performance of CGM systems when keeping its limitations in mind. To support clinicians and patients in selecting which CGM system to use in a clinical setting, care should be taken to make MARD more comparable by employing a standardized evaluation procedure.
AB - Background: The ongoing progress of continuous glucose monitoring (CGM) systems results in an increasing interest in comparing their performance, in particular in terms of accuracy, that is, matching CGM readings with reference values measured at the same time. Most often accuracy is evaluated by the mean absolute relative difference (MARD). It is frequently overseen that MARD does not only reflect accuracy, but also the study protocol and evaluation procedure, making a cross-study comparison problematic. Methods: We evaluate the effect of several factors on the MARD statistical properties: number of paired reference and CGM values, distribution of the paired values, accuracy of the reference measurement device itself and the time delay between data pairs. All analysis is done using clinical data from 12 patients wearing 6 sensors each. Results: We have found that a few paired points can have a potentially high impact on MARD. Leaving out those points for evaluation thus reduces the MARD. Similarly, accuracy of the reference measurements greatly affects the MARD as numerical and graphical data show. Results also show that a log-normal distribution of the paired references provides a significantly different MARD than, for example, a uniform distribution. Conclusions: MARD is a reasonable parameter to characterize the performance of CGM systems when keeping its limitations in mind. To support clinicians and patients in selecting which CGM system to use in a clinical setting, care should be taken to make MARD more comparable by employing a standardized evaluation procedure.
KW - Accuracy
KW - CGM
KW - Continuous glucose monitoring
KW - MARD
KW - Performance comparison
KW - Performance evaluation
KW - Precision
KW - Reproducibility of Results
KW - Blood Glucose Self-Monitoring/instrumentation
KW - Biosensing Techniques/instrumentation
KW - Humans
KW - Materials Testing
KW - Diabetes Mellitus, Type 1/blood
KW - Blood Glucose/analysis
UR - http://www.scopus.com/inward/record.url?scp=84973312721&partnerID=8YFLogxK
U2 - 10.1177/1932296815586013
DO - 10.1177/1932296815586013
M3 - Article
C2 - 26330485
AN - SCOPUS:84973312721
SN - 1932-2968
VL - 9
SP - 1030
EP - 1040
JO - Journal of diabetes science and technology
JF - Journal of diabetes science and technology
IS - 5
ER -