TY - GEN
T1 - Proposing the use of an 'Advocatus Diaboli' as a Pragmatic Approach to Improve Transparency in Qualitative Data Analysis and Reporting
AU - Friedl-Knirsch, Judith
N1 - Publisher Copyright:
© 2024 IEEE.
PY - 2024
Y1 - 2024
N2 - Qualitative data analysis is widely adopted for user evaluation, not only in the Visualisation community but also related communities, such as Human-Computer Interaction and Augmented and Virtual Reality. However, the data analysis process is often not clearly described and the results are often simply listed in the form of in-teresting quotes from or summaries of quotes that were uttered by study participants. This position paper proposes an early concept for the use of a researcher as an 'Advocatus Diaboli', or devil's advocate, to try to disprove the results of the data analysis by looking for quotes that contradict the findings or leading questions and task designs. Whatever this devil's advocate finds can then be used to reiterate on the findings and the analysis process to form more suitable theories. On the other hand, researchers are enabled to clarify why they did not include this in their theory. This process could increase transparency in the qualitative data analysis process and increase trust in these findings, while being mindful of the necessary resources.
AB - Qualitative data analysis is widely adopted for user evaluation, not only in the Visualisation community but also related communities, such as Human-Computer Interaction and Augmented and Virtual Reality. However, the data analysis process is often not clearly described and the results are often simply listed in the form of in-teresting quotes from or summaries of quotes that were uttered by study participants. This position paper proposes an early concept for the use of a researcher as an 'Advocatus Diaboli', or devil's advocate, to try to disprove the results of the data analysis by looking for quotes that contradict the findings or leading questions and task designs. Whatever this devil's advocate finds can then be used to reiterate on the findings and the analysis process to form more suitable theories. On the other hand, researchers are enabled to clarify why they did not include this in their theory. This process could increase transparency in the qualitative data analysis process and increase trust in these findings, while being mindful of the necessary resources.
KW - evaluation methodology
KW - Qualitative data analysis
UR - http://www.scopus.com/inward/record.url?scp=85212427050&partnerID=8YFLogxK
U2 - 10.1109/BELIV64461.2024.00005
DO - 10.1109/BELIV64461.2024.00005
M3 - Conference contribution
AN - SCOPUS:85212427050
T3 - Proceedings - 2024 IEEE Evaluation and Beyond - Methodological Approaches for Visualization, BELIV 2024
SP - 1
EP - 4
BT - Proceedings - 2024 IEEE Evaluation and Beyond - Methodological Approaches for Visualization, BELIV 2024
PB - Institute of Electrical and Electronics Engineers Inc.
T2 - 2024 IEEE Evaluation and Beyond - Methodological Approaches for Visualization, BELIV 2024
Y2 - 14 October 2024
ER -