Investigating the Temporal Aspects of AI Explanations

Abstract

The use of AI systems in different fields like in a clinical setting has increased in the last years. With such time sensitive environments using AI support in decision making comes a need for research on how time pressure affects human-ai collaboration in such tasks. For this, our study uses Where’s Waldo? picture puzzles to mimic the complexity of medical applications. In our online study, N=239 participants were presented with picture puzzles along with an AI suggestion whether Waldo is visible in the image. These images were split up into easy and difficult images to also explore the effects of task difficulty on decisions. For half of the decisions, participants were also presented with a time limit. Additionally, participants were presented with different XAI information in a between subject design. N=60 participants were in the baseline group, N=60 participants were presented with the confidence value of the AI, N=60 participants with mock-up LIME visualizations, and N=59 participants with both LIME and the confidence value. We measured the response time, correctness of the response, compliance with the AI’s suggestion and the participant’s trust in the system in a questionnaire. The results of this study demonstrate a negative effect of time pressure on the correctness of responses and the confidence of participants. Additionally, the compliance with and trust in the suggestion increases. Adding XAI visualizations to the presented images improves the response times but also increases the trust in and compliance with AI suggestions, primarily for true positive and false negative suggestions. The number of correct responses, however, does not increase with these visual explanations. This compliance with false negative suggestions raises concerns for the use of visual XAI in clinical settings as this might lead to a doctor wrongfully accepting a suggestion that a patient does not have a tumour when they do.

Date of Award	2025
Original language	English
Supervisor	Andreas Ernst Riegler (Supervisor)