Only Trust a Hidden Wizard: Investigating the Effects of Wizard Visibility in Automotive Wizard of Oz Studies

The Wizard-of-Oz method has been widely used recently as it allows mimicking automated vehicles with relatively few resources. In some studies, it is challenging to ensure that the wizard remains fully hidden from participants, despite this being a crucial aspect of such experiments. To determine whether participants' awareness of the wizard influences the outcomes of these studies, we conducted an experiment investigating participants' crossing behavior and subjective perception of a remote-controlled automated vehicle. Participants were exposed to two conditions: in one, they solely focused on a simulated vehicle driving autonomously; in the other, they observed a wizard with a remote control and were instructed to imagine the car was automated. Results, based on scales for user experience, acceptance, and trust, as well as crossing behavior, indicate similar results. However, participants' knowledge of the wizard necessitates careful interpretation when system errors are simulated. We conclude with recommendations for future Wizard-of-Oz experiments.