Enhancing the Supervision of Out-of-View Robots: A Study on Multimodal Feedback and Monitoring Screens

Khaled Kassem, Ambika Shahu, Christina Tüchler, Philipp Wintersberger, Florian Michahelles

Research output: Chapter in Book/Report/Conference proceedingsConference contributionpeer-review


Objective: investigating the effect of two support methods (multimodal feedback, monitoring screens, and a combination of both) on human dual-task performance, cognitive workload, and user experience when supervising an out-of-sight autonomous robot. Method: A 2x2 within-group user study was conducted in VR with 26 participants involving a cognitive-cognitive dual-task setting. Participants had to simultaneously solve math problems and supervise the robot. Different support methods were provided: multimodal feedback, a screen showing real-time robot activity, and a combination of both. Objective performance metrics and subjective feedback on cognitive load and user experience were collected using standard questionnaires. Data were statistically analyzed, and thematic analysis was performed on post-study debriefing interviews. Results: The support methods improved overall user experience and positively impacted robot collaboration performance while decreasing math task performance. Cognitive load was unaffected. Multimodal feedback with a monitoring screen was perceived as the most helpful. Conclusion: The results suggest that multimodal feedback can improve user experience and improve supervision, but may partially decrease primary task performance. The findings highlight the importance of examining the effect of support methods in specific situations, depending on task priority.

Original languageEnglish
Title of host publicationMensch und Computer 2023
Subtitle of host publicationBuilding Bridges - Proceedings
EditorsMarkus Stolze, Frieder Loch, Matthias Baldauf, Florian Alt, Christina Schneegass, Thomas Kosch, Teresa Hirzle, Shadan Sadeghian, Fiona Draxler, Kenan Bektas, Katrin Lohan, Pascal Knierim
PublisherAssociation for Computing Machinery
Number of pages5
ISBN (Electronic)9798400707711
Publication statusPublished - 3 Sept 2023
Event2023 Mensch und Computer Conference, MuC 2023 - Rapperswil, Switzerland
Duration: 3 Sept 20236 Sept 2023

Publication series

NameACM International Conference Proceeding Series


Conference2023 Mensch und Computer Conference, MuC 2023


Dive into the research topics of 'Enhancing the Supervision of Out-of-View Robots: A Study on Multimodal Feedback and Monitoring Screens'. Together they form a unique fingerprint.

Cite this