Skillab - A Multimodal Augmented Reality Environment for Learning Manual Tasks

Ambika Shahu, Sonja Dorfbauer, Philipp Wintersberger, Florian Michahelles

Research output: Chapter in Book/Report/Conference proceedingsConference contributionpeer-review


The paper investigates the usage of AR-based systems in teaching manual skills. We propose Skillab, a novel AR-based scaffolding system. It aids in the learning of manual work and functions as a multimodal immersive tool for feedback, including muscle actuation. As our initial investigation, we made a floor lamination tutorial. We evaluated our system’s performance and user experience in comparison to traditional paper instructions. With 20 participants, we conducted a between-group user study and obtained both qualitative and quantitative data. In terms of task performance, learnability, and overall user experience, Skillab significantly outperformed conventional paper instructions. In contrast to learning from paper instructions, Skillab training demonstrated a significant improvement in the systematic rating on the quality of the performed task. We believe that by demonstrating the potential of immersive multi-modal feedback technology for skill-building, researchers would be motivated to explore this area further.

Original languageEnglish
Title of host publicationHuman-Computer Interaction – INTERACT 2023 - 19th IFIP TC13 International Conference, Proceedings
EditorsJosé Abdelnour Nocera, Marta Kristín Lárusdóttir, Helen Petrie, Antonio Piccinno, Marco Winckler
Number of pages20
ISBN (Print)9783031422850
Publication statusPublished - 2023
Event19th IFIP TC 13 International Conference on Human-Computer Interaction, INTERACT 2023 - York, United Kingdom
Duration: 28 Aug 20231 Sept 2023

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume14144 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349


Conference19th IFIP TC 13 International Conference on Human-Computer Interaction, INTERACT 2023
Country/TerritoryUnited Kingdom


  • Augmented Reality
  • Immersiveness
  • Muscle Actuation
  • Skill Building


Dive into the research topics of 'Skillab - A Multimodal Augmented Reality Environment for Learning Manual Tasks'. Together they form a unique fingerprint.

Cite this