Skillab - A Multimodal Augmented Reality Environment for Learning Manual Tasks

Ambika Shahu; Sonja Dorfbauer; Philipp Wintersberger; Florian Michahelles

doi:10.1007/978-3-031-42286-7_33

Skillab - A Multimodal Augmented Reality Environment for Learning Manual Tasks

Ambika Shahu, Sonja Dorfbauer, Philipp Wintersberger, Florian Michahelles

Research output: Chapter in Book/Report/Conference proceedings › Conference contribution › peer-review

Abstract

The paper investigates the usage of AR-based systems in teaching manual skills. We propose Skillab, a novel AR-based scaffolding system. It aids in the learning of manual work and functions as a multimodal immersive tool for feedback, including muscle actuation. As our initial investigation, we made a floor lamination tutorial. We evaluated our system’s performance and user experience in comparison to traditional paper instructions. With 20 participants, we conducted a between-group user study and obtained both qualitative and quantitative data. In terms of task performance, learnability, and overall user experience, Skillab significantly outperformed conventional paper instructions. In contrast to learning from paper instructions, Skillab training demonstrated a significant improvement in the systematic rating on the quality of the performed task. We believe that by demonstrating the potential of immersive multi-modal feedback technology for skill-building, researchers would be motivated to explore this area further.

Original language	English
Title of host publication	Human-Computer Interaction – INTERACT 2023 - 19th IFIP TC13 International Conference, Proceedings
Editors	José Abdelnour Nocera, Marta Kristín Lárusdóttir, Helen Petrie, Antonio Piccinno, Marco Winckler
Publisher	Springer
Pages	588-607
Number of pages	20
ISBN (Print)	9783031422850
DOIs	https://doi.org/10.1007/978-3-031-42286-7_33
Publication status	Published - 2023
Event	19th IFIP TC 13 International Conference on Human-Computer Interaction, INTERACT 2023 - York, United Kingdom Duration: 28 Aug 2023 → 1 Sept 2023

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	14144 LNCS
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	19th IFIP TC 13 International Conference on Human-Computer Interaction, INTERACT 2023
Country/Territory	United Kingdom
City	York
Period	28.08.2023 → 01.09.2023

Keywords

Augmented Reality
Immersiveness
Muscle Actuation
Skill Building

UN SDGs

This output contributes to the following UN Sustainable Development Goals (SDGs)

Access to Document

10.1007/978-3-031-42286-7_33

Cite this

Shahu, A., Dorfbauer, S., Wintersberger, P., & Michahelles, F. (2023). Skillab - A Multimodal Augmented Reality Environment for Learning Manual Tasks. In J. Abdelnour Nocera, M. Kristín Lárusdóttir, H. Petrie, A. Piccinno, & M. Winckler (Eds.), Human-Computer Interaction – INTERACT 2023 - 19th IFIP TC13 International Conference, Proceedings (pp. 588-607). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 14144 LNCS). Springer. https://doi.org/10.1007/978-3-031-42286-7_33

Shahu, Ambika ; Dorfbauer, Sonja ; Wintersberger, Philipp et al. / Skillab - A Multimodal Augmented Reality Environment for Learning Manual Tasks. Human-Computer Interaction – INTERACT 2023 - 19th IFIP TC13 International Conference, Proceedings. editor / José Abdelnour Nocera ; Marta Kristín Lárusdóttir ; Helen Petrie ; Antonio Piccinno ; Marco Winckler. Springer, 2023. pp. 588-607 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{cc0ab7db886748958aeab81ce9c3072d,

title = "Skillab - A Multimodal Augmented Reality Environment for Learning Manual Tasks",

abstract = "The paper investigates the usage of AR-based systems in teaching manual skills. We propose Skillab, a novel AR-based scaffolding system. It aids in the learning of manual work and functions as a multimodal immersive tool for feedback, including muscle actuation. As our initial investigation, we made a floor lamination tutorial. We evaluated our system{\textquoteright}s performance and user experience in comparison to traditional paper instructions. With 20 participants, we conducted a between-group user study and obtained both qualitative and quantitative data. In terms of task performance, learnability, and overall user experience, Skillab significantly outperformed conventional paper instructions. In contrast to learning from paper instructions, Skillab training demonstrated a significant improvement in the systematic rating on the quality of the performed task. We believe that by demonstrating the potential of immersive multi-modal feedback technology for skill-building, researchers would be motivated to explore this area further.",

keywords = "Augmented Reality, Immersiveness, Muscle Actuation, Skill Building",

author = "Ambika Shahu and Sonja Dorfbauer and Philipp Wintersberger and Florian Michahelles",

note = "Publisher Copyright: {\textcopyright} 2023, The Author(s), under exclusive license to Springer Nature Switzerland AG.; 19th IFIP TC 13 International Conference on Human-Computer Interaction, INTERACT 2023 ; Conference date: 28-08-2023 Through 01-09-2023",

year = "2023",

doi = "10.1007/978-3-031-42286-7_33",

language = "English",

isbn = "9783031422850",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer",

pages = "588--607",

editor = "{Abdelnour Nocera}, Jos{\'e} and {Krist{\'i}n L{\'a}rusd{\'o}ttir}, Marta and Helen Petrie and Antonio Piccinno and Marco Winckler",

booktitle = "Human-Computer Interaction – INTERACT 2023 - 19th IFIP TC13 International Conference, Proceedings",

address = "Germany",

}

Shahu, A, Dorfbauer, S, Wintersberger, P & Michahelles, F 2023, Skillab - A Multimodal Augmented Reality Environment for Learning Manual Tasks. in J Abdelnour Nocera, M Kristín Lárusdóttir, H Petrie, A Piccinno & M Winckler (eds), Human-Computer Interaction – INTERACT 2023 - 19th IFIP TC13 International Conference, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 14144 LNCS, Springer, pp. 588-607, 19th IFIP TC 13 International Conference on Human-Computer Interaction, INTERACT 2023, York, United Kingdom, 28.08.2023. https://doi.org/10.1007/978-3-031-42286-7_33

Skillab - A Multimodal Augmented Reality Environment for Learning Manual Tasks. / Shahu, Ambika; Dorfbauer, Sonja; Wintersberger, Philipp et al.
Human-Computer Interaction – INTERACT 2023 - 19th IFIP TC13 International Conference, Proceedings. ed. / José Abdelnour Nocera; Marta Kristín Lárusdóttir; Helen Petrie; Antonio Piccinno; Marco Winckler. Springer, 2023. p. 588-607 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 14144 LNCS).

Research output: Chapter in Book/Report/Conference proceedings › Conference contribution › peer-review

TY - GEN

T1 - Skillab - A Multimodal Augmented Reality Environment for Learning Manual Tasks

AU - Shahu, Ambika

AU - Dorfbauer, Sonja

AU - Wintersberger, Philipp

AU - Michahelles, Florian

PY - 2023

Y1 - 2023

N2 - The paper investigates the usage of AR-based systems in teaching manual skills. We propose Skillab, a novel AR-based scaffolding system. It aids in the learning of manual work and functions as a multimodal immersive tool for feedback, including muscle actuation. As our initial investigation, we made a floor lamination tutorial. We evaluated our system’s performance and user experience in comparison to traditional paper instructions. With 20 participants, we conducted a between-group user study and obtained both qualitative and quantitative data. In terms of task performance, learnability, and overall user experience, Skillab significantly outperformed conventional paper instructions. In contrast to learning from paper instructions, Skillab training demonstrated a significant improvement in the systematic rating on the quality of the performed task. We believe that by demonstrating the potential of immersive multi-modal feedback technology for skill-building, researchers would be motivated to explore this area further.

AB - The paper investigates the usage of AR-based systems in teaching manual skills. We propose Skillab, a novel AR-based scaffolding system. It aids in the learning of manual work and functions as a multimodal immersive tool for feedback, including muscle actuation. As our initial investigation, we made a floor lamination tutorial. We evaluated our system’s performance and user experience in comparison to traditional paper instructions. With 20 participants, we conducted a between-group user study and obtained both qualitative and quantitative data. In terms of task performance, learnability, and overall user experience, Skillab significantly outperformed conventional paper instructions. In contrast to learning from paper instructions, Skillab training demonstrated a significant improvement in the systematic rating on the quality of the performed task. We believe that by demonstrating the potential of immersive multi-modal feedback technology for skill-building, researchers would be motivated to explore this area further.

KW - Augmented Reality

KW - Immersiveness

KW - Muscle Actuation

KW - Skill Building

UR - http://www.scopus.com/inward/record.url?scp=85173057545&partnerID=8YFLogxK

U2 - 10.1007/978-3-031-42286-7_33

DO - 10.1007/978-3-031-42286-7_33

M3 - Conference contribution

AN - SCOPUS:85173057545

SN - 9783031422850

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 588

EP - 607

BT - Human-Computer Interaction – INTERACT 2023 - 19th IFIP TC13 International Conference, Proceedings

A2 - Abdelnour Nocera, José

A2 - Kristín Lárusdóttir, Marta

A2 - Petrie, Helen

A2 - Piccinno, Antonio

A2 - Winckler, Marco

PB - Springer

T2 - 19th IFIP TC 13 International Conference on Human-Computer Interaction, INTERACT 2023

Y2 - 28 August 2023 through 1 September 2023

ER -

Shahu A, Dorfbauer S, Wintersberger P, Michahelles F. Skillab - A Multimodal Augmented Reality Environment for Learning Manual Tasks. In Abdelnour Nocera J, Kristín Lárusdóttir M, Petrie H, Piccinno A, Winckler M, editors, Human-Computer Interaction – INTERACT 2023 - 19th IFIP TC13 International Conference, Proceedings. Springer. 2023. p. 588-607. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-031-42286-7_33