Simo: Interactions with distant displays by smartphones with simultaneous face and world tracking

Teo Babic, Florian Perteneder, Harald Reiterer, Michael Haller

Research output: Chapter in Book/Report/Conference proceedingsConference contributionpeer-review

1 Citation (Scopus)

Abstract

The interaction with distant displays often demands complex, multi-modal inputs which need to be achieved with a very simple hardware solution so that users can perform rich inputs wherever they encounter a distant display. We present Simo, a novel approach, that transforms a regular smartphone into a highly-expressive user motion tracking device and controller for distant displays. Both the front and back cameras of the smartphone are used simultaneously to track the user's hand as well as the head, and body movements in real-world space and scale. In this work, we first define the possibilities for simultaneous face- and world-tracking using current off-the-shelf smartphones. Next, we present the implementation of a smartphone app enabling hand, head, and body motion tracking. Finally, we present a technical analysis outlining the possible tracking range.

Original languageEnglish
Title of host publicationCHI EA 2020 - Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems
PublisherAssociation for Computing Machinery
Pages1-12
Number of pages12
ISBN (Electronic)9781450368193
DOIs
Publication statusPublished - 25 Apr 2020
Event2020 ACM CHI Conference on Human Factors in Computing Systems, CHI EA 2020 - Honolulu, United States
Duration: 25 Apr 202030 Apr 2020

Publication series

NameConference on Human Factors in Computing Systems - Proceedings

Conference

Conference2020 ACM CHI Conference on Human Factors in Computing Systems, CHI EA 2020
CountryUnited States
CityHonolulu
Period25.04.202030.04.2020

Keywords

  • 3D interaction
  • Augmented reality
  • Distant displays
  • Mobile devices
  • Motion tracking
  • Smartphone
  • Spatial interaction

Fingerprint Dive into the research topics of 'Simo: Interactions with distant displays by smartphones with simultaneous face and world tracking'. Together they form a unique fingerprint.

Cite this