Online Handwriting Recognition using LSTM on Microcontroller and IMU Sensors

Daniel Lauß; Florian Eibensteiner; Josef Langer; Phillip Michael Christian Fred Petz

Online Handwriting Recognition using LSTM on Microcontroller and IMU Sensors

Daniel Lauß, Florian Eibensteiner, Josef Langer, Phillip Michael Christian Fred Petz

Publikation: Konferenzbeitrag › Papier › Begutachtung

Abstract

In this paper, we demonstrate an inertial measurement unit (IMU) based hand gesture recognition (HGR) on a lowpower microcontroller (STM32L476JGY) by using deep neural networks (DNN). To train the DNNs, a dataset was recorded which contains accelerometer and gyroscope readings from three IMUs mounted on the fingertips. With this dataset, various neural networks (NN) were trained and analysed. The best NN, in terms of accuracy and memory usage, was then selected and ported to the microcontroller. Finally, a runtime analysis of the model has been performed on the controller. The analysis showed that a LSTM is best suited for the detection of hand gestures. The selected model achieves an accuracy of 93 % and only takes up around 40 KiB of memory. In addition, the model has a throughput time of only 3.52 ms, which means that the prototype can be used in real time.

Originalsprache	Englisch
Publikationsstatus	Angenommen/Im Druck - 2 Sep. 2022
Veranstaltung	21st IEEE International Conference on Machine Learning and Applications - Nassau, Bahamas Dauer: 12 Dez. 2022 → 15 Dez. 2022 https://www.icmla-conference.org/icmla22/howtosubmit.html

Konferenz

Konferenz	21st IEEE International Conference on Machine Learning and Applications
Kurztitel	ICSIMA
Land/Gebiet	Bahamas
Ort	Nassau
Zeitraum	12.12.2022 → 15.12.2022
Internetadresse	https://www.icmla-conference.org/icmla22/howtosubmit.html

Zitieren

@conference{23cccb9d08a84a7d940ac51ad27884da,

title = "Online Handwriting Recognition using LSTM on Microcontroller and IMU Sensors",

abstract = "In this paper, we demonstrate an inertial measurement unit (IMU) based hand gesture recognition (HGR) on a lowpower microcontroller (STM32L476JGY) by using deep neural networks (DNN). To train the DNNs, a dataset was recorded which contains accelerometer and gyroscope readings from three IMUs mounted on the fingertips. With this dataset, various neural networks (NN) were trained and analysed. The best NN, in terms of accuracy and memory usage, was then selected and ported to the microcontroller. Finally, a runtime analysis of the model has been performed on the controller. The analysis showed that a LSTM is best suited for the detection of hand gestures. The selected model achieves an accuracy of 93 % and only takes up around 40 KiB of memory. In addition, the model has a throughput time of only 3.52 ms, which means that the prototype can be used in real time.",

keywords = "HGR, DNN, LSTM, Microcontroller, IMU",

author = "Daniel Lau{\ss} and Florian Eibensteiner and Josef Langer and Petz, {Phillip Michael Christian Fred}",

year = "2022",

month = sep,

day = "2",

language = "English",

note = "21st IEEE International Conference on Machine Learning and Applications, ICSIMA ; Conference date: 12-12-2022 Through 15-12-2022",

url = "https://www.icmla-conference.org/icmla22/howtosubmit.html",

}

TY - CONF

T1 - Online Handwriting Recognition using LSTM on Microcontroller and IMU Sensors

AU - Lauß, Daniel

AU - Eibensteiner, Florian

AU - Langer, Josef

AU - Petz, Phillip Michael Christian Fred

PY - 2022/9/2

Y1 - 2022/9/2

N2 - In this paper, we demonstrate an inertial measurement unit (IMU) based hand gesture recognition (HGR) on a lowpower microcontroller (STM32L476JGY) by using deep neural networks (DNN). To train the DNNs, a dataset was recorded which contains accelerometer and gyroscope readings from three IMUs mounted on the fingertips. With this dataset, various neural networks (NN) were trained and analysed. The best NN, in terms of accuracy and memory usage, was then selected and ported to the microcontroller. Finally, a runtime analysis of the model has been performed on the controller. The analysis showed that a LSTM is best suited for the detection of hand gestures. The selected model achieves an accuracy of 93 % and only takes up around 40 KiB of memory. In addition, the model has a throughput time of only 3.52 ms, which means that the prototype can be used in real time.

AB - In this paper, we demonstrate an inertial measurement unit (IMU) based hand gesture recognition (HGR) on a lowpower microcontroller (STM32L476JGY) by using deep neural networks (DNN). To train the DNNs, a dataset was recorded which contains accelerometer and gyroscope readings from three IMUs mounted on the fingertips. With this dataset, various neural networks (NN) were trained and analysed. The best NN, in terms of accuracy and memory usage, was then selected and ported to the microcontroller. Finally, a runtime analysis of the model has been performed on the controller. The analysis showed that a LSTM is best suited for the detection of hand gestures. The selected model achieves an accuracy of 93 % and only takes up around 40 KiB of memory. In addition, the model has a throughput time of only 3.52 ms, which means that the prototype can be used in real time.

KW - HGR

KW - DNN

KW - LSTM

KW - Microcontroller

KW - IMU

M3 - Paper

T2 - 21st IEEE International Conference on Machine Learning and Applications

Y2 - 12 December 2022 through 15 December 2022

ER -

Online Handwriting Recognition using LSTM on Microcontroller and IMU Sensors

Abstract

Konferenz

Fingerprint

Zitieren