Over the past few decades, computer scientists have developed a wide array of models and approaches to analyze different aspects of human behavior and communication, such as speech, emotions and gestures. Most existing techniques for hand gesture recognition rely on the use of wearable technologies with a single sensor and can only recognize a limited number of basic gestures.
Researchers at Comenius University Bratislava in Slovakia have recently developed WaveGlove, a new system for hand gesture recognition that utilizes multiple inertial sensors, instead of a single sensor. This new system, introduced in a paper pre-published on arXiv, is essentially a glove with an inertial sensor placed on each of its fingers.
“Our recent paper showcases the use of multiple inertial sensors for hand gesture recognition (HGR),” Matej Kralik, one of the researchers who carried out the study, told TechXplore. “By building a custom hardware prototype and proposing a novel Transformer-based model (a network architecture, which had a lot of success on natural language processing tasks), we demonstrate that using multiple sensors can have significant effect on the classification accuracy and allows for a richer vocabulary of gestures.”
Using WaveGlove, the hand gesture recognition system they created, the researchers were able to acquire two datasets containing over 11000 hand gesture samples. Kralik and his colleagues then designed two different gesture vocabularies, one containing 8 whole-hand movements (WaveGlove-single) and another containing 10 more complex, carefully designed hand gestures, for which individual fingers move differently (WaveGlove-multi).
“The presence of multiple sensors allows us to design and classify a richer vocabulary of gestures in comparison to single handheld sensors,” Kralik said. “Gestures we classify using multiple sensors are similar to those we already use in our daily lives. This makes the use of a device like WaveGlove easier and more natural.”
In addition to compiling datasets of hand gestures and confirming the effectiveness of the WaveGlove system, the researchers benchmarked over 10 classification methods for hand gesture recognition, some of which they had developed as part of their previous research. They evaluated these methods on numerous different datasets, as they hoped that this would help to standardize results in the field of hand gesture recognition.
The findings gathered by this team of researchers could have numerous valuable implications. Firstly, their work could inform research focusing on hand gesture recognition and help to enhance existing systems. Secondly, the WaveGlove prototype they created could ultimately be used to improve communications between humans and machines, allowing the latter to better interpret human hand gestures.
“To the best of our knowledge, we provide the first publicly available multi-sensor dataset of significant size in the area of Hand Gesture Recognition using inertial sensors,” Kralik said. “We also demonstrate that the optimal amount and placement of the sensors depends on the gestures being classified. We now aim to explore the further fine-tuning of the Transformer-based model we examined, increasing the size and variability of the dataset and expanding the multi-sensor vocabulary of gestures.”
For twins, gesture and speech go hand-in-hand in language development
More information:
WaveGlove: Transformer-based hand gesture recognition using multiple inertial sensors. arXiv:2105.01753 [cs.HC]. arxiv.org/abs/2105.01753
2021 Science X Network
Citation:
WaveGlove: A glove with five inertial sensors for hand gesture recognition (2021, June 15)
retrieved 15 June 2021
from https://techxplore.com/news/2021-06-waveglove-glove-inertial-sensors-gesture.html
This document is subject to copyright. Apart from any fair dealing for the purpose of private study or research, no
part may be reproduced without the written permission. The content is provided for information purposes only.
