February 17, 2021
feature
Tactile avatar is an artificial tactile perception and cognition system used as a surrogate for human tactile cognition with potential to produce smooth or soft and rough tactile sensations by its user. In a new report now published on Advanced Science, researchers used a piezoelectric tactile sensor to record varying physical information including pressure, temperature, hardness, sliding velocity and surface topography. In this work, Kyungsoo Kim and a team of scientists in information and communication engineering, neurology, brain and cognitive sciences in Korea engineered artificial tactile cognition by testing the tactile feelings of human participants to a variety of materials ranging from smooth or soft to rough. To account for the response variation among humans, Kim et al. designed a deep learning structure for personalization through training based on individualized histograms of tactile cognition while recording physical tactile information. The decision error of each avatar system was less than 2% for 42 different types of materials, where tactile data could be measured with 100 trials for each material. The tactile avatar machine categorized new experiences of materials based on the knowledge of the tactile training data to show a high correlation with the specific user’s approach. The scientists intend to propose an advanced method with tactile emotional exchange capabilities for advanced digital experiences in electronic devices.
Digital tactility
Innovations in electrical devices and signal processing have advanced the digital experiences based on the five human senses. For instance, virtual reality (VR) can provide unusual and auditory sensations, while augmented reality (AR) can provide more personalized experiences with 3-D spatial images and stereo sound across disciplines of entertainment and internet marketing. These technologies are also evolving to exchange emotions between humans and machines with considerable attention placed on tactile sensor-based technologies. In this work, the artificial tactile system provided ‘smooth/soft’ and ‘rough’ tactile sensations based on the user’s tactile feelings to develop a ‘tactile avatar.’ The method mimicked psychological tactile feelings based on a piezoelectric sensor system and a deep learning process. The human-like sensor and processing system provided an artificial tactile cognition system allowing the researchers to test the performance of the device for tactile decision-making and understand its performance with untrained or novel tactile materials.
Designing the tactile avatar
The human tactile system is complex and remains to be clarified in detail. To represent human tactile cognition, Kim et al. trained the system using individual tactile decision histograms. The setup obtained the baseline signal arising from touching and sliding processes in the parallel input layer. The tactile sensor generated signals relative to the hardness, temperature and surface features of the materials much like those made by humans. The slope and oscillation frequency of the touch signal contained information of the hardness and surface topography of the material. The team also built a decision processing system with combined neural network layers to emphasize specific features for sample classification. They processed two types of data on touching and sliding separately in the hidden layer. The neural network assigned multiple labels to different weights that reflected human tactile cognition. They expect the setup to have applications in online shopping and AR/VR environments for desirable tactile sensations. The process can also be included into an artificial skin system to resemble general human tactile feelings.
Understanding the human tactile system
To design a human-like tactile system, Kim et al. used 42 materials ranked from the smoothest to the softest and roughest, among 10 participants. The samples differed in surface structure, thickness and other characteristics selected from a large library of general materials used in clothing. The participants ranked the materials from 1 to 42, where 1 was smoothest and 42 the roughest. They averaged the test results and ordered the tactile materials accordingly. Human judgements were inconsistent during the ranking process and therefore the team accounted both tactile decision and tactile confusion to assess the sensitivity of humans during the sorting process. The researchers created a colored decision matrix to compare individual differences in tactile decision making in detail and included root-mean-square-error (RMSE) values. The results showed how a unique individuality supported tactile system will be required to mimic human tactile cognition.
Artificial sensor systems for deep learning and deep learning-based tactile discrimination
The team used multiarray tactile sensors made of piezoelectric materials to measure the surface information. They applied a machine-learning algorithm to produce artificial tactile feelings among 42 test tactile materials. The hardness of the material provided one of the main physical parameters to inform the deep learning process of tactile sensation in the study. The scientists used a durometer to measure this and determined the hardness based on piezoelectric signals. The researchers designed a tactile decision system based on the piezoelectric signals, contingent on a combination of neural network layers. They designed each parallel network to extract specific features and conduct complex classification processing in order to mimic human cognition for tactile classification based on tactile receptors. The researchers adopted labels to train neural networks and classified tactile materials with artificial sensors. During the training process, the team optimized the network to pick the correct tactile sample from among all untrained samples.
Optimized machine learning
The scientists then proposed a new approach to train the machine to make tactile decisions by using a human tactile decision histogram. To accomplish this, they mapped the human tactile decision histogram to the output nodes to train the machine, where the histogram contained information relative to the averaged tactile decision of the humans and decision confusion-based variance. Instead of focusing on higher classification accuracy, the researchers focused on mimicking human tactile decision making with the deep learning network. Kim et al. noted larger mean RMSE (root-mean-square-error) for human participants compared to the tactile decisions of the machine. The touching and sliding motions allowed the machine to compare a newly experienced material surface to multiple trained materials to classify new tactile materials based on pre-existing knowledge. Kim et al. formed the tactile avatar system on a neuromorphic system to reduce the time delay of computation and minimize the size of the machine to perform the realization.
In this way, Kyungsoo Kim and colleagues developed a tactile avatar system with a multiarray tactile sensor fabricated from piezoelectric materials and a deep learning process based on human tactile cognition. They expect future research to improve the capabilities of the tactile avatar to process tactile information, which will allow machines to replace humans in virtual spaces such as online shopping malls.
Psychosensory electronic skin technology for future AI and humanoid development
More information:
Kim K. et al. Tactile Avatar: Tactile Sensing System Mimicking Human Tactile Cognition, Advanced Science, doi.org/10.1002/advs.202002362
Chortos A. et al. Pursuing prosthetic electronic skin, Nature Materials, doi.org/10.1038/nmat4671
Rich S. I. et al. Untethered soft robotics, Nature Electronics, doi.org/10.1038/s41928-018-0024-1
2021 Science X Network
Citation:
Tactile avatar: Tactile sensing system mimicking human tactile cognition (2021, February 17)
retrieved 17 February 2021
from https://techxplore.com/news/2021-02-tactile-avatar-mimicking-human-cognition.html
This document is subject to copyright. Apart from any fair dealing for the purpose of private study or research, no
part may be reproduced without the written permission. The content is provided for information purposes only.
