In recent years, videogame developers and computer scientists have been trying to devise techniques that can make gaming experiences increasingly immersive, engaging and realistic. These include methods to automatically create videogame characters inspired by real people.
Most existing methods to create and customize videogame characters require players to adjust the features of their character’s face manually, in order to recreate their own face or the faces of other people. More recently, some developers have tried to develop methods that can automatically customize a character’s face by analyzing images of real people’s faces. However, these methods are not always effective and do not always reproduce the faces they analyze in realistic ways.
Researchers at Netease Fuxi AI Lab and University of Michigan have recently created MeInGame, a deep learning technique that can automatically generate character faces by analyzing a single portrait of a person’s face. This technique, presented in a paper pre-published on arXiv, can be easily integrated into most existing 3D videogames.
“We propose an automatic character face creation method that predicts both facial shape and texture from a single portrait, and can be integrated into most existing 3D games,” Jiangke Lin, Yi Yuan and Zhengxia Zou, the three researchers who carried out the study, wrote in their paper.
Some of the automatic character customization systems presented in previous works are based on computational techniques known as 3D morphable face models (3DMMs). While some of these methods have been found to reproduce a person’s facial features with good levels of accuracy, the way in which they represent geometrical properties and spatial relations (i.e., topology) often differs from the meshes utilized in most 3D videogames.
In order for 3DMMs to reproduce the texture of a person’s face reliably, they typically need to be trained on large datasets of images and on related texture data. Compiling these datasets can be fairly time consuming. Moreover, these datasets do not always contain real images collected in the wild, which can prevent models trained on them from performing consistently well when presented with new data. To overcome this limitation, Lin, Yuan and Zou trained their technique on a dataset of images captured in the wild.
“Given an input face photo, we first reconstruct a 3D face based on a 3D morphable face model (3DMM) and convolutional neural networks (CNNs), then transfer the shape of the 3D face to the template mesh,” the researchers explained in their paper. “The proposed network takes the face photo and the unwrapped coarse UV texture map as input, then predicts lighting coefficients and refined texture maps.”
Lin, Yuan and Zou evaluated their deep learning technique in a series of experiments, comparing the quality of the game characters it generated with that of character faces produced by other existing state-of-the-art methods for automatic character customization. Their method performed remarkably well, generating character faces that closely resembled those in input images.
“The proposed method does not only produce detailed and vivid game characters similar to the input portrait, but it can also eliminate the influence of lighting and occlusions,” the researchers wrote in their paper. “Experiments show that our method outperforms state-of-the-art methods used in games.”
In the future, the character face generation method devised by this team of researchers could be integrated within a number of 3D videogames, enabling the automatic creation of characters that closely resemble real people. The MeInGame model’s code and the dataset used to train it were published online and can be accessed by game developers worldwide at: github.com/FuxiCV/ MeInGame .
A system that automatically generates comic books from movies and other videos
More information:
MeInGame: Create a game character face from a single portrait. arXiv:2102.02371 [cs.CV]. arxiv.org/abs/2102.02371
2021 Science X Network
Citation:
MeInGame: A deep learning method to create videogame characters that look like real people (2021, March 4)
retrieved 4 March 2021
from https://techxplore.com/news/2021-03-meingame-deep-method-videogame-characters.html
This document is subject to copyright. Apart from any fair dealing for the purpose of private study or research, no
part may be reproduced without the written permission. The content is provided for information purposes only.
