A new approach to optical sensing, an increasingly in-demand technology

In the past decade, optical sensing tasks have become more demanding. As a result, it has become critical to build miniaturized, inexpensive sensors that can be integrated on-chip to enable mobile applications in smart phones, autonomous vehicles, robots, and drones. Also, algorithms are playing an increasingly important role in sensing, and many recent developments have utilized machine-learning algorithms.

In a new paper in Science, researchers in the lab of Prof. Fengnian Xia in Electrical Engineering introduce a new concept they call geometric optical deep sensing. The concept, which leverages innovations in device technology, condensed matter physics and deep learning, has the potential to move away from hardware-oriented approaches to software-oriented ones.

The paper was authored with collaborators at University of Texas, Bar-Ilan University of Israel and Vienna University of Technology of Austria. In this new concept, “geometric” indicates that the sensor outputs consist of multi-element data, which can be viewed as points in high dimensional vector spaces. “Deep” highlights the critical role of deep neural networks in this sensing scheme.

Shaofan Yuan, a former Ph.D. student in Xia’s lab and co-lead author of the paper, noted that conventional optical sensing requires multiple optical devices to fully capture the unknown properties of the light beam. These include different devices to measure the intensity, polarization, wavelengths, and the spatial distribution of the light. All these devices add up and make for a bulky and expensive system.

“Much effort has been made to make optical sensing devices compact and multifunctional in the past, and advanced machine learning algorithms have accelerated optical sensing solutions using miniaturized devices,” said Yuan, who added that future optical sensing technologies will be a highly interdisciplinary field. “This field will benefit from innovations in device structures, demonstrations of emerging optical and optoelectronic phenomena, and advancement in machine-learning algorithms.”

Chao Ma, a Ph.D. student in Xia’s lab and the other co-lead author of the paper, noted that device reconfigurability is key to achieving complicated optical sensing with a single device.

“A single reconfigurable device that can be operated at different states is essential to generate a multi-element photoresponse data capturing multiple unknown properties of light sometimes in an implicit manner, and then machine-learning algorithms can be used to interpret the data,” Ma said.

The scheme involves the use of reconfigurable sensors and deep neural networks for the information encoding/decoding processes. That is, the networks have been trained with known properties of light and can extract the right information from the multi-element outputs of reconfigurable sensors. Xia notes that it interprets the multi-element photoresponses much the way that image recognition programs do.

“If you want it to recognize an image, whether it’s a dog or cat or human beings or cars, you collect numerous photos with known information, then train it,” he said. “Then we just give the neural network an unknown figure, and that will tell you. The similar idea is used here.”

The researchers noted that the underlying principle of the scheme applies not just to light but other areas—for instance, for sensing magnetic fields. Xia said he and his collaborators are currently looking at potential applications. One possibility is using such integrated sensing devices to make autonomous vehicles safer.