A novel mixed light field technique that utilizes a mix of ray-controlled ambient lighting with projection mapping (PM) to obtain PM in bright surroundings has been developed by scientists at Tokyo Institute of Technology. This innovative technology utilizes a novel kaleidoscope array to achieve ray-controlled lighting and a binary search algorithm for removing ambient lighting from PM targets. It provides an immersive augmented reality experience with applications in various fields.
The work is published in the journal IEEE Transactions on Visualization and Computer Graphics.
Projection mapping (PM) is a fascinating technology that provides an immersive visual experience by projecting computer-generated images onto physical surfaces, smoothly merging real and virtual worlds. It allows us to experience augmented reality without the need for special glasses. As a result, PM is in high demand in various fields including enhanced stage productions, trying on clothing and makeup, and educational demonstrations.
Despite its potential, current PM methods face challenges in bright environments with ambient lighting. Ambient lighting drowns the entire scene in light, reducing the contrast of PM targets. This is why conventional PM solutions mainly function in dark environments. However, even in dark environments, PM fails to provide a natural scene; in a dimly lit scene, only the PM target is well-lit while the rest of the surroundings remain dark, causing it to appear overly bright. Additionally, non-PM objects appear too dark, breaking the immersion.
To address these issues, a team of researchers from Japan, led by Associate Professor Yoshihiro Watanabe from the Department of Information and Communications Engineering at Tokyo Institute of Technology, has recently developed an innovative new mixed light field approach for achieving PM in brightly lit environments.
“In this approach, instead of using normal ambient light, we introduced a mixed light field in which a ray-controllable light avoids the PM target while adequately lighting other areas within the scene while the PM projector exclusively illuminates the target,” explains Dr. Watanabe. Their findings will be presented at the 31st IEEE Conference on Virtual Reality and 3D User Interfaces in Orlando, Florida U.S., with the presentation scheduled for March 19 at 1:30 PM local time (UTC-4).
At the center of this novel approach lies the ray-controllable lighting unit. This unit reproduces a range of ambient lighting scenarios while also avoiding illuminating the PM target. To achieve this, the researchers developed a novel kaleidoscopic array using an array of mirrors positioned behind a lens array, which, in turn, was placed in front of a projector. This setup allowed the projector to produce a high-density light field, crucial for ray-controllable lighting.
Furthermore, to avoid illuminating the PM target, the researchers deployed a camera to capture images of the scene and identify the pixels from the projector that illuminated the PM targets, subsequently turning them off. To identify these pixels, they employed a simple binary-search-based method, resulting in effective mixed light field.
This innovative approach allowed them to achieve high-contrast PM presentations in brightly lit surroundings. Notably, it preserved the natural appearance and shadows of ordinary non-PM objects, addressing a key challenge in PM technology. Through several captivating augmented scenes, the researchers showcased the seamless coexistence of PM targets and ordinary objects, providing an immersive visual experience.
The entire configuration of the system (left) : from left to right, the projection target, mirror-lens array, projector for projection mapping, and projector for illumination. A close-up of the mirror-lens array. This configuration reflects rays emerging from the lens array, leading to an increased ray density (right). © Assocaite Professor Yoshihiro Watanabe
While the researchers identified some limitations, such as artifacts and the low efficiency of binary search algorithms with large PM targets, they have already identified potential solutions and are actively working to expand this approach in the future.
“Our experiments prove the effectiveness of using this technique for achieving natural PM presentations with accurate lighting for all objects. Mixed light field has the potential to usher PM for various practical day-to-day applications, such as attractions, for support in manufacturing and trying on makeup,” says Dr. Watanabe, highlighting the applications of their technology.
Overall, this approach marks a significant step for PM technology, paving the way for immersive augmented experiences in future.
More information:
Masahiko Yasui et al, Projection Mapping with a Brightly Lit Surrounding Using a Mixed Light Field Approach, IEEE Transactions on Visualization and Computer Graphics (2024). DOI: 10.1109/TVCG.2024.3372132
Provided by
Tokyo Institute of Technology
Citation:
An innovative mixed light field technique for immersive projection mapping (2024, March 18)