Images projected onto objects in the real world create impressive displays that educate and entertain. However, current projection mapping systems all have one common limitation: they only work well in the dark. In a study recently published in IEEE Transactions on Visualization and Computer Graphics, researchers from Osaka University suggest a way to bring projection mapping “into the light.”
Conventional projection mapping, which turns any three-dimensional surface into an interactive display, requires darkness because any illumination in the surroundings also illuminates the surface of the target object used for display. This means that black and dark colors appear too bright and can’t be displayed properly.
In addition, the projected images always look like they are glowing, but not all real objects are luminous, which restricts the range of objects that can be displayed. Displays in dark environments have another disadvantage. Multiple viewers can interact with an illuminated scene, but they are less able to interact with each other in a dark environment.
“To get around this problem, we use projectors to reproduce normal illumination on every part of the room except the display object itself,” says Masaki Takeuchi, lead author of the study. “In essence, we create the illusion of global illumination without using actual global illumination.”
Projecting global illumination requires a set of techniques that differs from those of conventional projection mapping. The research team used several standard projectors to illuminate the room, along with a projector with a wide aperture and a large-format lens to soften the crisp edges of shadows. These luminaire projectors illuminate the environment, but the target object remains in shadow. Conventional texture projectors are then used to map the texture onto its shadowed surface.
Illumination reproduction results. © Osaka University
Reproducing mirror-like material using projection mapping under various illumination conditions. © Osaka University
The researchers built a prototype environment and evaluated the performance of their approach. One aspect they evaluated was whether humans perceived the objects in aperture-color mode (where the colors appear to radiate from the object itself) or surface-color mode (in which the light appears to be reflected from a colored surface).
“To our knowledge, we are the first to consider this,” says Daisuke Iwai, senior author of the study. “However, we believe it is fundamental for producing realistic environments.”
The researchers found that using their method, they could project texture images onto objects without making the objects appear to glow. Instead, the textures were perceived to be the true colors of the object’s surface.
In the future, the researchers plan to add more projectors to handle the complex illumination in the areas next to the display object. Eventually, they aim to produce scenes that are indistinguishable from real-world three-dimensional scenes.
They believe that this approach will enable visual design environments for industrial products or packaging in which the participants can interact not only with their design under natural light but also with each other, facilitating communication and improving design performance.
More information:
Masaki Takeuchi et al, Projection Mapping under Environmental Lighting by Replacing Room Lights with Heterogeneous Projectors, IEEE Transactions on Visualization and Computer Graphics (2024). DOI: 10.1109/TVCG.2024.3372031
Citation:
Projection mapping leaves the darkness behind (2024, March 18)