New imaging technologies achieve real-time boundary detection, single-shot high-dynamic range imaging
Patent-pending imaging technologies created in Purdue University’s College of Engineering could be developed and commercialized for applications as diverse as medical imaging, autonomous navigation, surveillance, microscopy and advanced manufacturing.
Qi Guo, assistant professor in the Elmore Family School of Electrical and Computer Engineering, leads teams that have developed two technologies:
- CT-Bound, a fast and robust boundary detection method. It enables the discovery of structural information from extremely noisy visual data.
- MetaHDR, a single-shot high-dynamic range (HDR) imaging and sensing system. It eliminates the need for sequential captures in conventional HDR imaging.
A working prototype of CT-Bound was demonstrated and a research paper was presented at the Institute of Electrical and Electronics Engineers’ (IEEE) 26th International Workshop on Multimedia Signal Processing. A research paper about MetaHDR was published in the peer-reviewed journal Optics Express and was highlighted in Optica’s “Spotlight on Optics.”
Guo disclosed the innovations to the Purdue Innovates Office of Technology Commercialization, which has applied for patents to protect the intellectual property. Industry partners interested in developing or commercializing the work should contact Matt Halladay, senior business development manager and licensing manager — physical sciences, at mrhalladay@prf.org about track codes 70592 and 70751.
About CT-Bound
Guo said many applications require boundary detection from images with very low light levels, including autonomous navigation, manufacturing and medical imaging. He said detecting boundary structures from very noisy images is a common and challenging computer vision problem.
“Although image boundary detection has been broadly studied since the early stage of computer vision, the accuracy of current best boundary detection algorithms is still unsatisfactory when the input images have a very low light level,” he said.
CT-Bound decomposes boundary estimation into two tasks: local detection and global regularization.
“During the local detection, the model uses a convolutional architecture to predict the boundary structure of each image patch in the form of a predefined local boundary representation, the field of junctions,” he said. “Then it uses a feed-forward transformer architecture to globally refine the boundary structures of each patch to generate an edge map and a smoothed color map simultaneously.”
A thorough experimental study demonstrated CT-Bound achieved the highest or among the highest accuracy in detecting image boundaries from very noisy images compared to the previous best algorithms. The code, training data, testing data, additional results and a video demonstration of the proposed method are online.
“We also demonstrated that CT-Bound produces boundary and color maps on real captured images without extra fine-tuning and real-time boundary map and color map videos at 10 frames per second,” Guo said.
About MetaHDR
Guo said single-shot HDR imaging and sensing can be broadly applied where the full brightness profile of a moving environment needs to be measured, including in advanced manufacturing, autonomous vehicles, microscopic imaging and videography.
When objects with motion are measured, ghosting artifacts in the final reconstruction can occur; thus sophisticated postprocessing algorithms are required.
“Building an imaging system that can measure multiple exposure-bracketed images simultaneously could help avoid this tedious effort,” Guo said. “This can be critical when accurately measuring the HDR signal of dynamic scenes are of paramount importance, like capturing the fluorescence flickering in fluorescent microscopy or detecting faults on moving production lines.”
MetaHDR is a single-shot HDR imaging and sensing system that uses a multifunctional metasurface.
“The metasurface can split an incident beam into multiple focusing beams with different amounts of power, simultaneously forming multiple low-dynamic range (LDR) images with distinct irradiance on a photosensor,” Guo said. “Then the LDR images are jointly processed using a gradient-based HDR fusion algorithm, which is shown to be effective in attenuating the residual light artifacts incurred by the metasurface and the lens flare.”
The Purdue MetaHDR system captures HDR images with a single exposure or few exposures and creates multiple LDR images simultaneously.
“Current solutions for generating single-capture HDR images require specialized photosensors and circuitry,” Guo said. “MetaHDR utilizes more conventional camera and sensor technologies.”
Single-shot HDR photography and videography with real-time sensing applications make MetaHDR particularly useful for surveillance, microscopy and advanced manufacturing.
“We validated MetaHDR through experimentation using a fabricated metasurface,” Guo said. “Results showed more than a 50 decibel increase in dynamic range compared to most recent hardware-driven solutions to HDR imaging.”
Next development steps
Wei Xu, a Purdue electrical and computer engineering doctoral student, said the working prototype of CT-Bound achieves real-time boundary detection under high-noise conditions. The team has expanded on the technology into a broader algorithm called Blurry-Edges, which simultaneously performs boundary detection and depth estimation from noisy images.
“For this, we built a hardware prototype using a deformable lens to capture low-light image pairs with varying optical powers,” Xu said. “These advances set the stage for the next steps to be transitioning from research prototypes to deployable tools that can be integrated into industry imaging products.”
A follow-up paper that leverages CT-Bound-style image representation to measure object distances from low-light photographs has been accepted for this year’s Conference on Computer Vision and Pattern Recognition; it is scheduled to be published in June.
Yuxuan Liu, a Purdue electrical and computer engineering graduate student, said the next step to commercializing MetaHDR is to enable full-color imaging with an enlarged field of view.
“Building on recent advances in computational imaging and optical metasurfaces, we are exploring the integration of artificial intelligence, meta-optics and refractive lenses to achieve this goal,” Liu said. “This holds significant potential for reconstructing higher-dimensional information, such as spectrum, depth and polarization in a snapshot.”
Guo’s team is currently collaborating with Jian Jin, associate professor in Purdue’s Department of Agricultural and Biological Engineering, to explore MetaHDR’s potential use in agricultural phenotyping.
The team received support from Samsung Research America’s Global Research Outreach program to develop MetaHDR. They are seeking partnerships with industry collaborators, such as imaging device manufacturers and health care tech companies, to evaluate both technologies in real-life scenarios.