Aim

The project aims to develop new computational algorithms to optimize the acquisition and processing of event data (e.g., single-photon data) and enable fast imaging in extreme environmental conditions using statistical and learning based approaches.

Objectives

1. Develop new computational algorithms to optimize the acquisition and processing of event data (e.g., single-photon data).
2. Interface of statistical modelling, optimization and deep learning to solve multimodal inverse problems for 3D imaging.
3. Enable 3D imaging at unprecedented scales: km range, kHz frame rates and Mega-pixels.

Description

This project will propose new algorithms for multimodal imaging to enable 3D imaging at unprecedented scales: km range, kHz frame rates and Mega-pixel format. Our image processing group will work closely with optical system design groups in Heriot-Watt to combine multi-sensor data acquired at high frame rates using a 3D Lidar system and other sensing modalities (passive, active, neuromorphic). This combination aims to reduce the noise affecting the 3D images (due to imaging through fog, rain, or other scattering media), and to improve the spatial resolution of 3D Lidar videos thus enabling targets reconstruction or higher-level information extraction such as objects recognition and human pose estimation. A focus will be on the combination of statistical Bayesian models, optimization algorithms and state-of-the-art deep learning methods to solve these challenging inverse problems. In particular, the candidate will investigate the design of efficient networks (eg unrolling, or plug-and-play), the development of generative models for continuous 3D/4D scene representation and the use of knowledge distillation and burst imaging for multimodal data.

References

1. A. Ruget, M. Tyler, G. Mora-Martín, S. Scholes, F. Zhu, I. Gyongy, B. Hearn, S. McLaughlin, A. Halimi, J. Leach, "Pixels2Pose: Super-Resolution Time-of-Flight Imaging for 3D Pose Estimation", Science Advances, 2022.
2. S. Plosz, A. Maccarone, S. McLaughlin, G. S. Buller, A. Halimi, "Real- Time Reconstruction of 3D Videos from Single-Photon LiDaR Data in the Presence of Obscurants", IEEE-TCI, vol. 9, p 106-119, Feb. 2023.
3. I. Gyongy, S. W. Hutchings, A. Halimi, M. Tyler, S. Chan, F. Zhu, S. McLaughlin, R. K. Henderson, and J. Leach, "High-speed 3D sensing via hybrid-mode imaging and guided upsampling," Optica, Vol. 7, Issue 10, Sept. 2020.
4. A. Halimi, A. Maccarone, R. Lamb, G. Buller, S. McLaughlin, "Robust and Guided Bayesian Reconstruction of Single-Photon 3D Lidar Data: Application to Multispectral and Underwater Imaging," IEEE-TCI, 2021.
5. J. Koo, A. Halimi, S. McLaughlin, "A Bayesian Based Deep Unrolling Algorithm for Single-Photon Lidar Systems", IEEE-JSTSP, 2022.