A Prototype Multi-view Stereo Method for Determining Cloud Structure with Multi-angle Imagery

[14-Dec-2023]
Abstract:  Stereo-based 3D reconstruction techniques have long been used in the remote sensing and computer vision communities. Both fields have progressed divergently, leading to a rift in applications, terminology, and techniques. In remote sensing, the estimation of 3D cloud structure is typically referred to as a height retrieval problem. This problem is closely related to multi-view stereo (MVS) from the computer vision community, which jointly optimizes the 3D locations of estimated inter-image correspondences. The MVS approach, when compared to existing 3D cloud retrieval techniques, has a faster runtime and no human-in-the-loop, allowing its scalability to years of data, rather than a small collection of scenes. MVS has been applied to high-resolution satellite imagery, with non-simultaneous observations. Its applications in this domain are limited to non-transient features, such as digital elevation mapping. To our knowledge, there are no works on the application of MVS to wide-swath, multi-angle imagery. This work consists of an initial approach to MVS in multi-angle polarimetry and a discussion of the challenges therein: adaptation of the camera model, the lack of distinct keypoints in clouds, and the high mean scene depth. Estimation of 3D cloud structure over a wide swath could provide a useful source of data for climate models and large eddy simulations, and has applications for upcoming missions with multi-angle sensors, such as those in the forthcoming Plankton, Aerosol, Cloud-ocean Ecosystem (PACE) and Atmosphere Observing System (AOS) missions.