Sparse and Variational Models for Pan-sharpening of Multispectral Images

Abstract

Remote sensing is defined as a mechanism facilitating the measurement of object’s features on the earth’s surface through the data obtained from platforms such as aircraft and satellites. Remote sensing provides the observation, mapping, analysis, and management of various resources present on the earth. In the past few decades, the tremendous progress in remote sensing technologies has enriched the techniques of acquisition, processing, and analysis of acquired data. The imaging data collected by the satellite sensors can be characterized using features like spatial resolution, spectral resolution, radiometric resolution, and temporal resolution. This thesis investigates the optical images whose spectral range spans visible and near-infrared (NIR) regions of the electromagnetic (EM) spectrum. The images with high-spatial and high-spectral resolution are of immense interest for various remote sensing applications like land mapping, change detection, and object recognition. The current generation satellite sensors, namely QuickBird, IKONOS, WorldView, GeoEye, etc., incur constraints such as the trade-off between spatial and spectral resolutions, limited on-board storage of satellite platform, moderate signal to noise ratio of received signal energy. The afore-mentioned commercial satellites usually produce two kinds of images; Panchromatic (PAN) image with high spatial and low spectral resolution and multispectral (MS) image with high spectral and low spatial resolution characteristics. The physical and technological limitations of sensors prohibit the objective of achieving an image with the finest spatial and spectral resolution. Pan-sharpening (PS) is a remote sensing image fusion method that produces a highresolution multispectral (HRMS) image by synthesizing the low-resolution MS image with a corresponding high-resolution PAN image. The requirement of pan-sharpened data is steadily increasing, driven by the consistent diffusion of commercial products using high-resolution images like Google Earth and Bing Maps. To date, different classes of pan-sharpening methods such as component substitution, multi resolution analysis, and model based methods have been developed. Most of the conventional PS methods induce spectral distortion and spatial artifacts in the fused image. Further, there is a demand for an efficient fusion technique that yields a pan-sharpened image with balanced spatial and spectral qualities. This research concentrates on developing pan-sharpening techniques using a sparse representation mechanism. In PS problems, the fused image is obtained by imparting the missing spatial features extracted from the PAN image into the MS image bands. The sparse representation (SR) based PS methods exploit the sparse nature of spatial vdetails using an appropriate basis, usually termed as a dictionary. The construction of a pertinent dictionary that promotes the sparsity of PAN and MS images is the fundamental task in SR based pan-sharpening problems. Motivated by the existing SR based techniques, PS methods based on two different dictionaries, namely dual dictionary and multi-scale dictionary are proposed in this thesis. To cope with the computational complexity realized by the large-sized satellite images, the SR based methods adapt patch-based strategies. The PS mechanism is implemented on overlapped patches extracted from the source images. The overlapping mechanism results in redundant and inconsistent image features in the fused image. An alternative mechanism termed as convolutional sparse representation (CSR) is deployed to deal with the drawbacks in patch-based pan-sharpening techniques. The CSR based methods are robust to misregistration between source images and produce the pansharpened image with enhanced spatial and spectral features. In addition to these methods, a variational pan-sharpening scheme is developed in this thesis to preserve the spatial details and to reduce the spectral distortion. The pan-sharpening process is formulated as a constrained optimization function using the appropriate and reliable prior terms. The developed optimization problem is solved using a suitable minimization algorithm yields a pan-sharpened image. The proposed methods are evaluated using the datasets obtained over different geographical terrains. The experimentation is performed at full-scale and reduced-scale resolutions as specified by the Wald’s and QNR protocols. The pan-sharpening techniques developed in this thesis are validated using visual and quantitative evaluation.