Compressive Optical Systems

Recent work in the emerging field of compressive sensing indicates that, when feasible, judicious selection of the type of distortion induced by measurement systems may dramatically improve our ability to perform reconstruction. The basic idea of this theory is that when the signal of interest is very sparse (i.e., zero-valued at most locations) or compressible, relatively few incoherent observations are necessary to reconstruct the most significant non-zero signal components. However, applying this theory to practical imaging systems is challenging in the face of several measurement system constraints. Ongoing research at NISLab includes the design of coded aperture masks based on compressive sensing principles for enhancing image reconstruction resolution from low-resolution, noisy observations.

Poisson Compressed Sensing

We have derived performance bounds for compressed sensing (CS) where the underlying sparse or compressible (sparsely approximable) signal is a vector of nonnegative intensities whose measurements are corrupted by Poisson noise. In this setting, standard CS techniques cannot be applied directly for several reasons. First, the usual signal-independent and/or bounded noise models do not apply to Poisson noise, which is non-additive and signal-dependent. Second, the CS matrices typically considered are not feasible in real optical systems because they do not adhere to important constraints, such as nonnegativity and photon flux preservation. Third, the typical l2–l1 minimization leads to overfitting in the high-intensity regions and oversmoothing in the low-intensity areas. In this paper, we describe how a feasible positivity- and flux-preserving sensing matrix can be constructed, and then analyze the performance of a CS reconstruction approach for Poisson data that minimizes an objective function consisting of a negative Poisson log likelihood term and a penalty term which measures signal sparsity. We show that, as the overall intensity of the underlying signal increases, an upper bound on the reconstruction error decays at an appropriate rate (depending on the compressibility of the signal), but that for a fixed signal intensity, the error bound actually grows with the number of measurements or sensors. This surprising fact is both proved theoretically and justified based on physical intuition.

Compressive Coded Apertures

Appropriately designed coding masks can be used to dramatically increase the resolution of imaging systems. Much of the existing work in this field assumes multiple low resolution images are collected, such as in the system described above, so that the number of observations is roughly the same as the number of pixels in the high resolution image. In a variety of practical settings, however, collecting several low-resolution observations is not feasible because of time, sensor size, or data storage limitations. In light of these constraints, we have developed a coded aperture scheme for collecting a single low-resolution observation and using it alone to reconstruct the high-resolution scene. In particular, building from the theory of compressive sensing using Toeplitz-structured matrices, we established a method for generating coded aperture masks; these random masks can be shown to result in an observation model which, with high probability, will allow very accurate reconstructions of many everyday images, with crisper edges and improved feature resolution over reconstructions achieved without the benefit of coded apertures. This is demonstrated in the figure below.


Compressive coded aperture superresolution results. (a) Original high-resolution image. (b) Reconstruction from a MURA coded aperture camera system, upsampled using nearest-neighbor interpolation. (c) Observed coded aperture image, with one quarter as many pixels as (a), using a code derived with my group's method. (d) Reconstruction from observations in (c). Note the improved resolution in the rings and moon.

A video demonstration of this approach is below and here. The left half is the result of low resolution video acquisition, and the right half is the result of compressive coded aperture video acquisition. Notice how details on the volvox surface are much clearer with the coded aperture approach.

Compressive Spectral Imaging

Spectral images, which contain spectral information for each spatial location in the image, are useful resources in a variety of scientific and engineering contexts because of the information they implicitly encode about the nature of the materials being imaged. Spectral information can be vital for tasks such as monitoring the health of a forest ecosystem, increasing our understanding of solar physics, or examining tissue and organisms used to study cellular dynamics.

Despite this potential, however, many modern spectral imagers face a limiting tradeoff between spatial and spectral resolution, with the total number of voxels measured constrained by the size of the detector array. This constraint limits the utility and cost-effectiveness of spectral imaging in many contexts. To mitigate this tradeoff, NISLab and the lab of Prof. D. Brady are together developing several spectral imaging systems and associated reconstruction methods that have been designed to exploit the theory of compressive sensing. By exploiting this theory, we achieve single-shot full spectral image estimates using compressed sensing reconstruction methods to process observations collected using an innovative system design. One example physical system collects coded projections of each spectrum in the spectral image. Using the novel multiscale representation of the spectral image based upon adaptive partitions as described in the above section on hyperspectral imaging, we are able to accurately reconstruct spectral images with an order of magnitude more reconstructed voxels than measurements. Simulation results associated with this approach are displayed in the figure below. The intensities of a test spectral image in several spectral bands and several representative spectra are displayed in the first row. The total number of measurements collected by our system is equal to the number of pixels in just one of these images, resulting in a very challenging inference problem.
Spatio-spectral reconstruction results. (a) True intensity in 4th spectral band. (b) True intensity in 8th spectral band. (c) True intensity in 12th spectral band. (d) Representative spectra from true spatio-spectral data cube. (e) Estimated intensity in 4th spectral band, computed using multiscale spatio-spectral regularization. (f) Estimated intensity in 8th spectral band. (g) Estimated intensity in 12th spectral band. (h) Representative spectra from estimated spatio-spectral data cube.

Thin Infrared Imaging Systems

As an example of computational optical sensor design, NISLab group has worked together with Prof. D. Brady's lab to show that infrared camera systems can be made dramatically smaller by simultaneously collecting several low-resolution images with multiple narrow aperture lenses rather than collecting a single high-resolution image with one wide aperture lens. The infrared camera uses a three by three lenslet array, and we achieve image resolution comparable to a conventional single lens system which is an order of magnitude larger.

The high-resolution final image generated by this system is reconstructed from the noisy low-resolution images corresponding to each lenslet; this is accomplished using a computational process known as superresolution reconstruction. The novelty of NISLab's approach to the superresolution problem is the use of wavelets and related multiresolution methods within an Expectation-Maximization framework to improve the accuracy and visual quality of the reconstructed image, as shown in the below figure.
Caption: Thin sensor image reconstruction. (a) Conventional optics (upper right) and our lenslet array optics (in box on lower left). (b) Observations collected by lenslet array. (c) Observations corresponding to center lenslet in array (enlarged). (d) Reconstructed high resolution image.

Shifted Excitation Raman Spectroscopy

The theme of data-starved inference for point processes has guided the NISLab research on innovative tools for Raman spectroscopy. Raman spectroscopy is widely used to study vibrational modes of molecules in a material; since these modes depend upon the chemical bonds in the material, Raman spectroscopy has enormous potential for material identification in a broad range of applications. However, the effectiveness of this technology is impeded by strong fluorescent background spectra which overwhelm the weaker Raman spectra. We have addressed this challenge via shifted excitation Raman spectroscopy, the process by which the Raman spectrum of a material is estimated using multiple noisy observations of the spectrum collected at different (shifted) excitation frequencies. More excitation frequencies yield more information about the separate Raman spectrum and fluorescent background, but at the cost of significantly lower photon counts and more noise. My research yielded a photon-limited signal analysis framework which resulted in dramatic improvements in Raman spectrum estimation, despite the significant increase in noise for each observed spectrum. This is demonstrated via simulation in the below figure, where the first row shows typical measurements collected with two, ten, or forty excitation lasers, and the second row shows the associated reconstructions, which yield up to an 84% reduction in error from conventional approaches. Using this work as a foundation, Prof. D. Brady and his lab at Duke University have built a novel Raman spectrometer for applications such as blood alcohol testing.

Shifted Excitation Raman Spectroscopy experimental results. (a) Representative one of two observed spectra. (b) Representative one of ten observed spectra. (c) Representative one of forty observed spectra. (d) Reconstructed Raman spectrum using two excitation frequencies. (e) Reconstructed Raman spectrum using ten excitation frequencies. (f) Reconstructed Raman spectrum using forty excitation frequencies.