R. Sharma and M. Pavel, "Multisensor image registration," and
M. Pavel and R. Sharma, "Model-based sensor fusion for aviation."
The image registration work resulted in the development of an
algorithm which has shown considerable success in registering
images from different sensors.
The starting point of the algorithm
is the method previously developed for the program by
Hel-Or.
This method uses a combination of a pyramid representation and
gradient descent to find the parameters of a projective transformation
to map one image into another.
This method does not usually work if the two images are not from
the same sensor because
1) the polarity of image gray scale images may be reversed, and
2) features in one image can be missing in the other image.
The polarity reversal problem is minimized by computing a linear
transformation of the gray scale value in one image separately
for each pixel.
The missing features problem is minimized by simply removing
a fixed percentage (10%) of the worst fitting pixels from the
goodness-of-fit calculation.
Although these calculations are iterative and thus slow when
computed for two new images, small corrections in a running
calculation can be done fairly rapidly.
The image fusion work has taken a different approach from the previous fusion work reported here. The simplest image fusion technique is just to add the images. The problem is that only if the two images are the same is the sum as good as the original. If a feature appears in only one image, its contrast is cut in half. If the polarity of a feature reverses in the two images, the result can be almost no contrast. The pyramid feature selection of Burt and Adelson selects in each spatial frequency band and spatial position, the signal with the greatest absolute amplitude. This rule does fairly good job of keeping the most important features visible. Our past work can be summarized as a revision of this technique, using intensity transformations to give the images the same polarity when possible and by using statistical reliability as the amplitude measure, so that noise will not blot out features just because it has high amplitude in some spatial frequency ranges.
The current paper presents a method that does not select between two gray scale images, rather it uses different colors to allow a fused image to be formed by the visual system, while at the same time allowing features that only occur in one sensor image to be identified with that sensor. The basic idea is to form a fused image as a luminance image based on the common information in the two images and then to use each of the opponent color channels, red-green and blue-yellow, to indicate the source of unique information.
References (HTML) are provided in image fusion and registration compiled from other papers generated by this program over the past few years.
GIF images from the registration paper:
FLIR Image
Radar Image
FLIR and Radar side by side
Registered and Combined Images
GIF images from the fusion paper:
Long Wave Image
Medium Wave Image
A-2 Fused Images
YIQ Fused Images
