Methodology

This project involved reconstructing color images from Prokudin-Gorskii's digitized glass plate negatives by aligning the red, green, and blue channels. The alignment was achieved through a multi-step approach combining exhaustive search, image pyramids, border cropping, and edge detection.

Exhaustive Search:
I began by implementing an exhaustive search within a 15x15 pixel window to align the green and red channels to the blue channel, using the L2 norm (SSD) to evaluate alignment accuracy.

Image Pyramid for Efficiency:
To improve performance, I introduced an image pyramid technique, which recursively downsizes the images to align them at different scales. I made the pyramid levels adaptive such that the coarsest level's resolution was set to approximately 300 pixels. This allowed me to align images at lower resolutions, progressively refining the alignment at higher resolutions. The optimal shift found at each level was scaled and applied to the next finer level, thus significantly speeding up the alignment process without sacrificing accuracy.

Dynamic Border Cropping:
Recognizing the impact of noisy borders on alignment, I implemented an automated border cropping strategy. The border size was dynamically set as 15% of the image size at each pyramid level to account for variations in resolution. This step helped to remove edge artifacts and focus the alignment on the central part of the images, where the content is most important.

Normalization and Window Refinement:
For each pyramid level, the window size was refined dynamically, starting with a larger window at coarser resolutions and reducing it at finer resolutions. This allowed for more precise shifts at higher resolutions and ensured the process remained computationally efficient while maintaining alignment accuracy.

💡

Edge Detection:
In more challenging images, such as the "Emir," I implemented edge detection using a combination of grayscale conversion, Gaussian filtering, and Sobel filters. By focusing on the edges, I improved alignment by ensuring that structural elements of the images (e.g., lines and shapes) were matched accurately, leading to better results, particularly in cases where intensity variations between channels caused issues.

In more details, this is how the naive implementation of the emir, using image pyramid looks before and after:

We can tell that due to the irregular brightness of the image the alignment becomes harder than the other.

That we why we apply edge detection: we apply the Sobel filer to each of our rgb grey scale channels, I then apply a gaussian filter to make the image more regular and we get the result on the right hand side. that is what we put into out pyramid_align function to get the image properly aligned, getting the result from below:

Images with Sobel and gaussian filtering applied. (each filter represents a channel