CS180 Project #1: Colorizing the Prokudin-Gorskii Photo Collection

Context

Sergei Mikhailovich Prokudin-Gorskii was a pioneer of color photography, capturing the vast Russian Empire before 1918. His unique method involved taking three separate photographs through red, green, and blue filters. This project involves digitally combining these images to recreate the original color scenes.

Main Approach

The primary objective of this project is to align and colorize the Prokudin-Gorskii glass plate negatives through sophisticated image processing techniques. This process involves meticulously separating and then overlaying three filtered images (representing the Blue, Green, and Red channels) to reconstruct the original color photographs with minimal visual distortions.

Data

Each glass plate image will be divided into three equal parts representing the Blue, Green, and Red channels. The program will align the Green and Red segments to the Blue segment and output the (x,y) displacement vectors used for alignment. A predefined margin of 0.05 is cropped off from each side of every channels to avoid the borders affecting the evaluation of alignment metrics, based on the provided cropping function.

Displacement Window Search

The alignment process is driven by an exhaustive search across a defined range of possible displacements. The ideal alignment is identified by selecting the displacement that maximizes Structural Similarity Index (SSIM) , a method that quantifies the visual impact of differences between the compared images. This metric is chosen over Normalized Cross-Correlation (NCC) as it focuses on enhancing structural integrity and reducing discrepancies in color intensity between the channels.

Pyramidal Approach for Alignment

For high-resolution images, exhaustive search becomes computationally expensive. To manage this, an image pyramid approach is employed. The image is processed at multiple scales, beginning from the coarsest scale (smallest image) and progressing to finer scales, refining the alignment at each step at a reduction factor of 0.5. This method allows for rapid convergence to the best alignment with considerable computational efficiency.

Bells & Whistles

Two additional functions are implemented: automatic cropping and automatic contrasting.

Automatic Cropping

The auto_crop function performs automatic cropping on grayscale images, allowing the removal of black and white borders. A outer parts, by default 0.1, of the image are thresholded. This function uses two thresholding operations:

• thresh1: Inverts pixel values above 235, turning white areas into dark ones.
• thresh2: Binarizes the image by setting all pixels below a 35 to 225.

Original

Thresh

Cropped

Automatic Contrasting

Contrast enhancement is achieved via auto_contrast, which applies Histogram Equalization to each color channel of an image, which modifies the distribution of pixel intensities across each color channel by the mapping based on the Cumulative Distribution Function (CDF) from the histogram. This method ensures that the dynamic range of the tones is fully utilized, improving the overall visual perception of the image.

Outputs

Displacements are based on cropped images.

R: [2, -4], G: [3, -6]

R: [4, 25], G: [-4, 59]

R: [22, 50], G: [40, 105]

R: [15, 59], G: [12, 123]

R: [16, 39], G: [23, 89]

R: [9, 56], G: [12, 119]

R: [10, 81], G: [13, 177]

R: [2, -3], G: [2, 3]

R: [25, 52], G: [35, 108]

R: [-11, 33], G: [-27, 140]

R: [28, 78], G: [37, 175]

R: [17, 55], G: [10, 113]

R: [3, 3], G: [3, 6]

R: [7, 41], G: [29, 85]