Warping, Stitching, and Image Registration
Date: October 10, 2024
Lecture Duration: 2.5 hours
Topic Overview: This lecture focuses on geometric manipulations that allow us to combine multiple images into a single coherent view. We explore how to warp images using projective geometry, stitch overlapping photographs into seamless panoramas, and align images across different modalities or viewpoints through image registration. Additionally, we cover High Performance Computing (HPC) basics and the Fast Normalized Cross Correlation method for efficient template matching.
1. High Performance Computing (HPC) Basics
We discussed how to leverage High Performance Computing to process large datasets and train complex models.
- SLURM Jobs: We reviewed how to schedule jobs on NYU HPC using SLURM and follow best practices for running compute-intensive tasks.
2. Fast Normalized Cross Correlation (NCC)
We analyzed the computational bottlenecks of naive Normalized Cross Correlation and introduced optimizations.
- J.P. Lewis’s Optimization: We explored how precomputing integral images and using a sliding window approach drastically speeds up NCC.
3. Image Warping and Geometric Transformations
We revisited geometric transformations with a specific focus on Projective Transformations (Homographies).
- Homography Matrix: We mathematically defined the 3x3 matrix \(H\) that maps points from one plane to another.
- Image Warping: We implemented backward warping to smoothly transform an image onto a new perspective grid without leaving “holes” in the output pixels.
4. Panorama Stitching
How do we take a series of overlapping photos and create a wide-angle panorama? We broke this down into a systematic pipeline:
- Feature Matching: We used techniques from previous lectures (SIFT/ORB) to find corresponding keypoints between overlapping images.
- RANSAC (Random Sample Consensus): Real-world feature matching is noisy. We introduced RANSAC as a robust method to estimate the Homography matrix while ignoring incorrect matches (outliers).
- Blending: Once images are warped into the same coordinate space, we discussed blending techniques (like feathering and Laplacian Pyramids) to hide the seams caused by exposure differences.
5. Image Registration
Image registration is the process of aligning two or more images of the same scene taken at different times, from different viewpoints, or by different sensors.
- Alignment Metrics: We explored how to measure “goodness of fit” between images.
- Applications: We discussed medical imaging (aligning MRI/CT scans) and satellite imagery alignment.
Interactive Demonstration
Below is the complete Jupyter Notebook used in class. It contains Python implementations for HPC job scheduling, Fast NCC, Homography estimation, RANSAC, and Panorama Stitching.