Neural Style Transfer

Implemented a style transfer algorithm that takes a style image and content image as inputs, and uses a deep convolutional neural network (VGG-19) to output the content image with the artistic style of the style image. Used images at each convolutional layer of the features module, normalized image tensors, and computed content and style loss. Performed content reconstruction using convolutional layers of the VGG network and performed style reconstruction using correlations between different features in CNN layers.

click for implementation details

Facial Keypoint Detection

Created convolutional neural networks to automatically detect facial keypoints, using Pytorch as the deep learning framework. First implemented and trained a 3-layer convolutional neural network for nose tip detection, next implemented and trained a 5-layer convolutional neural network for full facial keypoint detection (58 points), and finally trained a ResNet18 on a larger dataset of 6666 images to find 68 facial keypoints. Performed data augmentation to prevent overfitting.

click for implementation details

Face Morphing & Mean Face of a Population

For morphing: Defined corresponding facial features using dlibs facial landmark detector, triangulated the 68 points using Delaunay triangulation and performed 45 inverse affine warps to warp one triangle to another using interpolation. Stitched the 45 midway faces into gifs.
For the mean face of a population: used the Danes dataset and found the average shape of each image and performed inverse warping on triangularized images with the average shape.

click for implementation details

Image Warping & Mosaicing

Performed image mosaicing by taking at least 2 photographs, registering, projective warping, resampling, and compositing them. Computed homographies and used them to warp images. Used corner detection and feature matching, and finally used RANSAC (RANdom SAMple Consensus) to compute the optimal homography to perform automatic image stitching.

click for implementation details

Colorizing the Prokudin-Gorskii Photo Collection

Produced color images from grayscale Prokudin-Gorskii glass plate images by dividing the glass plate images into R, G, and B color channels. Aligned the channels by finding the optimal displacements between each channel with the least sum of squared distances, and stacked each channel to produce a color image. Optimized performance using an image pyramid: sequentially calculated displacements over multiple scaled images, starting from the coarsest image to the actual image size, and aligned the images using pixel displacements scaled to the appropriate factor relating the downsampled image to the actual resolution.

click for implementation details