As a final project for my Computational Photography class at Georgia Tech, I created an application that uses a single video camera to create tonemapped HDR images in real time. This project’s inspiration comes from Soviet Montage Production’s DSLR HDR video.
The app runs on Linux PCs (using USB or Firewire cameras) and Android phones (using the built-in camera). Custom OpenCV code manages the different exposure images, and generates a basic HDR image. The HDR image is fed to a tonemapping algorithm by Mantiuk et al, creating either a ‘ghostly’ or ‘painterly’ effect.
An example of the Mantiuk tonemapping effect made with a DSLR camera and Luminance-HDR can be found here.
Some example project result images and videos can be seen below.
Three images are captured from a single camera, each with varying exposures (low, medium, high). These are then added into a single HDR image by adding the logarithm of each image. The HDR image is fed to a tonemapping algorithm by Mantiuk et al, taken from the Luminance HDR project, and modified for this application.
Mantuik et al’s tonemapping operator can work in different two modes: contrast mapping, or contrast equalization. Both methods are fairly computationally intensive, requiring severe down-scaling of the raw camera images to keep processing time reasonably fast.
On Linux, USB cameras are supported (and captured) by OpenCV, while Firewire cameras are handled by a custom libdc1394 wrapper. Most USB cameras only support changing the brightness (not exposure), which generates a faux-HDR image that then gets tonemapped. An AVT Guppy machine vision firewire camera was also used for testing, because it allows changing the shutter speed and adjusting gain. This camera produced much better results than any webcam tested.
On Android, the built-in camera was controlled via Android camera APIs in Java. Unfortunately, there is a massive delay between setting the exposure, and when the camera actually gets to that exposure. After each exposure change, an arbitrary number of dummy frames are discarded before grabbing an image, in attempt to give the camera time to adjust. This waiting for the camera’s exposure change takes up about as much time as the actual processing, exaggerating an already slow image processing loop.
Pros / Features:
- Single camera, live High Dynamic Range Image viewing
- Mantuik tone mapping operators
- Contrast Mapping (faster, exaggerates shadows, darker)
- Contrast Equalization (slower, exaggerates colors, brighter)
- Cross platform (Android / Linux / with a little work, Windows)
- Various camera support (USB / Firewire / Android)
- No image alignment pre-processing needed (assuming little camera movement)
- OpenCV + OpenMP
Cons / TODOs:
- Very low resolution
- Low frame rate (exposure change time limits frame rate)
- Android’s camera exposure change is terribly slow
- No fancy GUI
- Manual adjustment of camera settings required (trial-and-error based)
- Results are extremely dependent on quality/extent of the camera’s exposure changes (quality = actually getting to the desired +/-2 EV)
- Port code to GPU
- Use a faster tone-mapping-operator
Download ViewerCV on Android Market.
Related Android computer vision post.
Class presentation slides.