Assembled by Marc Levoy and Andrew Adams in September 2009.
This FAQ is now out of date. Instead, look at our SIGGRAPH 2010 paper.
What is computational photography?
Any image produced by a digital camera obviously requires computation to produce it. The admittedly vague phrase computational photography, which evolved organically in the graphics and vision research communities, refers to computational techniques that go beyond what today's digital cameras can do. Here is a set of Powerpoint slides from a (slightly out of date) survey of recent work in this area. Alternatively, here are direct links to web sites or papers about some of these techniques: tone mapping for high dynamic range imaging, in-camera panoramic stitching, motion blur removal, handshake blur removal, post-capture refocusing, depth of field extension (not the same as refocusing), flash-noflash imaging, advanced video stabilization, dark flash photography, glare removal, dehazing.
How does the Frankencamera help?
To implement the techniques listed above, you typically need a camera, a separate computer, and sometimes an optical bench. For techniques involving multiple exposures, you also need a static scene, since you can't program most cameras to shoot pictures with varying parameters fast enough to capture moving objects. By building a camera that is hand-holdable, self-powered, plug-together, and programmable at microsecond granularity, we hope to make it easier for researchers to experiment with these techniques, and to do so in the field, i.e. outside the laboratory. This should in turn make it easier to tackle real-world photographic problems.
Doesn't CHDK already let you do this?
No, they're very different beasts. The Canon Hack Development Kit, a.k.a. CHDK, lets you install scripts in certain Canon point-and-shoot cameras (but not SLRs). These scripts, written in a micro version of BASIC and supported by a vibrant open-source community, allow you to override camera parameters, perform exposure or focus bracketing, trigger the shutter on motion or other events, display histograms, customize menus, or even play video games on the camera's LCD viewfinder. However, since they doesn't replace the camera's firmware, you can't use them to reprogram the camera's pre-capture sequence (metering and focusing) or to replace its post-processing algorithms (demosaicing, denoising, sharpening, white balancing, tone mapping, etc). The Frankencamera fills this gap.
How many photographers really want to reprogram their cameras?
Probably not many. But as CHDK shows, there are plenty of enthusiasts, independent developers, and some big companies (including our sponsors) who would love to write plugins for cameras. One goal of our project is to explore what kind of software stack a camera would need to support such plugins.
What's inside the Frankencamera?
As the Stanford Report article says, the computational core is a Texas Instruments OMAP3 "system on a chip", which includes a CPU, GPU, and DSP. The imaging chip is an Aptina MT9P031 (the same as in the Nokia N95 cell phone), mounted on an Elphel 10338 sensor board, and the lenses are off-the-shelf Canon EOS lenses driven by a Birger Engineering lens mount that gives us real-time control over the lens's functions. The operating system is Linux. Many other details are still in flux, including the user interface affordances and software architecture. Eventually, we will submit a paper for publication that describes the system in detail.
Isn't there already a Linux camera, the Elphel?
Yes there is. In fact, our first Frankencamera prototype was built around an Elphel camera. However, these aren't really standalone cameras. Once you add a viewfinder and power supply with batteries, you get an awkwardly sized and shaped device. More importantly you get high latency between the sensor and viewfinder. That's why we switched to a board, based on the TI platform, that includes an LCD touchscreen.
How good will Frankencamera's pictures be?
Our current prototype uses the same imaging chip as a Nokia N95 cell phone. It's not bad (as our albums of N95 pictures show), but it's definitely not SLR quality. We're working on another version of the Frankencamera that will have an SLR-quality sensor. We also plan to replace the imaging sensor of the N95 on our current version with that of Nokia N900. We don't expect to beat Nikon or Canon on raw image quality, but as we argue above, image quality is only part of the story.
When can I buy a Frankencamera?
No time soon. The Frankencamera is aimed at researchers in the computational photography community, so they can develop new algorithms and publish them, and at teachers at the university level, so they and their students can play with these algorithms. Our goal is to start distributing cameras within 12-18 months, but the numbers will be small (dozens of cameras), they won't be cheap ($1K if we're lucky), and you'll need to be an experienced programmer to use one.
Surely there must be a way into this garden?
Well, the Frankencamera and the soon-to-be-released Nokia N900 Internet tablet are both based on the Texas Instruments OMAP3 chip, and they both run Linux. We aim to develop a software stack that runs on both platforms. Thus, if you don't care about tinkering with hardware, and the N900's 5-megapixel sensor and Carl Zeiss optics are good enough, then you can simply buy an N900 when they start shipping (in October). N900-compatible Frankencamera software may be available from us sometime in 2010.
The software for N900s was indeed released, in July of 2010. Click
here
for details.
We hope to start selling Frankencameras in early 2011. They will
cost several thousand dollars.