Marc Levoy
Computer Science Department
Stanford University
Extended abstract
Appears in the proceedings of
Electronic Imaging and the Visual Arts (EVA) '99
Florence, Italy
March 26, 1999
Recent improvements in laser rangefinder technology, together with algorithms developed at Stanford for combining multiple range and color images, allow us to reliably and accurately digitize the external shape and reflectance of many physical objects.
As an application of this technology, we have embarked on a multi-year project to create a high-quality 3D computer archive of the sculptures and architecture of Michelangelo. As of this writing, we have scanned the the David, the Unfinished Slaves, and the St. Matthew, all located in the Galleria dell'Accademia in Florence. In the coming weeks we will scan the statues and architecture of Michelangelo's Medici Chapel, also in Florence.
The goals of this project are scholarly and educational. Our sponsors are Stanford University, Interval Research Corporation, and the Paul G. Allen Foundation for the Arts. In this extended abstract, I will summarize the major technical components of our scanning system, the current status of our project, and some of the technical challenges we have faced so far.
(For a non-technical overview of the project, look first at the article written for the Stanford Campus Report, then at the slides from a 1997 talk on the project, then at a file of frequently asked questions.)
From a technological standpoint, the Digital Michelangelo Project contains two components: a collection of 3D scanners and a suite of software for processing range and color data.
Our principal scanner is a laser triangulation rangefinder and motorized gantry, built to our specifications by Cyberware Inc. and customized for scanning large statues. The scanner consists of a low-powered laser, range camera, white light source, and high-resolution color camera. The laser and range camera permit digitization of 3D points with a Z-resolution of 0.1mm, an X-Y sample spacing of 0.25mm, and a standoff from the statue of 112cm. This resolution is sufficient to capture Michelangelo's chisel marks. The light source and color camera permit measurement of RGB surface radiance with a pixel size of 0.125mm.
The scanner is mounted on a 4-axis motorized gantry consisting of a 7-meter vertical truss, a 1-meter horizontal arm that translates vertically on the truss, and a pan-tilt head that translates horizontally on the arm. The scanner can mounted in several positions on this pan-tilt head, providing a wide range of scanning configurations. The maximum working volume of the gantry is 3 meters wide by 7.5 meters high, tall enough to scan Michelangelo's David on its pedestal.
For those hard-to-reach places (we're not moving the statues), we have also brought with us to Italy a jointed digitizing arm and small triangulation laser scanner manufactured by Faro Technologies and 3D Scanners Ltd. Finally, to enable us to scan the architectural settings of Michelangelo's statues, for example in the Medici Chapel, we have brought with us a prototype time-of-flight laser scanner manufactured by Cyra Technologies.
The second technological component of our project is the software we have written for processing range and color data.
Our range processing pipeline consists of aligning the scans taken from different gantry positions, combining these scans together using a volumetric algorithm, and filling holes using silhouette carving and similar techniques. Since gantry movements are not tracked in hardware, alignment is bootstrapped by aligning each scan to its neighbor interactively. This is followed by automatic pairwise alignment of scans using a modified iterated-closest-points (ICP) algorithm and finally by a global relaxation procedure designed to minimize alignment errors across the entire statue. The result of our range processing pipeline is a single, closed, irregular triangle mesh.
Our color processing pipeline consists of compensating for ambient lighting, discarding pixels affected by shadows or specular reflections, and factoring out the dependence of observed color on surface orientation. This last step requires knowing surface orientation with high accuracy. Our range data is adequate for this task, although only barely so. This step also requires knowing the bidirectional reflectance distribution function (BRDF) of the surface being scanned. For marble statues, we have successfully employed a simple dichromatic model consisting of a colored diffuse term and a white specular term. The result of our color processing pipeline is a diffuse RGB reflectance for each vertex of our triangle mesh.
Non-photorealistic renderings of our datasets are also possible. For example, by coloring each vertex of a mesh according to its accessibility to a virtual probe sphere rolled around on the mesh, a visualization is produced that seems to show the structure of Michelangelo's chisel marks more clearly than a realistic rendering. We believe that the application of geometric algorithms and non-photorealistic rendering techniques to scanned 3D artworks is a fruitful area for future research.
A significant challenge we have faced in this project is the size of our datasets. So far we have scanned 6 statues. Our largest dataset is of the David. It was acquired over a period of 4 weeks by a crew of 22 people scanning 16 hours per day 7 days a week. The dataset contains 400 individually aimed scans, comprising 2 billion polygons and 7,000 color images. Losslessly compressed, it occupies 60 gigabytes. Although most of the techniques used in this project are taken from the existing literature, the scale of our datasets has precluded the use of many published techniques, and it has forced us to modify or re-implement other techniques. In addition, as in any large-scale production project, data management is a significant part of the overall software effort.
A second logistical challenge we have faced in the project is insuring safety for the statues during scanning. Laser triangulation is fundamentally a non-contact digitization method; only light touches the artwork. Nevertheless, light and heat can potentially damage art, so their levels must be controlled. Our scanning beam is a 2mW red semiconductor laser, but its power is spread into a line 35cm wide at the statue surface, and it moves nearly continuously during scanning. Our white light source is a 250W incandescent bulb, but its power is conducted through a fiber-optic cable, which effectively blocks heat, and its light is spread into a disk 50cm wide at the statue surface. In both cases, energy deposition on the statue surface is negligible compared to ambient lighting levels in most museums.
Our primary defense against accidental collisions between the scanner and the statue is is our long standoff distance - over a meter. However, we have found it difficult to maintain this standoff while also reaching all parts of a large statue. To protect statues from motorized collisions with the scanner, our design includes an elaborate system of manual and automatic motion shutoffs and interlocks. To reduce the chance of damage in the case of inadvertent contact, our scanner head and pan-tilt assembly are encased in foam rubber. Despite these measures, our gantry is moved and aimed by human operators, so the possibility of operator error is omnipresent. To reduce this risk, our scanning teams include at least two people, one of whom operates as a spotter, we establish and follow rigid protocols, and we try to get enough sleep.
A third logistical challenge we have faced is the development of meaningful, equitable, and enforceable intellectual property agreements with the cultural institutions whose artistic patrimony we are digitizing. While this is an issue in any project of this sort, it is made more acute in our case by the fame of Michelangelo's sculptures. Since the goals of our project are scientific, our arrangement with the museums is simple and flexible: we are allowed to use and, to a limited extent, distribute our models and computer renderings for scientific use only. In the event we, or they, desire to use the models commercially, there will be further negotiations and probably the payment of royalties. The corollary issues of distribution, verification, and enforcement, although difficult in principle, are simplified in the near term by the size of our datasets; they are simply too large to download over the Internet. Similarly, distinguishing our computer models from other models of Michelangelo's statues is not currently a problem, since none exist. In the long term, we are investigating methods of 3D digital watermarking as they apply to large geometric databases. However, this is still an open area for research.
Although the primary goals of this project are to scan the statues, our team and equipment are currently involved in several other 3D scanning projects in Italy.
One such sideproject is the scanning of the architectural settings of Michelangelo's statues. For this purpose we are employing a time-of-flight laser scanner manufactured by Cyra Technologies. This scanner has a Z-resolution of about 5mm and a typical X-Y sample spacing of about 4mm at a distance of 10 meters. Its maximum calibrated range is 100 meters. Our prototype configuration includes an ultra-high resolution (2K x 2.5K pixel) color camera. Using this scanner, we have built a colored 15-million polygon model of the Tribune del David in the Galleria dell'Accademia. In the coming weeks we will also digitize the Medici Chapel. In this latter endeavor, we are aided by the fortuitous presence of a 25-meter-high scaffolding tower in the center of the chapel. By elevating our scanner to the intermediate floors of this tower, we can look both upward and downward on the architectural decorations, enabling us to build a computer model of unprecedented completeness.
Unfortunately, our computer model of the Medici Chapel will be an irregular triangle mesh, like those produced by our other scanners. While a simplified version of this mesh might suffice for virtual reality flythroughs, it is not useful for most practical architectural applications. Converting this dataset into conventional graphical representations such as plans and elevation drawings is not easy. Converting it into a segmented, structured, and annotated architectural database is even harder. In fact, both are open research problems.
A second sideproject in which we are involved is the fusion of 3D scanning and the traditional diagnostic imaging modalities used in art restoration. Specifically, during our scan of Michelangelo's David, we haved also acquired a 1mm-pixel photographic dataset of the statue under ultraviolet illumination. By using a calibrated camera and a simple correspondence-based method for locating the camera in 3-space, we have mapped this data onto our 3D model, producing a per-vertex UV fluorescence map of the statue. This map, which shows the location of waxes and other organic materials, will be used when planning future cleanings and restorations of the David.
A final sideproject is the scanning of the Forma Urbis Romae, a giant map of ancient Rome carved onto marble slabs in circa 200 A.D. The map now lies in fragments - over 1,000 of them. Piecing this map together is one of the key unsolved problems in classical archeology. Fortunately, the fragments are several inches thick, and the broken surfaces give us strong three-dimensional cues for fitting the pieces back together. Over the next few months, we plan to scan these fragments and use our automatic mesh alignment tools to search for new matches among the fragments.
The web pages of the Digital Michelangelo Project begin with
http://graphics.stanford.edu/projects/mich/
or in Europe:
http://graphics.stanford.firenze.it/projects/mich/
For information about our 3D scanning software:
http://graphics.stanford.firenze.it/projects/faxing/
and learn about our demonstration 3D fax of a Happy Buddha statuette:
http://graphics.stanford.firenze.it/projects/faxing/happy/
For images and a detailed description of our Cyberware scanner:
http://graphics.stanford.firenze.it/projects/mich/mgantry-in-lab/
for computer renderings from our scan of Michelangelo's St. Matthew:
http://graphics.stanford.firenze.it/projects/mich/matthew/
and, finally, for the latest images from our scan of Michelangelo's David:
http://graphics.stanford.firenze.it/projects/mich/david/