Marc Levoy
Computer Science Department
Stanford University
Extended abstract
Appears in the proceedings of Eurographics '99
September 7-11, 1999, Milan, Italy
(Click here for a pdf version of this document.)
Recent improvements in laser rangefinder technology, together with algorithms developed at Stanford for combining multiple range and color images, allow us to reliably and accurately digitize the external shape and reflectance of many physical objects.
As an application of this technology, I and a team of 30 faculty, staff, and students from Stanford University and the University of Washington spent the 1998-99 academic year digitizing the sculptures and architecture of Michelangelo. During this time, we scanned the David, the Unfinished Slaves, and the St. Matthew, all located in the Galleria dell'Accademia in Florence, the four allegorical statues in the Medici Chapel, also in Florence, and the architectural settings of both museums. In the months ahead we will process the data we have collected to create 3D digital models of these works.
The goals of this project are scholarly and educational. Our sponsors are Stanford University, Interval Research Corporation, and the Paul G. Allen Foundation for the Arts. In this extended abstract and invited talk, I will outline the technological underpinings, logistical challenges, and possible outcomes of this project.
From a technological standpoint, the Digital Michelangelo Project contains two components: a collection of 3D scanners and a suite of software for processing range and color data.
Our principal scanner is a laser triangulation rangefinder and motorized gantry, built to our specifications by Cyberware and customized for scanning large statues. The rangefinder has a standoff distance of 1.2 meters, a Z-resolution of 0.1mm, and an X-Y sample spacing of 0.25mm - sufficient to capture Michelangelo's chisel marks. The scanner head also contains a calibrated white light source and high-resolution color camera with a pixel size of 0.125mm on the statue surface. The scanner head is mounted on a 4-axis motorized gantry with a working volume 3 meters wide by 7.5 meters high - tall enough to scan Michelangelo's David on its pedestal. For those hard-to-reach places, we have also used a commercial jointed digitizing arm and small triangulation laser scanner manufactured by Faro Technologies and 3D Scanners.
Our range processing pipeline consists of aligning the scans taken from different gantry positions, combining these scans together using a volumetric algorithm, and filling holes using silhouette carving and similar techniques. Since gantry movements are not tracked in hardware, alignment is bootstrapped by aligning each scan to its neighbor interactively. This is followed by automatic pairwise alignment of scans using a modified iterated-closest-points (ICP) algorithm and finally by a global relaxation procedure designed to minimize alignment errors across the entire statue.
Our color processing pipeline consists of compensating for ambient lighting, discarding pixels affected by shadows or specular reflections, and factoring out the dependence of observed color on surface orientation. This requires knowing the bidirectional reflectance distribution function (BRDF) of the surface being scanned. For marble statues, we have successfully employed a simple dichromatic model consisting of a colored diffuse term and a white specular term. The result of our range and color processing pipeline is a single, closed, irregular triangle mesh with a diffuse RGB reflectance at each vertex.
Non-photorealistic renderings of our datasets are also possible. For example, by coloring each vertex of a mesh according to its accessibility to a virtual probe sphere rolled around on the mesh, a visualization is produced that seems to show the structure of Michelangelo's chisel marks more clearly than a realistic rendering. We believe that the application of geometric algorithms and non-photorealistic rendering techniques to scanned 3D artworks is a fruitful area for future research.
The Digital Michelangelo Project has been as much a production project as a research project, and we therefore have faced logistical challenges throughout the project.
One significant challenge we have faced is the size of our datasets. Our largest dataset is of the David. It was acquired over a period of 4 weeks by a crew of 22 people scanning 16 hours per day 7 days a week. The dataset contains 400 individually aimed scans, comprising 2 billion polygons and 7,000 color images. Losslessly compressed, it occupies 60 gigabytes. Although most of the techniques used in this project are taken from the existing literature, the scale of our datasets has precluded the use of many published techniques, and it has forced us to modify or re-implement other techniques.
A second logistical challenge we have faced is insuring safety for the statues during scanning. Laser triangulation is fundamentally a non-contact digitization method; only light touches the artwork. Nevertheless, the digitization process involves positioning a scanner close to a precious artwork, so accidental collisions between the scanner and the statue is a constant threat. To prevent collisions, we have used a combination of scanner design features - in particular a long standoff distance, as well as safe operating procedures and extensive training of our scanning crew.
A third logistical challenge we have faced is the development of meaningful, equitable, and enforceable intellectual property agreements with the cultural institutions whose artistic patrimony we are digitizing. Since the goals of our project are scientific, our arrangement with the museums is simple and flexible: we are allowed to use and distribute our models and computer renderings for scientific use only. The corollary issues of distribution, verification, and enforcement, although difficult in principle, are simplified in the near term by the size of our datasets. In particular, they are too large to download over the Internet. Similarly, distinguishing our computer models from other models of Michelangelo's statues is not currently a problem, since none exist. In the long term, we are investigating methods of 3D digital watermarking as they apply to large geometric databases.
The first question people ask us about these models is whether we plan to use them to make copies of the statues for sale. We have no such plans. However, our technology can certainly be used to scan and replicate statues. Among the other clients we envision for these models are art historians, museum curators, educators, and the public.
For art historians, our methods provide a tool for answering specific geometric questions about statues. Questions we have been asked about Michelangelo's statues include computing the number of teeth in the chisels employed in carving the Unfinished Slaves, determining the smallest size block from which each of the allegories in the Medici Chapel could have been carved, and determining whether the giant statue of David is well balanced over his ankles, which have developed hairline cracks. Aside from answering specific questions like these, art historians envision computer models becoming a repository of information about specific works of art. Our model of the David, when it is completed, is expected to become the official record of diagnostics and restorations performed on this statue.
For educators, computer models provide a new tool for studying works of art. In a museum, we see most statues from a limited set of viewpoints. Computer models allow us to look at statues from any viewpoint, change the lighting, and so on. In the case of Michelangelo's statues, most of which are large, the available views are always from the ground looking up. Michelangelo knew this, and he designed his statues accordingly. Nevertheless, it is interesting and instructive to look at his statues from other viewpoints. Looking at the David from unusual directions has taught us many things about the statue's ingenious design.
For museum curators, while models displayed on a computer screen are not likely to replace the experience of walking around a statue, they can nevertheless enhance the experience. As an experiment, we allowed selected visitors in the museums of Florence to manipulate our models interactively. We found that the computer focuses their attention on the statue and allows them to view it in a new way. By exploring the statue themselves, they turn the viewing of art into an active rather than a passive experience. The art museum becomes a hands-on museum.
Finally, for the public, we think that interactive viewing of computer models may eventually have the same impact on the plastic arts that high-quality art books have had on the graphic arts; they give the educated public a level of familiarity with great works of art that was previously possible only by traveling.
Although the primary goal of this project was to scan statues, our team and equipment have been involved in several other 3D scanning projects in Italy.
One such sideproject is the scanning of the architectural settings of Michelangelo's statues. For this purpose we have used a time-of-flight laser scanner manufactured by Cyra Technologies. This scanner has a Z-resolution of 5mm and a typical X-Y sample spacing of 4mm at a distance of 10 meters. Using this scanner and an attached color camera, we have built a colored 15-million polygon model of the Tribune del David in the Galleria dell'Accademia and a 50-million polygon model of the Medici Chapel. Unfortunately, these models are irregular triangle meshes, like those produced by our other scanners, and are therefore not useful for most practical architectural applications. Converting such a dataset into a conventional graphical representation such as a plan or elevation drawing is not easy. Converting it into a segmented, structured, and annotated architectural database is even harder.
A second sideproject in which we are involved is the fusion of 3D scanning and other imaging modalities. Specifically, during our scan of Michelangelo's David, we also acquired a photographic dataset of the statue under ultraviolet illumination, thereby producing a per-vertex UV fluorescence map of the statue. This map, which shows the location of waxes and other organic materials, will be used when planning future cleanings and restorations of the statue.
A final sideproject is the scanning of the Forma Urbis Romae, a giant map of ancient Rome carved onto marble slabs in circa 200 A.D. The map now lies in fragments - over 1,000 of them. Piecing this map together is one of the key unsolved problems in classical archeology. Fortunately, the fragments are several centimeters thick, and the broken surfaces give us strong three-dimensional cues for fitting the pieces back together. Our primary tool in this project will be automatic search algorithms that operate on geometric signatures computed from our models of these broken surfaces.
Although we had run many back-of-the-envelope calculations in preparation for scanning the statues of Michelangelo, the actual difficulty of the task surprised us. In particular, the statues contained more recesses and partially occluded surfaces than we anticipated, and positioning our gantry to reach them required more time and effort than we imagined. We typically spent 50% of our time scanning the first 90% of the statue, 50% on next 9%, and the last 1% was unscannable. To improve on these numbers, a scanner system would need a more compact scanner head, more positional and angular flexibility, a variable standoff distance, automatic tracking of the gantry position, and a suite of automatic view planning software.
Another hard lesson we learned was about scattering of laser light from the crystal structure of marble. The effect of this scattering is to introduce noise into the range data. In joint experiments with researchers from the National Research Council of Canada, we found that this scattering is dependent on surface polish, highly dependent on incident beam angle, and can be reduced but not eliminated by narrowing the laser beam. Furthermore, although the scattering is related to laser speckle, it would occur even with incoherent illumination. In short, there appears to be a fundamental resolution limit for structured light scanning of marble surfaces.
More information about the Digital Michelangelo Project, including images of our scanners in use and renderings of our computer models, can be found at: http://graphics.stanford.edu/projects/mich/.
Figure 1:
Our laser scanner gantry positioned in front of Michelangelo's David.
From the ground to the top of the scanner head is 7.5 meters.
Figure 2: A 1-million polygon, 2-millimeter model of David's head, constructed from our 2-billion polygon, 0.25-millimeter dataset of the statue. The color and shading are artificial.
Figure 3: A single raw scan of the unfinished statue of St. Matthew. The X-Y resolution is 0.29mm. At this scale, Michelangelo's chisel marks can be seen. The color and shading are artificial.
Figure 4: A non-photorealistic, accessibility-shaded coloring of the same mesh. To us, it seems to show the structure of Michelangelo's chisel marks more clearly than figure 3.