Sculpting Virtual Reality

Science News,  Sept 18, 1999  by Damaris Christensen

• E-mail
• Print
• Link

3-D models offer new ways of seeing art

In 1555, Michelangelo took a hammer and broke his unfinished sculpture of Mary Magdalene, the Virgin Mary, Nicodemus, and the body of Christ into 15 separate pieces. A contemporary of the temperamental Renaissance man repaired the broken statue, although the left leg of Christ was still missing.

It still isn't clear what prompted Michelangelo to mutilate his work, but art historian Jack Wasserman of Temple University in Philadelphia hopes that a new digital representation of the sculpture will offer him some clues. Already, thanks to the work of a team at the IBM Thomas J. Watson Research Center in Yorktown Heights, N.Y., he has been able to see what the main part of the statue would have looked like before the broken pieces were replaced.

Michelangelo may have originally intended the sculpture, now known as the "Florentine Pieta," to be mounted above his grave. The pieta now sits at eye level in the Museum of the Opera del Duomo in Florence, not far from where the artist is buried.

Digital imaging will allow art historians to see the statue as it looked before Michelangelo's fit of fury, says Wasserman. Additionally, once the researchers create a digital model, a computer program should be able to separate and animate the bent and intertwined human forms, he says.

"The ability to stand each figure of this pieta up straight without distorting the dimensions and proportions will provide valuable insight into the question of what Michelangelo's proportions were like, his general concept of proportions, [and] how he meant the work to be viewed at his tomb site," says Wasserman.

Digital models enable art historians to view sculptures from different heights and angles and in different lighting, he explains. As more become available, researchers and possibly museum visitors could use three-dimensional models to look at sculptures in ways that are impossible in works' current settings.

Such models are better than photographs at showing the proportions within a figure and between figures in a sculpture, he says. Such graphics may be able to restore on the computer screen the damaged areas of artworks. Moreover, digital models may bring a fuller sense of distant masterpieces to people than two-dimensional photographs can.

Building an accurate, 3-D model of a sculpture in virtual reality, however, is nearly as arduous as carving away the original stone. Researchers must make thousands of measurements and align hundreds of photographs to place every fragment of the artwork in space. The steps required to convert these data into an image on a computer screen multiply the complexity of the task.

The more detailed the model is, the more computer memory it takes up and the more difficult it is to display, rotate, and modify the virtual sculpture.

The idea of digitizing Statues is not entirely new. Practical techniques for 3-D scanning have been around for years, and several museums have successfully copied their works of art to the computer screen. However, before the work of this IBM team and that of a group from Stanford University, which spent the past year digitizing many of Michelangelo's other sculptures, no one had captured a large statue with enough precision to serve as a primary resource for scientific work, says Stanford's Marc Levoy.

Wrasserman originally approached IBM about scanning the "Florentine Pieta" because he found that detailed photography was insufficient to answer his questions. By manipulating the model on his own personal computer, he wanted to study the statue at scales ranging from meters to millimeters.

To help Wasserman, the IBM team needed to capture large amounts of data efficiently and render accurate images from various points of view with several lighting combinations. "The volume of data we have to handle carries the problem beyond the scope of existing techniques," says IBM's Gabriel Taubin.

Still, the IBM team worked as much as possible with available technology. They adapted a black-and-white camera system that plastic surgeons have used to take 3-D pictures of patients' faces. The system projects a pattern of stripes onto the face or sculpture. It then takes six pictures simultaneously from different positions.

A computer algorithm next uses the stereo images to compute the 3-D shape of the object, based on how parts of the picture shift from one stereo image to another. Because the 3-D camera system scans an area of only about 10 by 10 inches, the researchers must patch many scans together to model the entire sculpture. The stripes help the computer match corresponding parts of the pictures.

The first step in converting the many scans into a complete model is to take points measured at 2-millimeter intervals on each surface and combine them into a 3-D cloud of points. The next step is to connect the points into triangles, essentially building a 3-D mosaic of some 14 million tiles. Further calculations smooth the surface of the statue and-based on the color images taken with a different camera--add information on the statue's color and texture.