The Resting Satyr in Augmented Reality

Host Organization: The Eskenazi Museum of Art

Supervising Faculty: Juliet Graver-Istrabadi, Dr. Julie Van Voorhis, Dr. Bernard Frischer

Collaborators: Andy Hunsucker, Leif Christensen

[SLIDE: Team Names]

Before I begin, I want to take a moment to thank all the members of this project, as well as our fantastic organizers, Angelica and Joe. The discussions and presentations yesterday were excellent, and I hope that we can have an equally enlivening discussion here at the American Academy today.

[SLIDE: Marble Faun]

There is something magical in viewing a work of art. In a moment of exchange, enchantment, and intuitive understanding, an epiphany occurs. We are struck, often speechless, for a few precious moments, and transported to another world. For many of us, this is how we came to study the material that we specialize in. Nathaniel Hawthorne, upon viewing the Resting Faun in the Capitoline, was inspired to give him a rather specific character: “Perhaps it is the very lack of moral severity, of any high and heroic ingredient in the character of the Faun, that makes it so delightful an object to the human eye and to the frailty of the human heart.”

Art possesses an enigmatic capacity of being able to transport us to another world. Through augmented reality, we hope to heighten this epiphanic power by encouraging the museum visitor to pause, and look more closely. Digital presentations in museums, as we already saw yesterday, can bot educate and excite non-domain expert visitors.

[SLIDE: Augmented Reality Experience]

Questions remain, however, about the efficacy and impact of this technology. The deepest question - do we really need to augment art? - is one worthy of careful consideration. Designers of digital installations often claim that they will enhance the experience of a viewer. Yet curators often express concern that these presentations may in fact detract from the time a visitor spends looking at the actual objects.

In the fall of 2016, I was asked by Juliet Istrabadi, the curator of the Eskenazy Museum of Indiana University, and by our very pro-technology museum director, David Brenneman, to work with them to create an Augmented Reality experience that could help visitors to better understand the Roman torso of an ancient statue type known as the "Resting Satyr." Juliet  had observed that, as a fragmentary sculpture, visitors tended to walk by the object without paying it much attention. She and her director turned to me, as someone in the doctoral program in Virtual Heritage, for help and assistance. I had previously experimented with the Microsoft Hololens, one of the most advanced AR devices currently under development. For me, the project offered the chance to apply my knowledge of this new technology to a concrete problem.

Since our first meeting, my team and I have grappled with the question that is the main subject of this paper: can Digital Humanities interventions, such as augmented reality exhibits, make material from antiquity more accessible in a way that does not detract from the primacy of the object, its materiality, and the potential for epiphany? Stated in another way, does augmented reality detract from the aura that surrounds a work of art?

[SLIDE: Goldfinch]

The aura is a unique phenomenon of distance that is tied to the presence of a work, and in the field of digital art history, it seems therefore critical to consider how we interacting with it. Walter Benjamin addressed a similar question in his essay “The Work of Art in the Age of Mechanical Reproduction.” While his essay was primarily aimed at photography, the concerns underlying the decay of the aura with photography are almost identical to those at the heart of our question for today: do digital interventions detract from the original? The process of 3D digitization used by our lab begins with photography, so it is apt to briefly consider Benjamin’s discussion. He relates a work of art to something akin to a cult object - the further away one is, for example looking at the alta,  but unable to approach, the more heightened the impression of the aura. Prior to the invention of photography - and one could say by extension, digital photogrammetry, a pilgrimage was necessary to view a work of art. After the invention of mechanical reproduction the work of art was emancipated from its dependence on ritual.

[SLIDE: Capitoline Museo]

Yet while the pilgrimage is no longer required, people still chose to make it - years have passed since Benjamin’s essay, and yet still people travel here to Rome, and visit museums such as the Capitoline, where they marvel at works of art such as the Resting Satyr, the subject of our discussion today. Why? Benjamin points out that despite the perceived decay of the aura through mechanical reproduction, aura is intrinsically tied to presence; there can be no replica of it. It is like an invisible cord, tied from our eyes to the sculpture itself. The singularity of the shot--the first step in creating a photogrammetric model--substitutes the camera for the public. Consequently, the aura vanishes. The cord snaps.

So much for photography. But what about photogrammetry, specifically photogrammetry applied to Augmented Reality experiences within museums? This is the large issue I wish to address in my paper today. I will first outline the art historical research that formed the basis of our design, summarize the progress of our project to date, and synthesize the methodology that we used to test the impression the experience makes on a visitor.

[SLIDES: Satyr in the Eskenazi]

Our augmented reality experience evolved out of a research question: how does the replica of the Resting Satyr in the Eskenazi relate to the material record of antiquity--including other sculpture, and display contexts? The Resting Satyr torso in the Eskenazi is fragmentary but exquisitely carved, and part of a large series of replicas. Elizabeth Bartman, in Ancient Sculptural Copies In Miniature, adding to the 1968 dissertation of Peter Gercke, catalogs over 100 replicas, of which one-fifth are miniature in scale. Several of the most famous European museums have at least one full-size replica of the Resting Satyr, including the Ny Carlsberg Glyptotek, the Glyptothek Museum in Munich, and the Prado, to name just a few. Bartman notes that Rome has the most copies, and yet even so,  in antiquity these replicas were spread across the Roman empire: some as far as the Hadrianic Baths in North Africa. Partly because of its location, and its association with Hawthorne’s The Marble Faun, the most iconic replica is in the Capitoline Museum.

[SLIDE: Glyptothek Satyr]

However, restorers have so thoroughly reworked the Capitoline piece that some scholars, including Oscar Antonsson, argue that it no longer qualifies as antique. By contrast, the Glyptothek Resting Satyr in Munich is not only one of the most complete of the extant replicas, it is also one of the best-preserved; for efforts in recent years have stripped it of accumulated “restorations” undertaken in the 18th century.

[SLIDE: Pliny Passage]

The Resting Satyr was attributed to Praxiteles by Johann Winckelmann primarily on the basis of a passage in Pliny: “Praxiteles, who was more successful in marble and hence was more famous also made, however, very beautiful works in bronze: the Rape of Persephone, a Katagousa, and a Dionysos with a figure of "Drunkenness" and the famous Satyr to which the Greeks attach the surname periboetos ["the much talked of" or "the notorious"]

[SLIDE: Satyrs]

To be sure not all scholars agree that the statue conventionally called “The Resting Satyr” is the sculpture mentioned in this passage. However, the majority of scholarship still supports the attribution. The Resting Satyr is often referred to as ἀναπαυόμενος (anapauomenos), the resting one. He appears to be between 18 and 20; a true ephebe. The musculature of his neck, shoulders, and torso is aquiline, and a long line extends from the leaning arm’s right shoulder to the left hip. From other replicas, we know that this line would extend all the way to the ankle resting delicately behind the other foot. The relaxed contrapposto pose gives the impression that despite his precarious tilt, our Satyr is remarkably content and deeply settled into himself. We can imagine that if Dionysus called out, it would take our Satyr several seconds at least to stir. His left hand clasps an item within (unfortunately now lost) gently, not tightly. Even the fingers of the hand that rests on the hip are placed lightly, with the palm turned outward in a relaxed, yet sassy pose. Formally brilliant, the Resting Satyr mediates conflicting elements—human and animal, rustic and urbane, natural and posed. Perhaps it is this paradoxical quality that drew the Romans to reproduce him on such a large scale, and display replicas of him across the empire.

[SLIDE: Pausanius Quote]

Display context for Dionysiac sculpture such as the satyr varied between the Greek and Roman periods. Pausanias tells us in his Description of Greece that a satyr statue stands at the entrance to the sanctuary of Dionysus at Megara. “Polyidus also built the sanctuary of Dionysus, and dedicated a wooden image that in our day is covered up except the face, which alone is exposed. By the side of it is a Satyr of Parian marble made by Praxiteles.”

[SLIDE: Passage 2]

He also notes that another Satyr is displayed in the Road of the Tripods. “The place takes its name from the shrines, large enough to hold the tripods, which stand upon them, of bronze, but containing very remarkable works of art, including a Satyr, of which Praxiteles is said to have been very proud.”

[SLIDE: Vase Images]

The ancient Greeks considered Satyrs to be liminal creatures that could move easily between our prosaic world and the world of myth. They were dangerous, wild, and unpredictable. As early as the Homeric Hymn to Aphrodite, we find descriptions of their amorous relations with nymphs. On Greek vases, we find images of Satyrs that are often violent, dancing, drunk, or all three.

[SLIDE: House of Marcus Lucretius in Pompeii garden]

The Romans also associated the satyr strongly with Dionysus, and they are included, along with a host of other figures, in the festival procession described by Athenaeus. However, unlike the Greeks, Romans chose to display satyrs in spaces associated with rest, or otium. One of the most famous examples is the garden in the Villa of the Papyri, where we find not only the Drunken Satyr, but also a flute player. In atrium houses, such as the House of Marcus Lucretius in Pompeii, there is often an entire garden devoted to Dionysiac figures such as the satyr arranged around a central Bacchus. Such gardens were places for contemplation, but they are also playful. Placing a wild creature in a place for reflection is a curious choice. Such a combination of chaos and containment, reflection and ardor within the visual program of a garden, is a clever balance not unlike the paradoxical quality of our Resting Satyr.

[SLIDE: Bartman Reconstruction]

As the display context of the sculpture shifted from divine to domestic in the Roman period, so did the complexity of display. These so-called copies were often placed in new relationships to each other in the Roman period, and in displays never envisaged by their original artists. For example, the Resting Satyr, now in the Centrale Montemartini was part of a sculptural display in a Roman house, most likely dating from the 1st or 2nd century AD, that was discovered in the 1940s underneath the Via Cavour. In the pendant display, a Resting Satyr is positioned opposite the figure of a general, and behind these two sculptures stand two Pothos figures. As Bartman points out, this particular display seems to be arranged with style in mind rather than programmatic complexity.

[SLIDE: HoloLens]

Communicating such a complex narrative for a sculpture in a digital format has proved quite a challenge. Before we could begin to tell a story, we had to determine how to communicate visually with a museum visitor, using an entirely new technology: the Microsoft HoloLens.

[SLIDE: Bodystorming]

We therefore held a series of “bodystorming sessions” in a room designed to mimic the museum in lighting and formality. We placed a 3D printed sculpture on a pedestal, and asked participants to demonstrate how they might look closely and take notes on a sculpture with the HoloLens. During bodystorming, one viewer mentioned that they would particularly like to see related archaeological material or more complete examples of the same sculpture.

[SLIDE: Eskenazi]

In Version 1, our goal was to take the findings from the user interaction study and craft an entirely visual narrative. We chose three sculptures: the Glyptothek, the Montemartini, and a digital reconstruction of the Eskenazi Torso. We immediately faced some technological challenges.

[SLIDE: Video]

First and foremost was displaying the photogrammetric models in the HoloLens. The headset can only display fairly low-level poly models, whereas our models consist of 450,000 vertices on average. The HoloLens simply could not display all the models at once. To solve this problem, we implemented a reduction method that maintains the quality of the model and its texture. This method was invented by a visiting professor in our lab, Dr. Gabriele Guidi, and his research assistant, Davide Angelou. The next challenge was to secure permission from the museum to display the experience in the gallery. Yet again the question arose: will augmenting the work of art really help a visitor to examine the piece more carefully? In this case, the design seemed to speak for itself. There would be no audio in this experience—only holograms of other known and related works. The best analogy would be that the museum was borrowing a sculpture digitally, rather than transporting it. Of course this has legal implications that I deeply hope will be resolved by work such as the charter presented yesterday. For now, we rely on permissions for photographic reproduction.

[SLIDE: Methods]

User testing for the experience was designing by myself and Andy Hunsucker, a PhD student in Human-Computer Interaction Design at IU. We compiled a set of methods to capture both the visitor’s initial impression, and their impression after having brief period to consider what they had seen. To capture the initial impression, we recorded the visitor’s visual experience using an in-headset camera that is part of the HoloLens. We can see and hear exactly what the visitor is experiencing, along with their comments. Alongside this, we also recorded separate audio tracks, which are useful to extract the visitor’s immediate auditory reactions. Finally, we setup a post-experience interview to collect visitor impressions after they had a few minutes to process what they had seen. I will share the highlights of these experiences, but full transcripts will be published at the end of the summer.

[SLIDE: Version 1]

In the first test, information about the three replicas of the Resting Satyr was displayed on small labels in augmented reality in front of the digital pieces. By placing the labels in augmented reality, we were able to reinforce the user expectation of the museum space. While all participants we interviewed agreed that the experience was consistent with their expectations within a museum, the purely visual experience failed to augment the sculpture. Most participants expressed that they had no sense of what they were supposed to be seeing. One visitor guessed that all the replicas were just more modern copies by different artists of a lost original. Another was more fascinated with the digital models than the torso in the museum. It quickly became apparent that, to achieve our goals, the experience needed an audio component.

[SLIDE: Version 2]

One of our primary concerns was avoiding designing cultural narratives into the experience, that while attractive in elegance are weak in historical support. For example, in Version one, I mentioned that one viewer guessed that each digital sculpture was just a more modern copy of a lost original. As Professor Elaine Gazda argues brilliantly in many books and articles, Roman artists were not merely “copying” Greek originals when they created a replica. Roman works of art were not merely straightforward copies of lost Greek “originals," known from literary references. Such narrativization often ignores or suppresses inconvenient questions about how the Romans integrated sculpture into their own cultural context and what they borrowed, emulated—or were inspired by—in a meaningful, even original, way. We did not want to communicate these restrictive viewpoints to visitors and inadvertently proliferate older art historical narratives that have since been revised. We therefore introduce our sculpture as one of many replicas, and discuss the Resting Satyr’s importance and role in both the Greek and Roman period, as well as the notion of the “copyist addition.” The Glyptothek Satyr presents an excellent opportunity to introduce this concept, as in this version, a cista covered with a Satyr mask rests beside the main figure, further connecting the satyr to Dionysiac ritual.

Based on our feedback from Version 1, to avoid inadvertently instantiating these types of narratives, the experience needed cues beyond the bare models and labels. The next version therefore included audio. Voice recordings presented a series of free-roam narrative segments related to the Resting Satyr. We hoped to achieve a conversational atmosphere, like you might have while visiting the museum with an expert as a guide. Like an audio tour, the experience is piped through the headset and does not interfere with the experience of other visitors. It solves the same problems as an audio tour in terms of enhancing the affordances of the museum label, but unlike an audio tour it is capable of tracking the visitor’s changing visual focus.

Once complete, version 2 was presented to the friends of the museum. In post-experience interviews, viewers were much more responsive about the educational elements of the experience. For example, one participant found the additional models uniquely helpful for visualizing the complete sculpture, and the accompanying audio helpful in gaining a fuller historical understanding of the piece.  Instead, most of the problems in version 2 instead surrounded user interactions. Viewers were not prepared to learn a new set of technology controls for audio within the five-minute experience. The advantage of the purely visual experience had been that it required no onboarding. After the introduction of audio, we frequently heard questions like “The sound went away. Can you fix it?” In addition, while version 2 had been more successful at communicating a complex narrative for the sculpture, it still did not give the viewer a sense of display context. In addition to providing additional education content, our hope was that adding in display context would also finally help enhancing the aura around the sculpture.

[SLIDE: Version 3]

The current design was produced in collaboration with a group of undergraduate students working under Professor Bernard Frischer and I. the students were asked to research the Resting Satyr, and their final project was to design a digital exhibit for the Eskenazi museum.

In the current design, a visitor wearing the Microsoft Hololens first encounters the physical Resting Satyr torso without any digital content. Some audio cues and small visual prompts slowly begin to appear after the HoloLens detects that the viewer has moved around the sculpture and looked carefully for a few minutes.

[SLIDE: Pasquino]

The augmented reality narrative begins in earnest with the Resting Satyr speaking to the visitor as if he were a speaking statue, like the ones right here in Rome. Rather than political and social commentary, however, he tells what he has seen during his long life. Beginning in the ancient Greek period, he discusses Satyrs themselves and their liminal role joining myth and reality. He considers workmanship and practices of sculptors, as well as his own materiality. He then takes a viewer to the temple of Dionysos, where they are invited to consider how he might have been displayed, who might have chosen to dedicate him there, and why.

Next, the Resting Satyr shrinks in size, and takes a viewer to a Roman atrium house. Studies of smaller decorative copies created for the courtyards of houses and private gardens and for less grand fountain houses and waterworks.

[SLIDE: Cicero]

To help the visitor question why certain sculptures were chosen for domestic display, we present the letters of Cicero to Gallus, which demonstrate how a Roman public figure like Cicero went about choosing sculpture in the first century BC. Authorized to purchase sculptures on Cicero's behalf, Gallus is said to have procured the wrong sorts of statues: he chose Bacchant subjects to adorn Cicero's library. Gallus seems to have justified his decision by comparing the statues to "Metellus's Muses" - a comparison which sends Cicero into a carefully staged rhetorical rage. After viewing several Roman display contexts, the visitor is invited to indicate which sculptures they also might have chosen for display.

Finally, the Satyr brings the viewer to Rome, to the pendant display underneath the Via Cavour. The viewer is invited to contrast the style of the Resting Satyr in the Montemartini to our Resting Satyr, as well as the style of the other sculptures in the pendant display. Then a reconstruction of the house surrounds the Resting Satyr torso, and the other sculptures from the pendant display slowly appear.

When the experience closes, the Resting Satyr returns the viewer to our gallery in the Eskenazi and invites them again to consider carefully his narrative, and how works of art both convey and accumulate stories around them. Although we foresee difficulties of previous versions persisting in our latest design, we are hopeful that it will be an excellent next step in achieving our goal of maintaining object primacy, while encouraging close looking.

[SLIDE: Conclusion]

The experience I’ve presented today, while not perfect, is the closest we’ve come to achieving our original goal of conveying the historical context of the Resting Satyr through natural, curious examination of the object itself.  Although we have not been able to fully test this latest version, and are certain more revisions will be needed, it is an excellent next step. I hope you have found our design journey an interesting one. More than anything, I hope to have critically reflected for a brief period if an augmented reality experience detracts from the aura of a work of art. In the experience of the work of art we transcend the confines of our historical moment and the walls of the museum to touch, and be touched by, everyone else who has encountered it. We do not achieve such a transcendent encounter through technology; rather, the best implementations of technology remove the barriers between us and that encounter: egotism, ignorance. The aura and presence of the artwork insists on the humanistic process of mourning and explication. Augmented reality technology may help us tell the story, but it is the work of art that moves us. Thank you.