25/01/2018

Visgraf's project combines virtual reality, cinema, and theater.

Karine Rodrigues

Thursday night in Rio, 15 minutes before the start of William Shakespeare's "The Tempest," the area in front of the auditorium is buzzing. Filmmakers, musicians, TV writers, choreographers, and artistic directors chat while waiting for the theater to open. Nothing unusual in the cultural scene, were it not for the fact that the performance is at IMPA.

At first glance, the space may seem unusual for an artistic performance. But, precisely because it produces excellent research in pure and applied mathematics, it is understandable that it would be the setting for something as innovative as an experiment that seeks to push the boundaries of what is possible in theater and virtual reality.

The show was created by researchers from the Visgraf (Vision and Graphics) Laboratory at IMPA , in partnership with a group of performing arts professionals. It combines theatrical performance, film, game technology, and virtual reality. Multidisciplinarity, in fact, drives the laboratory, which was founded in 1989 to conduct research in computer graphics. It is coordinated by Luiz Velho, who holds a degree in design and computer graphics and is a senior researcher at IMPA.

In the nearly full, hundred-seat auditorium, the audience doesn't quite know what to expect. Anthropologist, TV screenwriter, and music researcher Hermano Vianna thinks about the discomfort the virtual reality glasses will cause him – the magic of 3D immersion has its price… He breathes a sigh of relief when Luiz Velho clarifies that the equipment will be used by the actors, not the guests.

For about 20 minutes, three excerpts from the play written by the English playwright between 1610 and 1611 are shown on the screen. It tells the story of Prospero, who lost his title as Duke of Milan after being betrayed by his brother, Antonio. Isolated on an island with his daughter Miranda and his assistant, the spirit Ariel, he uses his magical powers to create a storm at sea and shipwreck the ship where Antonio was.

What you see on the screen are avatars, entirely digital figures created from three actors – Helena Varvaki (in the role of Prospera, a female version of Prospero), Daniela Salles Abreu (Miranda), and Rick Yates (Ariel) – who are in two rooms located on another floor of IMPA and are acting in a three-dimensional environment using virtual reality equipment. The action, it should be noted, is live.

When the show ends, the production team gathers in front of the audience to exchange ideas about the experiment. Helena reveals to the audience how difficult it was to adapt to the virtual reality equipment, and the peculiarity of performing in another dimension.

“I gave the entire technical team a lot of trouble. I got very upset when they messed with my head,” confesses the actress, highlighting that, despite the strangeness, the immersion in the world of virtual reality is very intriguing.

The show's director, Manoel Prazeres, explains the process: “What the actors see when they are performing is not what we see on the screen. They are in a three-dimensional environment. They extend their hand and actually see the hand of their avatar,” he details, adding that the equipment captures the body movement and voice of the actor, who gives life to the avatar.

Prazeres, who also holds a bachelor's degree in Mathematics, worked as a software developer and later moved into the performing arts. He observes that, within virtual reality, the actor needs other ways to express themselves. He had to learn to master the technology. Since everything happens live, he needs to select the cameras in real time. It's as if he were acting alongside the actors.

Questions are popping up in the auditorium. Liana Brazil, artistic director of SuperUber, a creative studio with international reach in the areas of culture, design, and entertainment, wants more details about the levels of body movement capture. Someone mentions the strangeness caused by the use of a mask on the avatar, quickly justified as a way to hide the immobile eyes – virtual reality glasses prevent the tracking of these movements.

Luiz Velho explains that the experiment is the result of a research project that began nine months ago and combines theater and VR Kino (virtual reality technology). "It's the first time that actors, using virtual reality, are totally immersed in a virtual setting and their performance is displayed live to the audience on a cinema screen, as real-time computer graphics, under the interactive control of a director."

Designer Hans Donner asks for the microphone. He recalls the time in the early 1980s when he worked with Luiz Velho at Rede Globo, taking his first steps in the field of computer graphics. "This is impressive!" he says, asking how much investment, in terms of time and money, is needed for the developed technology to be able to track body movement perfectly.

“You have to be patient. If we wait, all this won’t be expensive,” Velho replies, emphasizing that the important thing in the work presented is that technology is at the service of art. “Creativity is what makes this thing work. The important thing is the artistic work. We can explore many possibilities that don’t yet exist. It’s necessary to focus on the tool itself, as a form of communication, entertainment, and everything it will bring in the future.”

This prompts filmmaker Sandra Kogut to highlight how the experiment allows for the emergence of new narratives. "I was very curious," she says, considering that the show began with the most difficult aspect, a kind of subversion of traditional languages. "In theater, the most intense sensation is having a person in front of you. And in film, you believe what you're seeing."

Luiz Velho then explains the importance of innovation and says that he will submit two papers based on "The Tempest" to the world's largest conference on computer graphics and interactivity, Siggraph.

Leaving the auditorium, Hermano Vianna comments on how interesting it is to act alongside a machine, to be inside its world in real time, as the experiment demonstrated. “It’s professional theater; you see the avatar’s performance on the screen, but the people are in other places. And all at the same time. It could all go wrong. In these live experiments, I’ve seen things go wrong in the most powerful companies,” he observes.

The idea of presenting “The Tempest” to a specialized audience arose from the need for an exchange with experts in the field, considering the multidisciplinary nature of the project. In fact, the audience included, among others, Lucia Modesto, visual effects supervisor at TV Globo; Rosa Maria Araujo, president of the Museum of Image and Sound (MIS); Analivia Cordeiro, choreographer; designer Eliane Jobim and musician Paulo Jobim; and Carlos Gradim, president of the Rio Art Museum (MAR).

In addition to the Visgraf team – Leo Carvalho (technical direction), Djalma Lucio (system development), Julia Gianella (research and design), and Vitor Rolla (research and sound design) – the show was produced by Affonso Beato (virtual cinematography and lighting) and Eduardo Cronemberg (masks and costumes).

This Thursday, at 3:30 PM, in auditorium 3 of IMPA, Luiz Velho and Manoel Prazeres will detail the technology used in the seminar "Making The Tempest: a glimpse into Shakespeare's magic". Based on what has already been done and the debate held after the screening, new experiments are sure to emerge from "The Tempest". Brazilian innovation is grateful.