Premiering as as one of 30 projects in Tribeca’s immersive category in April, Ethan Shaftel’s 360 film Extravaganza questions the role of VR as a touted ‘empathy machine’ – trapping its audience in the subservient role of a character preprogrammed to perform in a stunningly offensive puppet show for a delighted, clueless executive. DN spoke to Ethan about the roots of his VR passion, the opportunities the medium provides for indie filmmakers and how he drew on the staging techniques of theme parks and haunted houses to most effectively tell this ‘look anywhere’ piece of biting satire.
When we last spoke, you highlighted your interest in the concept of point of view as a strong motivating factor behind the creation of the sci-fi short Flesh Computer. Is that what brought you to the realm of the VR narrative?
First person point-of-view was a central concept to Flesh Computer and a lot of my work, and it’s also very relevant to virtual reality. But it’s important to note that what is meant by POV is very different between traditional film and VR. In film, POV is an abstract concept that is constructed out of many many choices that all add up — camera placement, editing, and screenplay structure all work together so that the audience experiences the story vicariously through the protagonist, through their emotional POV. And occasionally in film there are literal POV shots where you look ‘through the eyes’ of the protagonist. But these are actually relatively rare because first person POV is a pretty bad way to see and understand action, and certainly the worst way to see the face and emotions of our protagonist and feel what they feel. POV shots in movies are used as detail to bring emphasis, as in Flesh Computer when certain moments are seen through the eyes of the computer, or through the eyes of the dying housefly — but those shots only have real meaning because of the juxtaposition with other coverage of the action to tell us WHAT is happening to these characters.
In VR, POV is a concrete concept. It really is camera placement, where you are literally INSIDE the virtual world. In VR you can be inside the head of a character, or you might be an ‘objective’ camera seeing the action from outside the characters, but you are always located somewhere SPECIFIC in space, in a way that a viewer never is in traditional film.
Let’s have them be mocked, attacked, ogled, and killed, a chance to see the worst in the both the protagonist and the audience itself.
As a kid I was enormously taken by Myst as a first person cinematic narrative, even though it was of course also a game. Most of my career has been spent in linear narrative film but I do have a background in interactive design, and a background in immersive ‘spatial’ media such as video installations and multi-screen media for pop concerts, theme parks and other environments. This required me as an artist to think about viewer Point Of View as a much more concrete concept then in film.
So when this ‘wave’ of VR started to appear, it just seemed like a perfect fit for my background and interests. That kicked off a period of about 18 months of writing and prototyping VR experiences Last fall we decided that the key to getting any of these projects off the ground was to create one standalone VR movie to get into a film festival, a movie that we had the resources to make ourselves — and because of my collaborators that meant a largely animated move. Tribeca had the latest entry deadline of the festival season, so we made that our target. Everything else went on hold as we sprinted through production on Extravaganza, and luckily the gamble paid off because it was accepted into Tribeca immersive film category and premiered at the festival this April.
The film’s conceit provides a logical vantage point from which the viewer can witness the story, along with the motivation for their active but externally directed participation in it. What inspired the story and spacial logic of Extravaganza?
You’ve hit the nail on the head with your question. Along with POV, the other key concept in VR is ‘agency’ — what you can or can’t do, what effect you have in that world, and the logic behind that. The idea of making the viewer a puppet was one way of reconciling the the fact you are an implied participant by virtue of being locating INSIDE the world of the story as a character with your lack of control of the characters actions in that story. In some projects that is reconciled by making the viewer very passive, locked up or in a wheelchair for instance. But the puppet idea allows viewers to be very active, a full participant in the show, while also acknowledging that the viewer is FORCED to perform the actions against their will.
Everything about the movie grew out of that. For instance, to make that conflict more extreme, I decided that the show you are forced to perform in should be horrible, one that you would never have chosen to participate in. Rather than be the hero, which is how most film stories are experienced via the protagonist, let’s make the viewer a series of side characters and victims in various ways. Let’s have them be mocked, attacked, ogled, and killed, a chance to see the worst in the both the protagonist and the audience itself — in the form of Paul Scheer’s character who is watching the puppet show inside the movie. So that is Extravaganza in a nutshell, and hopefully it’s an opportunity to use point of view to satirize and illuminate something and make a bit of a real point about entertainment, VR, and society.
This is certainly not the only workable way of reconciling the issue, in fact it’s probably going to look pretty cliche when we look back in the future. We’re relying on a set of assumptions and crutches that we will likely grow out of in the future. But for me, every successful VR piece must grapple with the POV / Agency issue in some thoughtful or clever manner, and the process of articulating how your VR experience deals with it is a big step in creating the piece itself.
How did you come to cast Paul Scheer in the role of lascivious executive? What was he playing against during filming?
I was introduced to Paul through a friend who is a comedy writer and producer. He knew that Paul was interested in VR professionally — he had been in a VR project from Funny or Die that went to Sundance last year — but also personally, since he owns a high end VR system and is just a big fan and believer in the technology and potential of the medium itself. Thankfully, he was totally enthusiastic. It is nice that right now the novelty of the medium means that actors aren’t yet inundated with indie VR jobs as they are for film, shorts, web series, etc. That applies to crew and tech personnel as well — on Extravaganza this was the first VR project for the cinematographer and production and post sound teams as well. Collaborators are interested in getting involved at least to find out what the fuss is all about.
When Paul has his face up close to the puppet show in the movie he’s looming 20 feet above you. We literally had him with his face just inches from the camera lens, resting his forehead on a little guard we rigged up, with little lights shooting right into his eyes. His whole performance was done staring at a couple of little dots we put around the lens for eyeline, and delivering his lines while really emphasizing the movement of his eyes and eyebrows, since that is mostly what is visible.
You integrate that live action footage with 3D animation, could you walk us through the production processes undertaken by yourself and Designer/Animator Frank Stringini to realize this immersive off-color puppet show?
Throughout Extravaganza most of your field of view is animation at any given time. Extravaganza is 360 degrees, so you can look in any direction at any time, but because most of your field of view is an animated world, the live action section is only a small slice of that 360. So we could shoot that with a traditional cinema camera instead of dealing with the difficulty of shooting 360. The tools are improving for that, of course, but there are a lot of obstacles still, both technically and just plain cost/availability of equipment.
Extravaganza was created in Cinema 4D, which has a great plugin called CV-VRcam from their Cineversity subscription to render out from a virtual 360 camera. So much of our work-flow was using the same animation tools that Frank already is used to for character design, modeling, animation, and lighting. The differences for VR came in conceiving of the first person experience and staging the action to work elegantly with that limitation of being one continuous shot. That is of course a non-trivial difference from traditional animation for film, but it’s not really a technical challenge as much as a creative one.
Stylistically, the film’s design seems work with some of the limitations I’ve noticed in other VR content – such as the screen door effect – instead against them. To what extent did you factor the technology limitations into the look and feel of the film?
The film was made to be both cross-platform and also (hopefully) future proof. We mastered Extravaganza at much higher resolution than any devices are capable of for playback, for instance, which took some time and effort but will hopefully increase the life of the movie in distribution.
There were some conscious choices to work with the limitations of the medium as well. For instance, when it comes to design of the characters and the puppet show ‘world’ all around you, we decided to make it feel very analog and low tech. Everything is like the inside of a cuckoo clock or mechanical box — the paint is peeling and blotchy, the metal is dented and imperfect. I think the effect is that if your particular VR device is going to be displaying Extravaganza low resolution, or with various other visual defects, that it works WITH the kind of shabby look rather than against it. I also find that the analog nature makes it more inviting then if we had every look digital and high tech.
First person is usually the worst way to see what your own body is doing. Unless your hands are in front of you, there is really not much to see but your chest.
Also, I am fairly sensitive to motion sickness and eye strain when it comes to VR. Now as the hardware of the VR headsets and playback improves these become much smaller issues. I didn’t want to dumb down the movie, but I did want to make sure that we were making something that would be a comfortable experience on existing headsets, so we made choices about the camera movement and contrast to mitigate issues. More and more research is being done on this because of VR’s resurgence, so I read a lot and we made the movement of our mechanized stage and characters work within some limitations that people have discovered. I believe it worked, since I’ve gotten a lot of feedback from viewers who have problems with other VR experiences that did not with Extravaganza.
The ability to explore a free-ranging narrative space is a key attractor of VR storytelling, what were the unforeseen challenges of staging a film within a ‘look anywhere’ environment? What techniques did you use to direct viewer attention?
This was the central focus of the work on the movie, how to ‘stage’ our puppet show so that you are both participating in it, and viewing it for the first time as a spectator, in a way that is elegant. There are so many considerations. For instance — first person is usually the worst way to see what your own body is doing. Unless your hands are in front of you, there is really not much to see but your chest. You certainly can’t see your own face or head. So every choice comes directly out of this challenge.
In Extravaganza you are always part of a pair or group of identical characters, so you can see yourself and your partner at the same time. This isn’t something that you really need to think about, but the result is you have a image in your minds eye of what YOU are doing that just couldn’t be there any other way. Similarly, we built a mechanized stage that moves you around in a circular way, and you see different scenes from different vantage points.
Sometimes the movement of your own hands in a particular direction draws attention to the next gag or the next character. Sometimes it’s lighting — one area of your view grows dark while a light brightens or changes color on your peripheral vision, which draws your eye there without you really noticing why you looked. Other times characters behind you cast an obvious shadow across whatever was most important in the moments leading up to it. You notice movement and turn to look at what cast the shadow. All sorts of techniques that designers have been using in theme parks and haunted houses or other immersive theater for a long time.
Sound is very important, of course, and Extravaganza has a fully spatial 360 sound mix. But the reality right now is that on most of the VR headsets it is not yet as effective as it needs to be to really draw attention the way you would in a piece of live theater, for instance, when an actor comes in from the back of the house. This is not just a hardware limitation, it’s also the software, the format and compression needed for large 360 spatial sound mixes to playback from your computer or mobile device. So yes, you can certainly hear the difference between something on your left and something on your right, but I’ve found I can’t rely on an off screen sound to really result in the viewer looking over in the correct direction automatically… yet. I think that will be the case as the tech develops for sure.
Pretty much every form of widely available technology brings with it moral panics and fevered decriers. Where do you fall in the argument of technology’s effects on society? Do you want Extravaganza to illicit a particular reaction from its objectified and exploited audience?
I think the simple theme of Extravaganza is that it’s not the platform or medium, or technology, that causes good or ill in society, but the art itself. The puppet show that you are a part of in the movie is not making the world a better place. The viewer of that show, Paul Scheer’s character, has his prejudices confirmed and catered to. He walks away less empathetic, if anything. By not challenging anything at all, the makers of that show are at BEST keeping the status quo, and at worst they are strengthening a prejudiced viewpoint. At the end of Extravaganza, the puppets manage to escape and destroy the puppet show from the inside. The puppets aren’t the problem, but their programming, the show they are forced to perform, is the thing that has to be destroyed, or changed. And maybe it’s the fact it was made to cater to the whims of a prejudiced audience that is the real problem.
Extravaganza premiered earlier this year as one of the projects in Tribeca’s immersive category – a programme largely dominated by well funded, big name projects. How difficult do you feel it is for indie filmmakers to make a mark in the VR space at the moment? What was your experience eliciting feedback on something which is such a singular experience?
I’ve met many great indie VR creators — some from film, some from games, some from photography and fine art. I think that because the medium is new, there is a lot of room to make a mark. Some of it’s just a numbers game, and fewer people are making VR than are writing novels or making web videos. But because the medium is new, there are obstacles that don’t apply to a creator looking to start writing, or making short films or indie games.
The cheapest 360 cameras are not yet attached to your smartphone, but they are definitely less expensive than your iPhone, so that barrier is low. YouTube and Facebook both support 360 video. So if you are just getting into making movies for the web, you could easily decide on 360 as a format, especially if you conceive of your work as a desktop or magic-window based experience (when you move your phone around to ‘look’ into the 360 environment in all directions).
It’s not the platform or medium, or technology, that causes good or ill in society, but the art itself.
The big challenge is that so few people have head mounted displays, that if your wok depends on that (like Extravaganza because of the body presence as a puppet) then it is really difficult to find much of an audience online. It’s hard to control the distribution to make sure people are seeing it right, and that is a big part of making your movie work. So the great part of the film festivals that are showing VR right now, is that it’s a way to make sure people are seeing it on a head mounted display.
As far as gauging the reaction of viewers — it is in some ways much more intimate to watch someone experience your VR movie at a festival. You are only watching one or two people at a time, and you can see exactly where they look and what captures their attention. You can see if they are smiling or frowning, and hear if they gasp or laugh. Plus at the festival it was easy to hang around and chat with people about it. But of course it’s different than the atmosphere of a packed house at a festival premiere for a traditional film. VR is very different than film in many ways, but one of them is how you experience it. At least right now, because massive social VR is not off the ground… yet. But that’s coming fast.
Will you be venturing back into VR?
The projects that I’m pitching now are all VR or VR-related, but they are all very different from each other and from Extravaganza. And they are all designed to fit within current technology, current distribution and monetization in different ways, so I’m hoping they are appealing to producers and financiers working in VR in different segments. These include location based experiences with multiple participants, to fully interactive narratives meant for download and home view.
But right now I’m trying to keep the emphasis on the same fundamentals I explored in Extravaganza, which is how to tell a first person narrative that is both participatory AND includes the type of arc and catharsis and pacing that people love about protagonist-based narratives in film. I have multiple answers to that question, and the feedback has been really good from the companies and creators I’ve gotten access to thus far!
