For the screen-obsessed, watching a film at home poses a challenge: how can you concentrate while you’re also on your phone? Elijah Allan-Blitz’s Remembering – which has now launched on Disney+ in the USA, with an aim to roll out internationally later — leans into the distractions of technology by creating a film that can be significantly augmented by an iPhone or an iPad. It starts with a writer, played by Brie Larson, trying to remember something, before being magically transported into a world of pure childlike imagination. Prioritising mood and ambience over plot, the film shows us dolphin clouds, fairies cascading down rainbows, giant waterfalls and butterflies that threaten to pop out of the screen. And if you download the Remembering: The AR Experience app and hold up your device at the right time, they literally do; treating us to an extensive, immersive world that takes over your living room. The result is both enchanting and awe-inspiring, presenting many possibilities for the future of augmented reality. We had the chance to talk to Allan-Blitz about his experiences in the VR/AR sphere, combining 360 degree VFX worlds with immersive augmented reality, and his hopes for what the medium can achieve.
First of all, I’d love to know what got you into this augmented reality technology in the first place, and what you think its possibilities are for cinema?
Those are both big questions. My real foray into it was working in VR, which I’ve been primarily focused on for the last eight years, just really immersed in that space. I had seen an incredible VR experience at a TED conference eight years ago and I just knew this was part of the future. It wasn’t like the ultimate answer, it was just such a powerful medium and I wanted to be involved in it. And I had the opportunity to work on a lot of incredible projects. Through my work in VR I connected with the Disney StudioLAB who made Remembering possible. So this is actually my biggest jump into AR. This is the first full-on AR experience that’s going to be shared with the world. You haven’t seen anything like this because no one’s ever done this before.
Why not utilise that device to actually incorporate you and immerse you deeper into the story that you’re watching?
This is completely new technology! How hands-on were you in terms of development? Was there a big team and was it something you had before Disney came onboard? Or was there a collaboration in that respect?
I had the idea to do this and had been talking about doing this for five years, and was always told that it was impossible. It was perfect that Disney said it is always kind of fun to do the impossible. So when I connected with the StudioLAB team and I described this idea, they were like “we’re in.” It wasn’t like: “We have all the answers and we know how we’re going to do it,” but it was this collaborative, open-minded nature where they wanted to figure it out together. And it took an incredible team. We also worked with this incredible company called Magnopus. It just took the work of so many individuals. As you know, filmmaking is the most collaborative art form that there is. It can be an idea for one person, but you can have all the great ideas in the world; it doesn’t matter until you actually execute them.
With this technology, there is a matter of accessibility. For example, at film festivals there might be a separate VR booth but you have to queue or book a slot, while at virtual events you need the expensive technology at home. But here, all you need is an up-to-date iPhone or iPad. Is it exciting to know that it is being beamed into living rooms across the USA?
When working in VR, I’ve poured my heart and soul into projects that only get to be seen by so many people, whether at festivals or those with headsets. But this is a standalone short film. You can watch this without the AR experience and have a beautiful little journey through the world of imagination. But if you want to go deeper, you have this device that a lot of people are looking at while watching movies anyway. Why not utilise that device to actually incorporate you and immerse you deeper into the story that you’re watching? Then one day, who knows what that will be, there will be AR glasses, and then it will be a really seamless transition of the world coming into your screen.
You haven’t seen anything like this because no one’s ever done this before.
What were the technical challenges of creating the VFX in the film, creating them in the app, and then making sure they sync seamlessly?
The whole process has been difficult. There are so many unknowns and this is really something that had not been done. But we shot the film on one of those volumetric stages in LA. It’s similar to how they shot The Mandalorian, where there are these giant 25 foot high LED panels that are almost 360 degrees. So when they’re in the film, they’re not looking at a green screen. They’re looking at entirely built-out CG environments. The little girl and Brie are looking at the unicorn made out of moonlight or the nighttime environments. It was such a special opportunity to take that and incorporate it into the AR experience.
Were you always thinking in terms of explorability in terms of the AR? I also talked to Evan Goldberg from StudioLAB earlier as part of the demo. He told me that it could work on a laptop or in a very small room. What was it like making it adaptable?
This was not an afterthought. I wrote this script with the AR experience as part of it. I knew there was going to be a moment where what you were seeing was going to come off the screen into your living room. AR is something that brings you deeper into the story if you choose to engage with it. The team at StudioLAB made it possible to scan your room, so it has an idea of what’s around you. There can’t be things that are obstructing your view but if you have open space in front of your laptop, you can play it off a smartphone and have a mini waterfall. So you’re going to be able to adapt it all to the different environments that people are viewing this in.
When they’re in the film, they’re not looking at a green screen. They’re looking at entirely built-out CG environments.
I’m just thinking now about future use cases. Could there be a version of a film like this in the future with customisation? Could users themselves change the landscape of the film?
This is something we talked a lot about. But because this one was so groundbreaking on its own, we weren’t able to do all the things that we necessarily wanted to do. But there is so much more; even just being able to track where your hands are in space. Like if you stuck your finger out in front of your device and one of the digital butterflies landed on it. Or if you were able to go and find a hidden Easter egg that would then change the arc of the story.
It does make sense that the film is from a perspective of a child because I remember having this feeling where you might not understand everything and make up your own meaning of what a film is about. Was it always important to have this childlike perspective that allows for this interaction with the story?
I actually used to teach kindergarten. Working with kids is always something where I was like: I want to make stuff with kids. A) because it’s so fun, and B) because of what they’re going to say and going to do. With the little girl Dusty Peak, the star of the movie, so much of this was stuff she said. We never called action once when she was on set. We just had her on set, in this beautiful environment, it’s all there, and we would talk with her. We told her that the camera was her robot friend Alexa and they were in a staring competition not to look into each other’s eyes. And we were just rolling. It’s really just being able to collaborate with the most open-minded people, which are kids. It’s really the most amazing experience because I get to learn too!
AR is something that brings you deeper into the story if you choose to engage with it.
So much of that world of imagination was taken from talking with Dusty before we even filmed. We asked her what it would look like and she talked about rainbows that fairies slide down and dolphin clouds; we just incorporated it into the scene. She comes into the scene and we capture her real reaction. The shot where you see the shooting star cross her screen reflected in her eyes, it’s not effects. That was a real reflection in her eyes that was coming off the LED stage she’s watching.
And how did she and Brie Larson get on on set?
Oh, the best. Brie is incredible at working with kids. Dusty is our neighbour, so we were actually going to be shooting this. We put something together during the pandemic with her and her family and thought about doing something. Then Disney asked what we were doing and it became this much larger project. It’s so fun watching them together: even the scene where they are mimicking each other in the end, that was completely impromptu. It was just the sweetest stuff.
What are you working on next?
That is still to be determined. There are a few different possibilities and we’re just figuring out with the response to the launch of this what’s the best way to expand it. There are a lot of open-ended questions that this poses that we can definitely continue to explore!
