Holography has become a staple of science fiction television shows and movies, ranging from holodecks where individuals can roleplay with simulated environments and characters, ads the size of skyscrapers populate the urban landscape, and principal cast members are made out of light particles. Although these productions are fun to watch, they take creative and technical liberties and these “holograms” are not truly holographic as science defines them, notes Jeff Barnes, EVP, creative development at Light Field Lab: “Hollywood’s portrayals of holographic technology are visually stunning but often defy the laws of physics and are only possible through visual effects.”
Cinematic depictions of holograms are evolving. In Ghost in the Shell (2017), for example, the portrayal transitioned from still to video camera capture. “We had to use vision cameras which are made for surveillance, have no shutter so there is no refresh, and have an electronic sensor that can be synced perfectly,” explains Guillaume Rocheron, who was a co-vfx supervisor for MPC Film on the live-action adaptation. “The rig consisted of 80 video cameras. We had three computers and boards everywhere to keep the cameras in sync. We shot at 24 fps, and MPC developed a pipeline to solve that on the render farm. We then brought it into Houdini, and, from that, created the shell of the subject. The whole concept was that it’s part of the environment and volumetric, like a real hologram.”
Arguably the most iconic holographic-like depiction is a pre-recorded message projected by R2-D2 of Princess Leia Organa (Carrie Fisher) pleading for assistance from a Jedi master in Star Wars (1977). “Princess Leia’s hologram was really effective because of how and where it was introduced in the movie,” states Praxis Film Works director and producer Robert Blalack who was part of the Oscar-winning visual effects team on Star Wars. “Its content was linked to the story.” Academy Award-lauded special effects cinematographer Richard Edlund, ASC had to rely on optical effects and practical solutions such as placing a television set on its side to have vertical rather than traditional horizontal raster lines in the video image. “I put a clip of the scene with Leia in the viewfinder of the camera, lined up the image, and shot the element with a reflex camera on a black stage with nothing but the TV set there. That image was then isolated by the roto department and was semi-superimposed over the plate. The plate had some bleed through of the image so it looked like it was hovering in space.”
Contributing to the success of the Princess Leia hologram is the fact that it is seen from various angles; this is something that ILM VFX Supervisor Scott Farrar expanded upon when a grieving Jon Anderton (Tom Cruise) watches home movies of his missing son in Minority Report (2002). “It’s hard to create a hologram that looks dimensional from a single viewpoint,” says Farrar. “The only way you can understand that the hologram is 3D is by circling around it. I had been fooling around using photographs [taken from various points of view] to create geometry for a 3D shape and character. I had visual effects coordinator Wayne Billheimer sit in a chair and stare at a washer hanging out in space. I did a big dolly move all the way around. Steven Spielberg let me direct the boy, who was shot on a flat riser with some sand. He was a composited element, and in the background we put the beach scene.”
Stereography and deep compositing have furthered the illusion of depth with the orrery (a mechanical model of the universe) in Prometheus (2012); however, set integration was not easy. “We had to repatch David’s [Michael Fassbender] face because the Earth he was holding was bigger than the real one,” says Jason Bath, former vfx producer at Fuel VFX. “It’s a lot of technical work to integrate it.” Deep compositing allowed for various layers of effects. “If we had to re-render the entire orrery each time we needed to add an Engineer, a planet or a laser beam, we couldn’t have created new versions fast enough,” he adds. “Deep compositing was a big part of the success of iterating, getting feedback and delivering that show.” Influencing the look was the ability to touch a spot on an iPad and expand it out with your finger. “We began to riff on the idea that in the center of the room could be almost a volumetric lens,” explains Paul Butterworth, who at the time was a vfx supervisor at Fuel VFX. “If you grabbed a bit of data and moved it into the center of the room it would unfold out.”
Shifting from physical to vocal interaction with holographic user interfaces, Tony Stark (Robert Downey Jr.) pilots his Iron Man suits via heads-up displays (HUDs) that are driven by his conversations with the AI operating system J.A.R.V.I.S. (Just A Rather Very Intelligent System). Inspiration for the layout and functionality comes from the military and car industry. “We attack the HUD more as a 180-degree cylinder, because often we didn’t end up designing anything for the back of Tony’s head,” remarks Stephen Lawes, creative director and co-owner, Cantina Creative. “The challenge there is what Tony is looking at and what he needs. Most of that information should be in front of him and we then fit things in that make sense. Ultimately, you have your horizon line in the middle of frame where it needs to be and we often had the altimeter right of frame. The dock widget that housed his weaponry and energy and RT levels was always to the left of frame.”
One economic sector that has the ability to develop and commercialize emerging technology is advertising, and Ghost in the Shell explores what the world will look like with high- and low-end volumetric ads and signage. “For the solograms [solid holograms] that are the scale of a skyscraper, MPC introduced this idea of a 3D pixel that breaks down into smaller cubes, so you get some of the imperfections that come from projecting into real world environments,” states David Sheldon-Hicks, co-founder and creative director of motion at Territory Studio. “Rupert Sanders [the director] did not want everything to be perfectly designed. You might have a fast food takeaway that does not have as high-definition projection on the side of a street as compared to Nike advertising on the side of a skyscraper. The other bit that was interesting to play around with was the idea of holographically projected street signage that could update and be fed by machine learning and artificial intelligence. There is a dynamic functionality to it that is beyond just advertising.”
Going from a commercial to a more domestic application of holography, Blade Runner 2049 (2017) introduces the idea of the virtual love interest Joi (Ana de Armas). “If Joi is performing, talking and interacting you don’t register that there’s a volume to her,” remarks DNEG VFX Supervisor Paul Lambert. “It’s only when she moves that you suddenly realize she is quite transparent.” Joi visually merges with human prostitute Mariette (Mackenzie Davis) in order to make love to K (Ryan Gosling), which required 3D models to be created for each of the actors. “Denis, John, and I had an iPad connected to the video village,” says Lambert. “It would show a dissolve between the two performances and there was a particular time when it happened to stop on a frame where both of their eyes were in sync. Denis said, ‘That’s the look!’ Joe Walker, the editor, along with Denis, would tell us at certain points where they’d like the eyes to come together in the edit. It was a true collaboration.”
What if holography evolves to the point where a person could actually displace the light particles? In Ready Player One (2018), Wade Owen Watts (Tye Sheridan) meets his rival Nolan Sorrento (Ben Mendelsohn) via his virtual reality game avatar, Parzival. “We pretended that there were multiple projectors in Sorrento’s office that would pick up and let go of the hologram as it moved about the room,” explains Digital Domain VFX Supervisor Joel Behrens. “In a traditional hologram, if you walked through, the projector beam would flicker and then it would reassemble. However, we thought a nice idea would be to have Sorrento interact with the projection as if he were in the same physical space as Parzival. Basically, as an insult, he is walking directly through him at one point in the scene. We had the hologram not only breakup but actually react from the motion of his body.”
One of the more accurate holographic representations is the Star Trek Holodeck, which is a room constructed entirely of holographic panels. “It conceptually demonstrates a light field representation of object formation where the viewer’s direct line of sight is maintained between the holographic object and the display surface,” remarks Barnes. “This is, of course, until the holographic characters somehow become sentient and walk off of the holodeck!”
An important component of Star Trek is staying true to the various timelines of the franchise, which includes how holograms are presented. “The second season of Star Trek: Discovery [CBS, 2017–present] tied into the original series [NBC, 1966–69], so the holograms had a fall-off from the glow as well as a little of transparency,” remarks Jason Zimmerman, who serves as the production visual effects supervisor for Star Trek: Discovery and Star Trek: Picard (CBS, 2020). “In the case of the multiple Cristóbal Rios [Santiago Cabrera] on the La Sirena in Picard, it was the opposite. We had progressed enough in the technology that each of the holograms felt like they were there all of the time.”
Technological advances are quickly closing the gap between fiction and reality, which intrigues Lawes. “The thing that we’re looking into right now is how movies influence real-world products and vice versa.”
And experiments are currently taking place in reverse engineering. “There are displays in early development that are not truly holographic but align closer with some of the examples used in movies and television,” states Barnes. “For example, Dr. Daniel Smalley and his team at Brigham Young University are trying to recreate the Princess Leia projection in what is referred to as a volumetric display.”
Sheldon-Hicks is fascinated at the creative and technological possibilities of holography. “I built a design studio on the dream of one day creating full-blown holograms, either for storytelling or communication or work,” he says. “The idea of watching a film volumetrically with your family instead of [on] a TV would be absolutely incredible.”