The most fundamental mission of a museum is to curate the past. “Look but don’t touch” was the museum’s mantra for decades. But thanks to a new generation of interactive AV technology, the museum is increasingly now touching you.
Those include:
• Virtual reality (VR) using headsets that can essentially transport visitors inside exhibitions, allowing them to be totally immersed in an environment
• Augmented reality (AR), which achieves a similar goal but lets the viewer keep one foot in their current reality by blending physical and digital realms.
• Artificial intelligence (AI), the latest technology being applied to increase informational feedback and enhance interactional communication.
This isn’t technology for its own sake, however. The number of museums in the US has been steadily increasing, with more than 33,000 of them in the US alone. This growth has led to more competition between institutions, whose funding sources range from grants and governmental appropriations to private and patron endowments, which also depend heavily on admission fees, visitor donations, and sales from the inescapable museum stores located at every egress, to fund more engaging technology. The numbers can be eye-watering: for instance, the Cleveland Museum of Natural History will open a $150-million expansion in 2024. But like any entertainment venue, museums are competing for consumers’ disposable dollars, and like any entertainment proposition these days, they’ve been turning to cutting-edge technology for that.
Immersive AV
It’s not just for IMAX cinema anymore. If you’ve seen ads for — or even been to — the touring “immersive art” installations featuring the works of van Gogh, or the Marc Chagall and Paul Klee installation at L’Atelier de Lumières[1] in Paris, you know the old masters still have a trick or two up their paint-stained sleeves. These installations utilize a combination of projection / mapping technology and 360-degree sound to take viewers inside the brush strokes. Produced by Lighthouse Immersive, the peripatetic van Gogh exhibitions have deployed 90 million pixels and 60,600 frames of video across 500,000 square feet of space in 22 cities so far, from New York to San Francisco, putting the Dutch artist’s 19th-century masterpieces like “The Potato Eaters” and, of course, “Starry Night,” into a three-dimensional medium no canvas could emulate.
More ambitious are the (so far) six ARTE MUSEUMS of digital art created by South Korean design company d’strict, the latest opening in Las Vegas on Nov. 29, 2023. The two-story, 30,000-square-foot structure comprises 14 immersive projection-mapped impressionistic installations inspired by natural phenomena. But the immersion doesn’t stop there. The exhibit’s multichannel audio ranges from meticulously detailed and lifelike sounds of nature to evocative background music, but the olfactory piéce de resistance are the recreated fragrances of nature throughout the exhibition, which further heighten visitors’ sense of immersion.
AR & VR
Augmented reality’s most basic interactive application is as a portal to annotation, usually through a QR code accessed through a viewer’s mobile device. These can also include audio narration, three-dimensional graphics, and animation. For instance, the Muséum National d’Histoire Naturelle in Paris launched “REVIVRE”[2] (“To Live Again”) let visitors come face to face with digital iterations of now-extinct animals via an AR experience using Microsoft’s HoloLens. Similarly, 13 of the animal skeletons in the Smithsonian’s Bone Hall get skin and musculature through its Skin and Bone app.
Virtual reality is a bit tricker but the payoffs can be spectacular. VR requires headsets, which can be bulky and make the wearer self-conscious, but those discomforts fade quickly when one uses them to, for instance, step into the year 1863 to explore R. W. Ekman’s 1863 painting, “The Opening of the Diet”, at the National Museum of Finland in Helsinki. The experience includes the chance to speak with the Russian Emperor Alexander II and other characters depicted in the painting. Similarly, London’s Tate Modern created a VR environment for its pre-pandemic Modigliani retrospective in which visitors were able to experience complete immersion in a 3D model of the artist’s Paris studio, which used the actual studio space as a template to digitally fabricate a faithful recreation of the artist’s studio as it would have been 100 years ago. (The room itself, on the Rue de la Grande Chaumière near Montparnasse, still exists but not as it was then.)
Hello Holo
True holograms, allow visitors to interact with virtual 3D artifacts within a real exhibit environment and give viewers the ability to use gesture to “move” and inspect these artifacts freeform in space. Being able to manipulate holographic objects gives museum goers an opportunity to virtually inspect an exhibit item while also being able to experience the frisson of proximity to a fragile and valuable original from a bygone era. Holograms are, as one paper3 expressed “a means to reanimate the past.” Holography has the potential to allow entire museums to become virtual and thereby vastly more accessible, and as a result, may also offer the often fund-starved museum sector another financial lifeline.
The Morpheus Project, initiated by self-described “deep-tech start-up” Perception Codes in London, is applying 3D objects to another digital phenomenon, the non-fungible token or NFT, to create virtual representations of art works — dubbed “Holo-NFTs — that patrons can purchase and own the blockchain-based nature of the NFT assure provenance. As Perception’s CEO explained[4] to the MuseumNext website, “The Morpheus Project allows museums and galleries to digitize their content and exhibit works holographically…. [T]he fact that the assets are then tokenized allows them to effectively raise funds from these assets.” It could represent the ultimate digitalization of “Doing well by doing good.”
Artificial Intelligence
While various technical “realities” are producing some amazing images and the proliferation of multichannel audio like the Dolby Atmos format can produce immersive aural cosseting, applying AI to them can supercharge that effect.
An example is “Voices From The Front” at the National WWII Museum in New Orleans, an interactive experience that lets visitors carry on narrative “conversations” with WWII veterans who are no longer living. Scheduled to open in early 2024 as part of the Museum’s new Forbes Gallery of Rare and Iconic Artifacts, museum goers are able to select individuals from the war era to including Iwo Jima veteran and Medal of Honor recipient Hershel Woodrow “Woody” Williams, aircraft factory worker Grace Janota Brown, and “Bloody Hundredth” bomber pilot John “Lucky” Luckadoo, and interact with and ask them questions. The AI program, created by StoryFile with technology by Ideum, will formulate responses in real time from upwards of 1,000 preprogrammed questions about the subject’s life and wartime experience from these and over a dozen other veterans, spoken in their actual voices.
The Next Step…
The degree to which interactivity can be taken has reached some fairly astounding levels, and in recent years has been extended to users’ own devices such as smartphones and tablets. For instance, visitors to the ArtLens Gallery[5] at the Cleveland Museum of Art can place their iOS or Android device on the ArtLens Wall dock, developed by Zagreb, Serbia-based DitDot Ltd., which consists of smart screens with motion detection, eye-tracking, and a camera. They can then move, distort, annotate, and otherwise reconfigure the virtual art displays. It’s essentially remixing art, not unlike how music fans can pull recordings apart digitally and break them into stems to remix their own versions of them.
Cuseum, a digital-engagement specialist that integrates augmented reality for cultural institutions including museums, has gone one step further, adding a connection to the “true crime” phenomenon fostered by podcasts like Serial. In 1990, the Isabelle Stewart Gardner Museum in Boston was the target of a still-unsolved theft of 13 pieces of art worth more than $500 million, including Rembrandt's "A Lady and Gentleman in Black.” Dubbed “Hacking the Heist,” an AR app allows museum visitors to focus a phone or tablet on an empty frame on a wall and the app then populates images of the purloined pictures[6]. Adding spice to the adventure is the fact that the museum is offering a $10 million dollar reward[7] for information leading directly to the safe return of the stolen works.
As museums move deeper into an interactive landscape, you may never look at them the same again. On the other hand, they’ll also increasingly be looking back at you.
[1] https://www.atelier-lumieres.com/en
[2] https://www.youtube.com/watch?v=So_4UzvyPO4&t=2s
[3] http://www7.bbk.ac.uk/vasari/2019/03/01/holograms-in-the-museum/
[4] https://www.museumnext.com/article/harnessing-the-value-of-museum-assets-with-holographic-nfts/
[5] https://youtu.be/XYRjaZl08lQ
[6] https://www.ditdot.hr/en/interactive-museum-exhibitions-examples-and-technologies
[7] https://www.gardnermuseum.org/organization/theft