While there’s a lot of talk these days about the use of artificial intelligence in art museums, the use of AI in society and science museums was one of the topics discussed at the Digital Summit organized by MuseumNext. MuseumNext on May 15 and 16.
Giving visitors a radically new way of seeing, experiencing and understanding collections, engaging them in a visitor experience that puts artificial intelligence at the heart of redesigned museum itineraries – that’s what the MIT Museum and the Dutch media museum Sound & Vision have done for their recent reopenings.
The use of artificial intelligence allows us to take a new step forward in the interactive and personalized relationship with collections, which is also the basis of a radical innovation in the field of memorial heritage, as demonstrated by the experiments of the USC Shoah Foundation in California.
Powerful in their ability to engage audiences and popularize historical, remembrance, scientific or social issues, these innovations are not without raising questions about individual freedom.
1. THE VISITOR EXPERIENCE AT A SOCIETY MUSEUM REINVENTED THANKS TO FACIAL RECOGNITION: THE CASE OF THE SOUND & VISION MEDIA MUSEUM IN THE NETHERLANDS
What should a media museum look like today? In the age of social media, when everyone is a producer rather than just a consumer of media, how can we design a relevant museum experience?
This is the challenge that the Dutch media museum Sound & Vision (Neutelings Riedijk architects) in Hilversum – one of the world’s largest media collections – set itself by rethinking its museography and creating, thanks to artificial intelligence, a new interactive, personalized and immersive journey through the contemporary media landscape, from traditional to social media.
On entering the museum, visitors are invited to create a profile via the Mediamuseum application, indicating their place of residence, age and media consumption preferences. They must also create their profile by allowing themselves to be photographed in dedicated installations. This will enable them to enjoy a completely personalized visitor experience, with each of the fifty interactive installations triggered by facial recognition.
The course is divided into several zones: Sharing, Informing, Selling, Telling and Playing. A gigantic installation called “Mediareactor”, made up of LED screens stretching over a surface area of 300 meters, links the different zones of the trail. Not only does it showcase the infinite flow of information that overwhelms us on a daily basis, it also highlights the role of the visitor as “producer”: content from personalized tours mingles with the museum’s archives and data flow, thus incorporating the avatar of the visitor-producer into the very heart of the museum.
While this resolutely innovative approach to a theme such as our relationship with the media is certainly to be welcomed, the concept does raise a few questions.
How can this experience contribute effectively to the stated aim of media literacy, given that it is built around our “profile” and “preferences”? Are we not condemned to evolve in a kind of giant Google, limiting ourselves to our comfort zone and preventing us from going beyond our known horizon? The museum has anticipated this question and assures us that, beyond the content linked to our profile and preferences, the algorithm has been precisely designed to take us beyond our media habits.
But how far does the promise of personal experience, which underpins the use of artificial intelligence, go? If a couple of the same age, sharing the same place of residence, homogeneous in terms of socio-professional category, visit the exhibition, how will the experience of one differ from that of the other? Difficult to judge from a distance, we’d have to go there to find out.
And what about the protection of personal data, from which the tour is fed? The museum team is reassuring on this point, pointing out that all personal data and profile photos are stored in the museum’s own closed circuit for the duration of their use, and are deleted every evening when the museum closes. And for those of us who don’t want to play the profile creation game, QR codes are integrated into the itinerary, enabling us to complete the tour anonymously. While the fact that visitors can choose whether or not to create a profile is to be welcomed, the question remains as to the nature of the experience in this case.
Even if the use of artificial intelligence raises a few questions, its deployment at the very heart of a scenographic journey dedicated to media education and citizenship stands out for its originality and the quest for real interactivity – the museum is in fact presented as “the first museum in the world that constantly adapts to visitors’ actions”.
The question of adaptation is also central to the issue of sustainability: if the museum provides an accurate snapshot of today’s media landscape, what will it look like in 5 years’ time, given the rapid evolution of the social media panorama? The museum team and the studio that designed the trail(Tinker imagineers) naturally anticipated this challenge and planned a system capable of evolving at the same pace as the practices of the visitor-media producer – a master stroke.
Adaptability and interactivity: two attributes of artificial intelligence applied to the museum sector that make it possible to effectively stage ever-changing social issues, or even concepts and ideas, as is the case at the MIT Museum.
2. ARTIFICIAL INTELLIGENCE AS A TOOL FOR MEDIATING SCIENTIFIC CONCEPTS AND AS A COLLECTION OBJECT: THE CASE OF THE NEW MIT MUSEUM
Making scientific thought accessible to all, and representing the most complex ideas in a playful and engaging way: this is the goal set by the MIT Museum(Höweler+Yoon architects) for its reopening in 2022. Over 20,000 m2, MIT has created a new museum trail inviting visitors to explore the Institute’s innovations in the STEAM (Science, Technology, Engineering, Art and Mathematics) field, including more than 20 interactive experiences produced by Bluecadet. As the idea is not to juxtapose scientific results, but to show how research and innovation shape our world, what could be more relevant than to integrate artificial intelligence both as a museum object and as a vehicle for the visitor experience?
Let’s take a look at some of the interactive features on offer: at the start of the tour, on a vast screen, The Window offers a playful, generative visualization of the MIT community, while presenting the radical innovations that characterize it, with the aim of highlighting the fact that MIT is a “unique collection of exceptional people”. Visitors are invited to add their own avatars, which leap happily across the surface of the screen.
Further on, an installation allows visitors to experiment with AI by inviting them to write poems with the help of Open AI’s GPT-3 computer software, which has been trained here specifically for this purpose. Visitors can take it in turns to try their hand at poetic writing with the interactive AI, which suggests its own contributions based on user input. When visitors are satisfied with their poem, they can save it and make it visible on the installation’s curved screens.
Here too, the question of personal data protection arises, but let’s salute the fact that this revolutionary technological innovation, AI, is enabling museums to reinvent the way they engage visitors in a radically new staging.
Nevertheless, among the places present at the Digital Summit where innovation linked to artificial intelligence is perhaps most powerful are museums and places of memory linked to the Shoah.
3. THANKS TO IA, A RENEWED APPROACH TO MEMORIAL HERITAGE AT THE UNIVERSITY OF SOUTH CALIFORNIA SHOAH FOUNDATION AND ITS PARTNER SITES
Faced with the threat of disinterest among the younger generation, the rise of anti-Semitism and the disappearance of the last survivors, a new generation of digital Holocaust remembrance projects is emerging. From the digital version of the Yad Vashem Museum in Minecraft to the virtual reality film Triumph of the Spirit , not to mention the publications on TikTok of the Mauthausen memorial in Austria, initiatives aimed at reviving the memory of the Shoah and reaching out to young audiences are multiplying.
At the Digital Summit, the magnificent “interactive biographies” project was presented. Entitled Dimensions in Testimony and carried out by the University of South California Shoah Foundation and numerous partner museums, the memories of Holocaust survivors are preserved and brought “alive” using cutting-edge audiovisual technologies and artificial intelligence. At the Digital Summit, the project was presented by Shannon Biedermann, curator of the Sydney Jewish Museum, where it is on display in the exhibition Reverberations: A Future for Memory – Sydney Jewish Museum.
Thanks to Dimensions in Testimony, it’s possible to talk in real time with the last survivors of the Shoah and with those who have recently died.
Is it difficult for you to talk about what happened to you?
What’s the worst thing you’ve ever witnessed in a concentration camp?
This is the type of question that was put to the survivors, who were interviewed and filmed full-length, against a green background, surrounded by 23 cameras. In all, no fewer than 1,000 questions were asked, and each answer was recorded in a separate video clip. Using automatic natural language processing (NLP) technology, “Dimensions in Testimony” associates search terms with the most appropriate response and broadcasts the associated video clip, creating a conversational experience.
Founded by Steven Spielberg in 1994, the USC Shoah Foundation, in partnership with the USC Institute for Creative Technologies, has deployed this project in more than 12 museums and memorial sites in the United States, South America and Australia, and online on its educational platform. Similar initiatives have been launched elsewhere, notably in Europe at the University of Munich.
We tested the experiment online at IWitness by asking survivor Pinchas Gutter the tricky question “Can you forgive?”, and his – poignant – answer was given to us in audio and written form, see below.
What more moving memory “experience” could there be? Conceived as a pedagogical device, this moving digital interaction is accessible to everyone, and would undoubtedly benefit from widespread international exposure in schools, universities and beyond.
This is undoubtedly the spirit behind a slightly different version of the interactive biographies that were launched on June 7 at Meta’s Berlin offices in partnership with the World Jewish Congress, as we were able to discover independently of the Digital Summit.
In Berlin, the conversation took place via a virtual reality headset, in English or German, and thanks to XR technology, drawings evoking memories of survivor Inge Auerbacher appeared alongside the “conversational experience”. The application is available free of charge online .
Using artificial intelligence to offer a new cognitive and/or emotional experience of subjects such as the media landscape, scientific research and memory, appears to be a major asset for the staging and mediation of largely intangible content. As long as issues relating to individual freedoms are kept in check, artificial intelligence placed at the heart of a museographic experience embarks the visitor on a process of co-construction, offering vast opportunities for museums and sites linked to the world of ideas, concepts and history.
Annabelle Türkis
