Philharmonia Orchestra
The Philharmonia Orchestra’s New Interactive Collaboration “Dream”
The Royal Shakespeare Company, Philharmonia Orchestra, Manchester International Festival, and Marshmallow Laser Feast present the world premiere of Dream: a new multimedia project, March 12 – 20, 2021
Esa-Pekka Salonen and the Philharmonia Orchestra reimagine composition for an interactive audience, providing a living score, adapting the pre-recorded soundtrack as the production happens
The Royal Shakespeare Company (RSC), in collaboration with Manchester International Festival (MIF), Marshmallow Laser Feast (MLF) and the Philharmonia Orchestra, invites audiences to participate in Dream, inspired by Shakespeare’s A Midsummer Night’s Dream. A new type of live performance, it is a shared experience between remote audience members and the actors. For digital audiences, the latest technology in gaming and theater are combined alongside an interactive symphonic score that responds to the movement of the actors. Set in a midsummer forest, the 20-minute, real-time performance will be streamed live to audiences who can watch for free or buy a ticket to participate, influencing the world of the actors. The 12 performances of Dream from March 12 -20 are timed so that audiences across the world can join. The March 13 performance will be followed by special post-show event with RSC Artistic Director Greg Doran, MIF Creative Director Mark Ball, and Robin McNicholas and Nell Whitley of MLF. Tickets are available here.
The Philharmonia, pioneers in combining digital technology with classical music to create immersive audio experiences, has designed an interactive symphonic score for Dream. Recorded by the Orchestra, it will create a living score that adapts and interacts live with the pre-recorded orchestral soundtrack through the movement of the actors—a total reimagining of composition for interactive audiences.
The score is made possible by Swedish composer Jesper Nordin’s ground-breaking music tool Gestrument, which allows an interactive musical layer soundtrack with recordings captured in spatial audio. Using Gestrument, performers generate music from their movement, improvising live to add layers of music to the pre-recorded score.
The installation will feature core classical repertoire and two new contemporary orchestral works: excerpts from Gemini, a recent composition by Esa-Pekka Salonen, the Philharmonia’s Principal Conductor and Artistic Advisor, and Ärr, composed by Jesper Nordin. The music was recorded by the Orchestra and Salonen on March 13, 2020, its last full-scale orchestral recording (100 players) before the global shutdowns.
Dream is performed in a bespoke 7×7 meter motion capture volume at Portsmouth University. The space includes an LED backdrop which displays the unreal world, allowing performers to see their place and act within the virtual environment. Vicon motion capture cameras and state-of-the-art facial rigging capture the movements of the performers, which drives the virtual avatars of each of the characters in real-time through a traditional performance lighting desk into Epic Games’ Unreal Engine. The live performance is mixed with pre-recorded animation sequences.
Puck leads the audience. As fireflies, the audience can guide him through the forest using their cursor. The actors perform and respond to audience interaction and direction, making each performance unique.
A bespoke web-player has been created for Dream to enable the effective distribution of real-time content from individual audience members to the Unreal Engine server and back to the audience. This new software allows the level of dynamic, real-time interaction working with a mass volume of users (up to 2,000) in a live environment.
Esa-Pekka Salonen, Principal Conductor, Composer and Creative Advisor said: “Immersive technologies are going to change the way we compose, perform, and experience music, and I believe we need to reimagine the symphony orchestra for this new landscape. This is primarily an R&D project—to reimagine immersive technology for live performance—where the creative process itself is one of the most valuable aspects of the whole project. Dream has brought us together with some of the world’s leading theater practitioners, VR world-builders and game designers to reimagine storytelling. The collaboration has moved from the stage to the real-time games engine. It has allowed composers to reimagine composition for a new landscape where the audience has agency, and the performers can become part of a living, dynamic score that is integral to the live performance.”
Luke Ritchie, the Philharmonia’s Executive Producer on the project, said: “Under Esa-Pekka’s leadership, the Philharmonia has been using digital technology to create immersive musical experiences for over a decade, and Dream represents the next step in this mission, with performers and audiences both interacting in a virtual space. It’s been a delight to work with two leading composers to reimagine composition for this new creative landscape.
“The Philharmonia’s R&D work with Jesper Nordin and Gestrument has implications for future orchestral experiences whether they be online, in-game, as a location-based experience or a live performance in the concert hall. In Dream, the actors will use Gestrument to perform music; in the future we want audiences to be playing this role as well, removing traditional barriers between performers and participants, and opening up a new world for the orchestra.”
The Philharmonia has expanded its program of immersive musical experiences with four VR films since 2015, allowing audiences to step inside the Orchestra. Its first large-scale walk-through installations, RE-RITE (2009, based on Stravinsky’s The Rite of Spring) and Universe of Sound: The Planets (2012), were succeeded by the audio-led VR experience the VR Sound Stage, which premiered at SXSW and Southbank Centre in 2018.
The project is the culmination of Audience of the Future, a major piece of research exploring how audiences might experience the intersection of different art forms and technologies in live performance. The consortium, led by the RSC, involves 15 specialist organizations in immersive technology, including leading British arts companies, global technology giants, and top British universities. Working collectively to apply their knowledge and expertise in performance, music, video production, gaming, and the research sector, they have created a new type of live performance for audiences to experience.
Dream is one of four Audience of the Future initiatives, supported by the government Industrial Strategy Challenge Fund which is delivered by UK Research and Innovation.