Engaging with fans is becoming more and more important. Fans want to know their favourite bands and artists inside out, and a song can do just that – a song represents a story from an artist to a fan. With that in mind it is no surprise that artists are in a constant search for new and engaging ways of telling these stories. Together with Belgium pop group Clouseau we had the chance to discover one of these new ways: combining a volumetric video and a music video.
Traditional music videos
The introduction of the music video in the eighties brought a big change in the way storytelling could be applied to a song. The music video gave artists the opportunity to visualize the story of the song and the role of the artist in it. It is fair to say that it intensified the intimacy between songs, artists and fans. Where first television was the only medium to watch music videos, nowadays, they are mostly watched online on platforms such as YouTube and Vimeo.
Source: 4DR Studios
New channels of visual engagement
It is widely expected that virtual and augmented reality will become the new channels for visual engagement. Uptil now there are a few good examples of this, think for example of Madonna at the Billboard’s Music Awards. But still most artists aren’t familiar with these new technologies, and so the use of it in performances or music videos is low. But times are changing and together with Warner Music Benelux and Clouseau we set an example in using augmented reality and volumetric video for music videos.
Living room concert
Clouseau used augmented reality and volumetric capturing for the introduction of their new album ‘Tweesprong’. Through the Tweesprong application fans could get their own living room concert. The use of volumetric capturing allows them to see the performance from any point of view, this gives the feeling that Clouseau is ‘really there’. The performance is enriched with drawings that are illustrating the story behind the song.
AR Software
For a proper and stable visualization of the volumetric content in augmented reality we used ARKit for mobile devices that run on iOS and ARCore for mobile devices that run on Android. Most mobile devices that run on iOS are suited for ARKit, so the app covers a big part of all mobile devices that run on iOS. Unfortunately for Android this is less, but the list of mobile devices that are suited for ARCore is updated quite often and getting larger every month.
Source: 4DR Studios
Challenges
Our HOLOYSYS equipment for volumetric video capturing only records video, meaning we have to record audio separately. This requires us to do an old fashioned clapsync so we can sync the recorded audio after production – meaning we can never watch the recorded video with the audio together during the process.
However, the HOLOSYS equipment is updated quite often with new features, one of which in the next update is integrated audio capture – a huge step forward.
Besides recording audio the size of the content was also a challenge in this project. It will be no surprise that volumetric video is quite heavy on mobile devices: a music video of 3 minutes could easily use up multiple gigabytes if you are not careful, and adding too many assets can cause the volumetric video to buffer causing an asynchronization of audio and video.
Optimizing all content in the post-production was essential for offering a workable app for the fans. This video is a good example of how we solved this.
Feel free to send us a message if you still have questions about this project and its process. We always like to give more insight to those who are interested.
To read more from 4DR Studios, head down to our Insights section.
