It is safe to say that most volumetric video companies are working hard to make this technology available to a broader audience. We all want the same thing, to popularize the use of a technology that we believe will change what immersive experiences mean to us. What differentiates us from the rest is which part of the volumetric video ecosystem we are making more readily available to anyone who might want to experiment with it.
On the capture side, we have high-end studios such as Metastage creating a performance library with on-demand holograms available at accessible prices. While Volograms are betting on user-generated content and have just released Volu. Volu is the first app that allows anyone with a mobile device to capture people in AR.
But while we see exciting developments in volumetric capture – the first step in the volumetric video content creation pipeline – we at Sense of Space have always focused on what comes next: post-production and distribution.
Since 2019, we’ve been building a volumetric video editing software that is made for creatives instead of technologists. With Sense XR, anyone can edit volcaps just as they would edit a 2D video. SenseXR is the most intuitive editing software out there for those who want to create experiences that mix volumetric videos, 3D elements, and audio without coding or needing complex 3D manual work.
But what we are bringing to the table now is our biggest attempt to popularize this technology so far.
Volumetric Video WebAR With a Push of a Button
After months of research and development, we are bringing a new feature to Sense XR that will allow all users to publish their volumetric video content to WebAR with the push of a single button. As a result, users will get a web link and a QR code that can be distributed to anyone, making their experience available to more than 3 billion devices around the world.
To make this a reality, we had to work hard on various important aspects of this technology. Many companies have been offering WebAR for non-volumetric 3D assets through the last few years. But only two of us were able to work through the complexities of experiencing volumetric video through a web browser. Here is why.
Excellent Compression and Optimization
To enable creators using Sense XR to publish great WebAR volumetric experiences, we needed to offer excellent compression and optimization of their data. As you may know, volumetric assets are a series of 3D models that change frame by frame, and the more detail you have in your asset, the heavier the experience will be. So it’s not uncommon that we at Sense of Space need to handle very heavy data in a very efficient way, to avoid web-based experiences running slowly.
To compress volumetric video so that it can be played fluently on mobile devices, we use a combination of geometry compression and texture compression, as well as reducing invisible details of the geometry. This way, we can ensure good visual quality without jeopardizing the performance and responsiveness of the experience.
A good compression and optimization system allows experiences to be viewed in most devices fluidly. It’s also a massive advantage for the creator’s pockets as smaller files mean cheaper streaming costs.
Image source: Sense of Space
The Only WebAR Player With Native Adaptive Streaming
While researching and developing our compression pipeline, we understood that an important part of a volumetric video WebAR service was adaptive streaming. Through this feature, we guarantee that experiences run smoothly across different types of network connections — either 4G, WiFi, or 5G. This allows end-users to see holograms in the highest possible quality and in the most efficient way for their specific scenario.
Our adaptive streaming solution constantly monitors the available bandwidth of the user and adjusts the volumetric video quality. So the user gets the best possible quality that can be played without interruptions.
Today, Sense XR is the only WebAR player with native adaptive streaming for volumetric videos. Due to our excellent compression and optimization system and our adaptive streaming capabilities, we are able to maintain the high frame rates in major browsers on desktops and mobile devices.
Image source: Sense of Space
SLAM Technology That Runs Directly in the Browser
Sense XR offers a unique platform-independent browser-based SLAM (simultaneous localization and mapping) tracking system. Our SLAM tracking system runs in a browser and does not require ARKit or ARCore to work. It calculates in real-time the device orientation and position based on the device gyroscope and accelerometer data combined with a custom-made computer vision system running in the browser.
This means that Sense XR users don’t need to choose one service for the WebAR tracking and infrastructure solution, and another service for compressing and streaming volumetric videos and complex 3D data. This would usually result in paying much higher costs. With our system, users get the best of both worlds, with all necessary infrastructure in one place. No need to contact several companies, no need to pay various fees!
We focused our efforts on offering volumetric video WebAR with a single click because our partners and clients really needed it.
On the content creation side, some don’t have the necessary technical background to code their way into hologram WebAR, and all of them suffer from budget and time restrictions. On the final user side, those who are willing to download the app to see a promotional experience are very few and far between. So these are the challenges Sense of Space is working to solve.
If you believe you would benefit from intuitive and accessible volumetric video WebAR in your projects, try Sense XR for free and let us know if we are on the right track.
Can’t get enough of Sense of Space? Follow the link to read another insightful piece by Sense of Space’s Raquel Cardoso (Chief Communications Officer).
