Playing the Meta game
R&D cycles and the future of synthesised reality
Many people seem to view Meta’s recent name change and increased focus on “the metaverse” to be a case of bull market hubris or trying to cash in on the excitement around Web3. In reality this move has been building in the background since Meta’s acquisition of Oculus back in 2014.
In a memo mulling over the acquisition of Unity in 2015, a topic for a follow-on blog post, Mark Zuckerberg set out his vision of the future:
“Our vision is that VR/AR will be the next major computing platform after mobile in the next 10 years…. The strategic view is the clearest, we are vulnerable on mobile to Google and Apple because they make major mobile platforms. We would like a stronger position in the next wave of computing. We can achieve this only by building both a major platform as well as key apps”
Facebook’s (as they were then known) goal was therefore clear, to accelerate the advent of the next computing platform and derisk their strategic reliance on Apple and Google via only offering consumer software that was primarily consumed on iOS and Android devices. This foresight proved very apt given Apple’s recent ATT changes and the increased focus in the wider ecosystem of building hardware to tie users into ecosystems rather than just building with bits for the margins they bring.
Meta are clearly therefore keen to be the lead horse in the race for a market defining mixed reality device as they see it as wider than just a video gaming play. Just take a look at Meta’s *highly ambitious* product roadmap for upcoming mixed reality devices, as indicated by The Information earlier this year:
Project Cambria - High price mixed reality headset1 ($800-$1k) - Q4 2022
Project Stinson - Low price virtual reality headset (c.$400) - 2023
Project Cardiff - Low price virtual reality headset (c.$400) - 2024
Project Funston - High price mixed reality headset ($800-$1k) - 2024
AR glasses - originally slated for 2024 now pushed back to 2026
This is much more akin to the annual/biannual product cycle of a smartphone company than the typical 5-10 year lifecycles of a games console.
R&D cycles in the XR space:
The quiet success of the Quest 2 has meant that Meta has had the nod from the market that there is sufficient demand for these headsets in the right format (Quest 1 proved out the increased demand for standalone headsets) and at the right price (Quest 2 retails for $300/$400) - at least for the base case of gaming2.
Meta can therefore double and triple down on its innovation R&D spend in the knowledge that there was an emerging market for these developments in future headsets. Meta’s alternating headset roadmap set out above of high-price MR headsets followed by lower-priced VR headsets has created a flywheel of R&D flowing down the headset price points and returning value along the way, see below.
One of the top analysts in the space, Ming-Chi Kuo, recently reported that Apple are already planning their own second product line of lower priced headsets that would facilitate a similar flywheel of their own.
To give a sense of the size of this R&D spend, Meta is spending ~$10bn a year on its Reality Labs division. I’ll say that again for emphasis. Ten. Billion. Dollars. I’m feeling like Dr Evil just thinking about it.
Key challenges being worked on in XR hardware R&D:
The scope and range of this R&D can best be seen in the recent company communications highlighting their prototypes and some of the various challenges they are looking to solve - summarised below for anyone who shares my hardware nerd status:
Hardware visual problem statements:
Varifocal: the ability to focus on different depths within a virtual environment, creating a more realistic sense of depth.
Distortion: removing sense of being able to sense the existence of the lenses that you are viewing the virtual environment through as they distort light.
Retina resolution: having enough resolution in the display to meet or exceed the capabilities of the human eye
High Dynamic Range (HDR): building in real world lightness and brightness - Quest 2 can generate c.100 nits. For context, a high-end HDR TVs reach around 1,000 nits and Meta’s starburst prototype can generate c.20,000 nits.
Wider feature development:
Weight: The goal of any XR device is to maxmise immersion and realism via hardware improvements whilst minimising weight - current weight is c.500g
Haptics: Simulating the feeling of touch to match the virtual environment
Eye and lip tracking: building in the capabilities to track eye and lip movements without the requirement of more hardware outside of the headset - to incorporate this into experiences.
UI: recent enabling of hand tracking will enable UI innovations in a similar way that multi-touch screens did for phones.
Aesthetics: The current models of VR whilst slick for industry standards are still relatively clunky, made of plastic and lack the instant appeal that the iPhone had at launch.
AI-driven spatial audio problem statements:
Visual Acoustic Matching: transforming audio clips to match what the virtual environment’s acoustics would be if it existed in the real world
Visually-Informed Dereverberation of Audio (VIDA): how to remove the real world influences of any audio clips so that it can then be pushed through the above acoustics process
VisualVoice: matches audio with movement to better understand both the origin of the sound within a space but also to better understand the implications and impact of that audio.
These challenges are key for full immersion and ostensibly to pass the “Visual Turing Test”, however more importantly they are key to truly unlocking the business use cases for VR and MR headsets that will drive mass adoption faster than a purely consumer driven adoption.
Implications for future of XR:
This gives a glimpse into the future of both XR headsets and some of the problem statements which will unlock new use cases as they get solved. For example improved retina resolution is key to unlocking Productivity as a use case - which will help drive mass adoption faster than the slower consumer adoption curve as people will get to test the technology for work purposes and see the benefits for their personal life - similar to smartphones being initially mainly useful for emails and maps but mass adoption and slow tech improvements unlocking use cases such as Uber.
Compare this with smartphones which post-iPhone have been more characterised by gradual improvements of hardware that already existed in the first generation of iPhone but with extremely large software changes happening beyond that.
The key overhaul needed for XR to go fully mainstream in my mind is the UI/UX. We are still in the early phases of development of what XR headset’s UI should look like and these are predominantly for early VR headsets and one with relatively little content to navigate through. Remember the UI for a Nokia 3310 or how old mobile keypads used to work for typing?
With the advent of body-tracking, AI improvements to verbal controls and Apple’s silent release of one of the best room-mapping capabilities on the market - we have all the ingredients for a massive improvement in UI, I think Apple’s experience and strong focus on UI is what has delayed the launch of their headset but also why their MR headset will set the standard of XR UI going forwards. This combination of hardware as well as software/UI uncertainty makes it very hard to know what the form or capabilities of the future or even near-term headsets will look like.
One thing is for sure however - the pace at which the hardware is going to improve will only continue to increase as the flywheels above take effect at Meta and Apple. Having two players in the space will not only increase the pressure to make advances, but allow both learn from each other’s successes and failures - Apple will bring a huge amount of UI and hardware expertise to the space which Meta has been building from scratch.
These two companies will likely set the standard for headsets and allow others to follow suit at a later point at lower price points - much like how Apple did for smartphones with android phones copying the iPhone format at a later stage but at lower price points.
Expected to be launched as a mixed focus B2B and premium fitness device
Meta has ambitions for the headsets beyond just games, with Productivity and Fitness being the two main wider use cases they are pursuing - check out FitXR for an example of the latter!