Apple AR/VR: Reality Bites Virtual Reality

by Jean-Louis Gassée

Augmented or Virtual Reality glasses and goggles dangle tantalizing prospects. But development challenges caused Apple to reset expectations.

Let’s set up the context.

For the better part of the last 18 months, rumors insisted that Apple was at work on what I’ll call AR/VR (Augmented Reality/Virtual Reality) devices. This in a context where we got to watch Microsoft’s adventures with HolosLens devices, Facebook’s with Oculus, Google daringly crude cardboard contraption, or ambitious Magic Leap and its valuation scenic ride. They all gave Apple’s participation in the AR/VR race had an air of inevitability. Commenters couldn’t help but position these putative devices as the company’s next product wave. In their view, after years of seeing iPhone sales struggle in a saturating market, smartglasses would Finally™ provide a much-needed rebound to Apple’s flagging fortunes.
Exemplified by notorious Ming-Chi Kuo’s never-ending stream of Apple predictions, the consensus was for an Apple AR/VR product to be launched in 2020, perhaps in the second half.

Then something out of the ordinary happened.

It seems Apple held an all hands meeting in the Steve Jobs auditorium for the team working on the rumored magical glasses. There, in front of a crowd numbering in the hundreds, some say up to a thousand, VP Mike Rockwell provided a time table: a hybrid AR/VR headset code-named N301 would come out in 2022, and “lighter” N421 glass frames would only see the light in 2023.

In itself, the all hands meeting isn’t unusual. But the content is, especially the distant 2022/2023 schedule statement, one we might better understand by looking at the building blocks needed for Apple to enter the AR/VR field.

But before we dig into the question of required hardware/software components, we need context. Apple devices are horizontal, as opposed to vertical, meaning they cover a broad range of customers and uses, as opposed to tools for a narrowly defined set of uses. In other words, Apple won’t make VR goggles for surgeons, or smart-glasses for robot maintenance technicians.

With this in mind, we can turn to the first required component: the CPU/GPU (central processor and graphics engine) combo able to provide a convincing virtual reality hallucination. Today, the most convincing VR experiences involve a goggle or helmet tethered to a muscular PC. Apple-like goggles or smartglasses need to be untethered, with their own integrated compute engines, just like the Apple Watch does, untethered to one’s phone.
For the device to have broad acceptance, a well-known but hard constraint rises up: the CPU/GPU needs to work with a small on-board battery. In rough terms, more compute/graphics power than today’s iPhone, with a smaller battery, lasting about a day of use. This is much easier said than done and, alone, might explain the distant ship dates.

Next, we have the glasshole problem. We recall the social awkwardness, and the vocabulary, generated by Google’s helpful smartglasses experiment. If, across the breakfast table, you see me wearing a pair of camera-equipped smartglasses, how comfortable are you going to be with the thought I might be recording the video and sound of our conversation. One could imagine camera-less smartglasses, but that would limit their power, or a camera on-off indicator on the frame and an evolution of social mores codifying face-to-face recording acceptance or proscription. Not an easy transition. One that might be eased by prior entrants in the field.

This leads us to use cases, best approached thru the lenses, pardon the pun, of app developers.
For VR goggles, games, fights, driving, flying and other narratives are well known and accepted. For smartglasses and their mixed reality applications (computer generated images projected on the semitransparent glasses worn by the user, focused to appear at a comfortable viewing distance), we’ll need more of the creativity of app developers. One can see a boring meeting supplemented by a superimposed display of alerts or email stream, or reading a book while walking in the street, now without the danger of colliding with a pole or a person. I personally prefer a Louvre visit supplemented by commentary on Madame Vigée-Lebrun’s subjects and technique. For a new set of VR/AR use cases, developers will need applications frameworks not offered by iOS. In turn, these new frameworks will require an OS platform to serve interactions radically different from those offered on smartphones and watches.

This is a forbidding set of challenges, one none of existing participants in the AR/VR race for consumers’ heads have really met. Microsoft has a $3,500 HoloLens 2 aimed at business users; Mark Zuckerberg admits that Facebook’s $2 billion bet on virtual reality ‘is taking a bit longer than we thought’ to pay off’; Google discontinues its DayDream VR platform; and Magic Leap, after raising more than $2.6B, is attempting to raise another round for which it had to put its patent portfolio in hock with JP Morgan Chase after failing to set the world on fire with its $2,295 Creator Edition device. It stands to reason that Apple, while keep its work to itself, has struggled along a similar path towards a really consumer-grade AR/VR product or product line. And we have Apple CEO Tim Cook recently opining on the difficulties ahead [as always, edits and emphasis mine]:

“[…] today I can tell you the technology itself doesn’t exist to do that in a quality way. The display technology required, as well as putting enough stuff around your face — there’s huge challenges with that.

The field of view, the quality of the display itself, it’s not there yet.”

My own guess regarding the rumored October all-hands meeting is this: Apple, as rumored, had had an on-going development program for AR/VR or Mixed Reality devices. Sometimes this year, the company decided there were too many obstacles on the path to a 2020 launch schedule and pressed the pause button, perhaps triggering rumors of a program cancelation. Then, having redrawn a roadmap to a credible consumer-grade device and to quell whatever anxiety arose from the development pause, management held a meeting for the entire set of hardware and software teams. Because it is a little bit unusual for details of such meetings to leak, down to product codes names and schedules, one is left to wonder if the spill wasn’t actually intentional.

— JLG@mondaynote.com