Apple recently released the new iPad Pro, the first Apple device to introduce lidar components for VR and AR experiences. To date, lidar has been used widely in industries such as autonomous driving, heavy industry automation, robotics, drones, and security. Apple’s move to include lidar is a clear sign that lidar is moving past its origins in industrial automation and inching closer to mainstream consumer adoption.
Apple’s team determined that lidar improves AR experiences in ways that they could not achieve with camera technology alone. For watchers of the sensors industry, this decision by Apple is significant because there is an ongoing debate about the viability of camera-only approaches. Many companies have tried to deploy camera-only AR experiences using algorithms to infer 3D spatial information, but these attempts have been dogged by glitchy user experience due to low quality spatial data.
What AR designers would like you to see (photo credit: Snapchat)
What users actually experience with camera-based AR (photo credit: information.com.sg)
The 3D data output by lidar sensors solves for these shortcomings of camera-only AR experiences, delivering accurate 3D depth information as well as immunity to shadows, glare and low/no light conditions. For Apple, the benefits lidar technology brings to the AR experience outweigh the costs. To make this math work, Apple has implemented a lidar architecture that will yield the highest performance, best reliability, and lowest cost over the long term: digital lidar.
The iPad Pro is equipped with a Digital Flash Lidar (a type of solid-state lidar) system. As the name suggests, just like a camera’s flash, a flash lidar detects an object by emitting a light wall, instead of scanning the laser beam point by point in a traditional mechanical rotary lidar.
The system uses vertical cavity surface emitting lasers (VCSELs) paired with and single photon avalanche diodes (SPADs) for the light detectors, the same as with FaceID. These two technologies form the foundation of Digital Lidar and are ideally suited for commercialization for a number of reasons:
We are happy to see Apple come to the same conclusion that we did when we first designed our high-performance digital lidar sensors. Ouster was the first lidar company to pursue this approach in 2015, with the earliest foundational patents in the area, and the first to release a digital lidar sensor in 2017. We have seen the cost, reliability, and performance of this technology continue to improve since then, and Apple’s decision to use this technology is one more step in the path to mass adoption.
Resolution comparison between commercial lidar and consumer lidar, taken by infrared camera (Image source for iPad is https://www.youtube.com/watch?v=xz6CExnGw9w）
Legacy analog APD and SiPM-based lidar systems (each paired with edge-emitting lasers) have historically been used in R&D but have struggled to break through to commercialization. These outdated systems have hundreds of lasers, detectors, and other components, manually integrated together and in constant need of recalibration. Costing upwards of $35,000 and with high failure rates in outdoor conditions, these complex systems do not have the cost profile or reliability required for commercialization.
Digital lidar systems used by Ouster, and now Apple, by contrast require only two chips. These systems are simple, affordable, and robust, able to withstand the toughest industrial (and now consumer) abuse. We have put our sensor through rigorous automotive-grade shock and vibration testing, and have an ingress protection rating of IP68 and IP69K, the highest protection level among all lidar sensors on the market.
Ouster digital lidar sensors going through vibration test
While Apple’s lidar system only needs to see a few meters, the performance requirements in applications like autonomous driving, industrial automation, and 3D mapping are far more demanding — for instance the ability to sense hundreds of meters. To achieve 200+ m performance, Ouster digital lidar sensors use the same underlying digital lidar technology, but with a more advanced patented optical design.
Left, image of indoor map from: https://sketchfab.com/alexijohansen. Right, a high resolution SLAM map made with an OS1-128
Apple’s commitment to VCSELs and SPADs via the iPad Pro will encourage more investment from supply chain partners and drive rapid improvements over the coming years. As VCSELs and SPADs improve in quality, Ouster’s digital lidar sensors will enjoy higher resolution, longer range, and higher precision without making any changes to the core system architecture.
An OS0-128, captured with an infrared camera
In fact, since we first designed our system in 2015, we’ve seen the performance of our VCSELs and SPADs improve by ~1000% while there has been little change in the analog technology used in other spinning lidar sensors.
At the beginning of 2020, we launched our second generation sensors, doubling our vertical resolution to 128 lines, and doubling the quality of our precision. At the same time, the maximum detection distance increased to 240 m, while still offering the lowest prices in the market. These improvements will only continue with time—increasing resolution, range, and precision, while driving down cost. In the high-performance lidar market, our digital lidar technology is delivering incredible performance and reliability at a competitive price. With Apple’s adoption of the digital lidar approach, we look forward to seeing the same benefits delivered to consumers across the world.