Apple recently released the new iPad Pro, the first Apple device to introduce lidar components for VR and AR experiences. To date, lidar has been used widely in industries such as autonomous driving, heavy industry automation, robotics, drones, and security. Apple’s move to include lidar is a clear sign that lidar is moving past its origins in industrial automation and inching closer to mainstream consumer adoption.
The need for spatial awareness
Apple’s team determined that lidar improves AR experiences in ways that they could not achieve with camera technology alone. For watchers of the sensors industry, this decision by Apple is significant because there is an ongoing debate about the viability of camera-only approaches. Many companies have tried to deploy camera-only AR experiences using algorithms to infer 3D spatial information, but these attempts have been dogged by glitchy user experience due to low quality spatial data.
The 3D data output by lidar sensors solves for these shortcomings of camera-only AR experiences, delivering accurate 3D depth information as well as immunity to shadows, glare and low/no light conditions. For Apple, the benefits lidar technology brings to the AR experience outweigh the costs. To make this math work, Apple has implemented a lidar architecture that will yield the highest performance, best reliability, and lowest cost over the long term: digital lidar.
Apple’s choice of Digital Lidar
The iPad Pro is equipped with a Digital Flash Lidar (a type of solid-state lidar) system. As the name suggests, just like a camera’s flash, a flash lidar detects an object by emitting a light wall, instead of scanning the laser beam point by point in a traditional mechanical rotary lidar.
The system uses vertical cavity surface emitting lasers (VCSELs) paired with and single photon avalanche diodes (SPADs) for the light detectors, the same as with FaceID. These two technologies form the foundation of Digital Lidar and are ideally suited for commercialization for a number of reasons:
- VCSELs and SPADs offer a superior performance, form factor, and cost profile. VCSELs are smaller, lighter, more durable, and easier to manufacture compared with other emitter technologies. SPADs can be densely packed on a chip, count individual photons, and have excellent time resolution, resulting in a simpler, smaller, more durable, and natively digital architecture in contrast to legacy analog lidar detectors such as APDs or SiPMs (which could not fit in a consumer device).
- VCSELs and SPADs support a more rugged and robust system because they can both be integrated onto a chip. Printing all the lasers and all the detectors onto chips greatly reduces the number of components in the system and improves durability and ruggedness.
- VCSELs and SPADs have costs that fall faster with scale, are cheaper to produce in high-resolution implementations, and are improving along with Moore’s Law — whereas edge-emitting lasers and legacy analog APD and SiPM detectors are mature and have little room for improvement.
We are happy to see Apple come to the same conclusion that we did when we first designed our high-performance digital lidar sensors. Ouster was the first lidar company to pursue this approach in 2015, with the earliest foundational patents in the area, and the first to release a digital lidar sensor in 2017. We have seen the cost, reliability, and performance of this technology continue to improve since then, and Apple’s decision to use this technology is one more step in the path to mass adoption.
Legacy analog APD and SiPM-based lidar systems (each paired with edge-emitting lasers) have historically been used in R&D but have struggled to break through to commercialization. These outdated systems have hundreds of lasers, detectors, and other components, manually integrated together and in constant need of recalibration. Costing upwards of $35,000 and with high failure rates in outdoor conditions, these complex systems do not have the cost profile or reliability required for commercialization.
Digital lidar systems used by Ouster, and now Apple, by contrast require only two chips. These systems are simple, affordable, and robust, able to withstand the toughest industrial (and now consumer) abuse. We have put our sensor through rigorous automotive-grade shock and vibration testing, and have an ingress protection rating of IP68 and IP69K, the highest protection level among all lidar sensors on the market.
While Apple’s lidar system only needs to see a few meters, the performance requirements in applications like autonomous driving, industrial automation, and 3D mapping are far more demanding — for instance the ability to sense hundreds of meters. To achieve 200+ m performance, Ouster digital lidar sensors use the same underlying digital lidar technology, but with a more advanced patented optical design.
Performance will continue to improve with chip upgrades
Apple’s commitment to VCSELs and SPADs via the iPad Pro will encourage more investment from supply chain partners and drive rapid improvements over the coming years. As VCSELs and SPADs improve in quality, Ouster’s digital lidar sensors will enjoy higher resolution, longer range, and higher precision without making any changes to the core system architecture.
In fact, since we first designed our system in 2015, we’ve seen the performance of our VCSELs and SPADs improve by ~1000% while there has been little change in the analog technology used in other spinning lidar sensors.
At the beginning of 2020, we launched our second generation sensors, doubling our vertical resolution to 128 lines, and doubling the quality of our precision. At the same time, the maximum detection distance increased to 240 m, while still offering the lowest prices in the market. These improvements will only continue with time—increasing resolution, range, and precision, while driving down cost. In the high-performance lidar market, our digital lidar technology is delivering incredible performance and reliability at a competitive price. With Apple’s adoption of the digital lidar approach, we look forward to seeing the same benefits delivered to consumers across the world.
