Listen to this article

(Update at 8:54 PM Eastern on August 19, 2021:) An Intel RealSense employee, who wished to remain anonymous, just emailed the following. “The official message is still that Intel has decided to wind down the RealSense business. I honestly don’t know what that means. [End-of-life] announcements for LiDAR and Face Authentication products by the end of the month. There will be a 6 month EOL and last time buy. Select stereo products will continue to be provided. D410, 415, 430, 450, modules and D415, 435, 435i integrated product lines will remain. D455 will be discontinued.”

This person added, “I don’t know how you wind down the RealSense business while still selling RealSense stereo products. This is very frustrating for us.”

(Update at 9:20 PM Eastern on August 19, 2021:) Anders Grunnet-Jepsen, CTO of RealSense Group, just posted this message to LinkedIn:

Intel “winding down” the RealSense line could be a bombshell for roboticists. For the past six years, RealSense has been a go-to for industry and academia for low-cost, high-quality depth sensing. It found its way into the production design for many now-deployed robot platforms, like the recently announced Corvus warehouse inspection drone and Avidbots’ Neo 2 automated floor cleaner.

It was featured in many Intel keynotes as a cornerstone product for the company’s future. And it was a very active project, with new sensors being released regularly, and updates to the RealSense SDK as recently as last week.

Surveying the depths of history

What the future holds for the RealSense products isn’t exactly clear yet. But if the cameras do go away altogether, that shouldn’t come as a surprise. After all, this has happened again and again in the world of 3D sensors.

Microsoft Kinect
One need not look too far back to recall the depth sensor that started it all – the original Microsoft Kinect. Upon its launch in 2010, roboticists seized upon it as a source for low-cost, high-quality depth sensing (sound familiar?) that could unlock sensing capabilities that were previously only available to companies and institutions that could afford multi-thousand dollar industrial 3D sensors. Co-developed by Israeli startup PrimeSense, the Kinect quickly found its way into academic and commercial robotic environments, being employed for tasks like obstacle avoidance, object detection, and skeletal tracking.

With the backing of Microsoft, the Kinect seemed like a sure thing, a safe choice as part of a vision-enabled system’s final design. And then, suddenly, Apple snapped up PrimeSense for $360 million in November 2013. The original Kinect became history. Roboticists who had relied on the Kinect – and, more importantly, Microsoft – were suddenly left in the lurch.

Project Tango
Just about a half year after Apple’s acquisition of PrimeSense, another promising depth sensing platform emerged from a huge tech company: Google’s Project Tango. Like the Kinect, Project Tango promised low-cost, high-quality 3D sensing along with a robust developer SDK to rapidly create applications that used depth (and, interestingly, the very first Project Tango developer devices used PrimeSense chips as part of their design).

Adoption from the developer community was swift, and Google subsidized and promoted some of the more promising applications, including a collaboration with NASA. Soon, Tango-specific handsets were being launched, including one co-developed with Intel that included a built-in RealSense. And then, after just three years, Google announced it was winding down Project Tango. It would be joining the Kinect in the ranks of defunct depth sensing platforms.

What, then, happened in the wake of the cancellations of the Kinect and Tango? What lessons can we learn from those examples that may apply to the RealSense platform? There are a few.

Future for RealSense and its users

When Kinect and Tango were cancelled, there were a few distinct reactions by their user bases: hoarding, switching, and innovating. Here at Tangram Vision, we believe these will paths could be followed by RealSense users as well.

Hoarding: Integrating a depth sensor into a complex, vision-enabled platform is no small feat. For many companies that had to do so before the advent of plug-and-play sensor management platforms like the Tangram Vision SDK, this process could take as long as six to 18 months.

Switching: Many of the more forward-thinking companies that had come to rely on Kinect and Tango decided to bite the bullet and simply switch to a new sensor that had guaranteed supply. For many, the simplest option was to switch to the original Structure Sensor by Occipital, which included similar components to the Kinect and was the closest sensor possible to a direct replacement. But many others chose to redesign their depth sensing systems entirely and integrated a completely different sensor — ironically, in many cases, that different sensor was an Intel RealSense.

Companies that currently rely on RealSense could choose to redesign their products to incorporate a different 3D sensor. Many engineering teams, no doubt, would have a sense of dread, however, that whatever sensor they choose to replace RealSense could ultimately suffer the same fate.

Further down in this article, we’ll explore a few of the alternatives, along with our best guess as to the supply stability of the platform and the financial stability of the company behind the sensor.

Innovating: The proliferation of depth sensing integrated into mobile handsets, advanced driver assistance systems (ADAS) and other rapidly growing markets has allowed optics component manufacturers like Sony, ST Micro and OmniVision to offer a wider range of off-the-shelf optics for those companies that want to “roll their own” depth systems. Indeed, the components required to build a passive stereo depth sensing system are now widely available and inexpensive, yet require a talented engineering team (including hardware, software, and firmware engineers) to pull it off. This path is very real – at Tangram Vision, we’ve been surprised at the number of robotics teams we’ve spoken with who have chosen to develop their own bespoke depth systems.

Alternatives to RealSense

RealSense was an excellent choice for near- to mid-range depth sensing for tasks like obstacle avoidance, object detection, and depth information for SLAM systems used for robotic navigation. It was also widely used in the current crop of machine-learning driven pick-and-place robotic systems used in supply chain contexts. Fortunately, there are many alternatives. For the sake of brevity, we’ll note what we believe to be the best direct replacement options.

Structure Core by Occipital
The Structure Core features an optics and sensing package that is very much like that which was found in the Intel RealSense D435i: a pattern projector, a frequency-matched infrared camera, an RGB camera, and an onboard IMU. From a range and field of view perspective, this sensor is a near match for what the D435i offers. Now entering its 13th year, Occipital has raised nearly $50 million from investors, and appears to retain stability to continue offering its products well into the future.

(Editor’s Note: the author was formerly a member of the Occipital team).

Orbbec Astra
Orbbec’s Astra range is extensive, much like the RealSense range was. The Astra+ module nearly matches the Intel D435’s 10 meter range, as well as its relatively wide field of view. According to Crunchbase, Orbbec has raised $200M, a formidable war chest. It has also recently partnered with Microsoft to further expand the possibilities of depth sensing.

Luxonis OAK-D
Marketed in partnership with OpenCV, the OAK-D is a unique depth camera that incorporates Intel’s Myriad X vision processing chip for on-device AI capabilities. This places it in a slightly different class than other depth sensors that don’t offer this feature, but may also make it difficult to use as a direct replacement for RealSense. And, for those who are wary of supporting Intel in the wake of its sudden elimination of RealSense, there are no guarantees that Intel’s refocusing on its core PC chip business won’t spell the demise of Myriad X either.

The OAK-D was launched with a very successful Kickstarter campaign, which saw it benefit from the relationship with OpenCV. We don’t have much insight into Luxonis’ financial stability; however, in the field we are increasingly observing many companies experimenting with its sensors. We think the future is bright for Luxonis, assuming Intel continues to support Myriad X.

For a simple way to compare many currently available depth sensors to the Intel RealSense range of depth sensors, take a look at the depth sensor visualizer and comparison tool we created at Tangram Vision. There’s also a great thread on ROS Discourse looking at RealSense alternatives.

So what went wrong?

RealSense hasn’t materialized into a big business for Intel. It also barely contributes to Intel’s core business of selling chips. And that core business has been under pressure with Apple’s introduction of their bespoke M1 chip and AMD’s increasing competitiveness. Therefore, its future has always in question. Some might even say it’s surprising Intel committed itself to the platform for the years it has been in existence.

Competing platforms from Occipital, Orbbec, Mynt, Stereolabs and others will benefit as companies that were once reliant on RealSense could look for alternatives. Sensor SDK companies like Tangram Vision could also benefit if companies seek a simpler integration path as they face adopting a new sensor into their existing and forthcoming product designs. And it’s likely the market will reward Intel shareholders as it recognizes the benefit of Intel further refocusing on its core product lines.

This story is long from over. We’ll continue to see RealSense-integrated products; engineers will contribute to open-source repositories that support RealSense; and the proliferation of depth sensing due to RealSense will be a lasting legacy that all of us in the robotics industry should be thankful for.

About the Author

Adam Rodnitzky is a co-founder and COO of Tangram Vision, which creates DevOps for perception. Adam was formerly General Manager of the Structure division for Occipital, and has worked in the computer vision industry for the past 15 years.

Prior to launching Tangram Vision, Rodnitzky was a mentor at StartX, a seed stage accelerator focused on commercializing startups connected with Stanford University, and VP of marketing and GM of Occipital.