More than 15 million users a week launch the Fyuse app to take and share the 3D interactive pictures the technology enables. But what these users may not realize is that their pictures are actually a training tool to help train the eyes of a new generation of devices.
Fyusion, the San Francisco-based developer of these 3D photography applications, has spent the past three years building out its app for consumers — as the first step toward becoming the default 3D imaging technology for consumers and businesses.
“Fyusion’s vision is that our fyuse 3D spatial photography Format will become ubiquitous, and the de-facto standard for representing a certain kind of 3D spatial data for consumers at scale,” Radu Rusu, Fyusion’s chief executive, wrote to me in an email. “Beyond the fyuse format itself, live 3D Visual Understanding in the camera has been a core capability that got baked into the format since day one, taking advantage of experience in 3D Point Cloud processing and robotics — whether the camera is a smartphone or head mounted device that we’re likely to all be wearing in a few years’ time, or a personal robot. We would like to enable all computers, from smartphones to robots to take advantage of fyuse and 3D Visual Understanding soon.”
To achieve that goal, the company has raised $22 million from the global venture capital firm NEA, the computer vision-focused investment firm Presence Capital and strategic investors like 2020 (an investment fund partnered with Hon Hai Precision Industry), Gionee and NTT DOCOMO Ventures.
With that new money in hand, the company is pursuing three new lines of business and expanding geographically with new offices in China and Japan, according to Rusu.
Fyusion’s visualization and imaging technologies are now going to be pitched more aggressively to business customers, as well as consumers. In automotive sales, the company is working with vendors like Cox Automotive in the U.S. and Gulliver in Japan. In e-commerce, the company plans to work with retailers like Walmart and others to present 3D images online. Finally, the company has lined up original equipment manufacturers like Huawei, ZTE, Gionee and TCL as partners to create AR-enabled images on their Android-based platforms.
Consumers should also expect to see Fyusion technology in head-mounted displays and other hardware before the end of 2018, according to Rusu.
“Over the last 12 months, we have taken the first iteration of the format together with the customer feedback, and built specialized enterprise solutions for our 3 focus verticals. This marks the launch of the enterprise Fyusion platform,” Rusu wrote.
In a sense, the work Rusu is doing at Fyusion is simply a continuation of earlier research that he conducted while working for the famed robotics lab Willow Garage.
“Teaching robots to see and understand the world in 3D at Willow Garage led us to believe that we could utilize consumer devices to bootstrap a ‘3D JPEG’ visual format,” according to Rusu. “So essentially, we saw the need to build a format for Machine Learning, forward facing and compatible with new hardware devices such as AR Head Mounted Devices, that would offer significant advantages long term to everyone due to its native properties of interactivity, immersiveness and information extraction.”
With the financing and partnerships in place, Rusu said that the next step is to continue to develop the technology and open it to third-party application developers. “The application of our expertise allows us to understand people, objects and scenes live in the camera, model them in 3D, and do so with extreme precision not possible with 2D images, which has long been a challenge for visual understanding,” writes Rusu.
He argues that while the market for visualization and image recognition technologies has been attractive for investors and a vital component for the continued development of new robotics and automation technologies, it’s been stunted by the offerings currently available on the market.
The company has more than 50 patents in computer vision and machine learning, and while the consumer application may be visually arresting, the underlying technology can be a more effective way to teach machines to see and understand what they’re seeing.
“We are a small start-up working in a space dominated by global players,” Rusu acknowledges,” but the core multi-angle and geometry of 3D capture gives us a strong technical advantage in visual understanding over any 2D approach.”
