CES Watch: Magic Leap Is Finally Leaping Into Reality


Magic Leap mixed reality

Augmented Reality is a hot topic as we head into CES and 2018, and there’s one startup that’s shaking up the emerging AR/VR landscape, and it’s not even available yet—nor is it augmented or virtual reality, but its own hybrid Mixed Reality.

Magic Leap One

Magic Leap, which has been swathed in mystery and operating in stealth mode, has revealed more in a feature in Rolling Stone and finally released Magic Leap One “creator edition” of its AR system, comprised of “oversized cyberpunk-y goggles, a puck-shaped external computer called a Lightpack, and a handheld controller” for what it’s calling its the Magic Leap One developers version.

It’s supposed to accept “multiple input modes including voice, gesture, head pose and eye tracking… and maps persistent objects onto the environment,” at the Verge notes. Magic Leap offers two sizes of its Lightwear goggles, with an option of custom forehead, nose, and temple pads. Prescription lenses will be built into models, which Rolling Stone‘s reviewer describes as so light he barely felt he was wearing them.

Rolling Stone’s writer comments, “The billion-dollar technology of Magic Leap seems so effortless at times that it would be easy to underestimate it. And in some ways, that’s one of the key innovations of the technology. It can feel like it’s not there.”

Magic Leap Lightwear goggles

The Lightpack clips to a belt or shoulder pad and is comparable to a MacBook Pro or an Alienware gaming PC with a dedicated graphics card. A second lower-powered computer takes care of world discovery and includes machine learning functionality, along with four built-in microphones, six external cameras and built-in speakers (like Microsoft HoloLens).

With CES looming in early January, where former Magic Leap VP Michael Leventhal is speaking on a panel about “The Augmented Reality Experience,” Magic Leap is excited to promote that its SDK will be available for developers in early 2018, followed by hardware and the availability of goggles next year—a promise first made in 2015.

Magic Leap Lightwear goggles

To be sure, perfecting this kind of technology is complicated and costly. The company has raised nearly $2 billion through four rounds of funding (three of them raising more than $500 million each). Magic Leap founder and CEO Rony Abovitz maintains that his product is not mobile computing, Oculus-style VR or even AR. Its MR, or Mixed Reality, a type of AR that includes awareness of surroundings in the physical world.

The key assertion from the exclusive Rolling Stone preview is that our eyes need specific parts of the light field to see and the brain fills in the ‘magical’ parts, resulting in the perception that something real is being seen—hence the magic leap that creates the experience that Magic Leap is capturing and commercializing.

The magic elixir is custom sensor/DSP chip that reconciles the lightfield with virtual images prompting the brain to fill in the gaps, as outlined in RS:

“There is a peripheral vision lens on either “corner” of the Magic Leap, two front-mounted cameras also on the corners, and two windows on the bridge above the nose that look like they might be for IR… Motion tracking likely uses at least two depth cameras — blasting an IR laser projector to draw a grid that can be interpreted by an IR camera. This is how Intel’s Realsense does it and (Microsoft’s) Hololens takes a similar approach. The other cameras are likely used for object recognition and lighting condition processing.”

Magic Leap logoWith partnerships include Lucasfilm and the Icelandic art rock band sigur rós, the company describes itself as: “We are visionaries, rocket scientists, artificial intelligence gurus, robotics wizards, visualization jedis, software ninjas, computing hobbits, film freaks, mathematical artists, psychedelic physicists, people people, business athletes, and music lovers. We are creating a whole new user experience that we call Cinematic Reality. Using our Dynamic Digitized Lightfield Signal, imagine being able to generate images indistinguishable from real objects and then being able to place those images seamlessly into the real world.”

Abovitz, a bioengineer who helped design the surgery-assisting robot arms of Mako Surgical Corp then sold it for $1.65 billion to fund the first four years of Magic Leap.

Google and Alibaba have invested hundreds of millions of dollars into Magic Leap, and David Nelson, creative director of the MxR Lab at USC Institute for Creative Technologies told Rolling Stone that the technology “is moving us toward a new medium of human-computing interaction. It’s the death of reality.”

Abovitz also told Rolling Stone, “You’re basically creating the visual world. You’re really co-creating it with this massive visual signal which we call the dynamic analog light field signal. That is sort of our term for the totality of the photon wavefront and particle light field everywhere in the universe. It’s like this gigantic ocean; it’s everywhere. It’s an infinite signal and it contains a massive amount of information.”

“There were two core zen ideas: The no-display-is-the-best-display and what’s-outside-is-actually-inside. And they turned out to be, at least from what we’ve seen so far, completely true. Everything you think is outside of you is completely rendered internally by you, co-created by you plus the analog light field signal. Everyone is inherently creative because everyone is constantly making their own Avatar world. The world you are living in, you are creating constantly; co-creating constantly, which is super exciting.”