Hyper Vision

THERE IS SOMETHING special happening in a generic office park in an uninspiring suburb near Fort Lauderdale, Florida. Inside, amid the low gray cubicles, clustered desks, and empty swivel chairs, an impossible 8-inch robot drone from an alien planet hovers chest-high in front of a row of potted plants. It is steampunk-cute, minutely detailed. I can walk around it and examine it from any angle. I can squat to look at its ornate underside. Bending closer, I bring my face to within inches of it to inspect its tiny pipes and protruding armatures. I can see polishing swirls where the metallic surface was “milled.” When I raise a hand, it approaches and extends a glowing appendage to touch my fingertip. I reach out and move it around. I step back across the room to view it from afar. All the while it hums and slowly rotates above a desk. It looks as real as the lamps and computer monitors around it. It’s not. I’m seeing all this through a synthetic-reality headset. Intellectually, I know this drone is an elaborate simulation, but as far as my eyes are concerned it’s really there, in that ordinary office. It is a virtual object, but there is no evidence of pixels or digital artifacts in its three-dimensional fullness. If I reposition my head just so, I can get the virtual drone to line up in front of a bright office lamp and perceive that it is faintly transparent, but that hint does not impede the strong sense of it being present. This, of course, is one of the great promises of artificial reality—either you get teleported to magical places or magical things get teleported to you. And in this prototype headset, created by the much speculated about, ultrasecretive company called Magic Leap, this alien drone certainly does seem to be transported to this office in Florida—and its reality is stronger than I thought possible.

I saw other things with these magical goggles. I saw human-sized robots walk through the actual walls of the room. I could shoot them with power blasts from a prop gun I really held in my hands. I watched miniature humans wrestle each other on a real tabletop, almost like a Star Wars holographic chess game. These tiny people were obviously not real, despite their photographic realism, but they were really present—in a way that didn’t seem to reside in my eyes alone; I almost felt their presence.

Subscribe to WIRED

Virtual reality overlaid on the real world in this manner is called mixed reality, or MR. (The goggles are semitransparent, allowing you to see your actual surroundings.) It is more difficult to achieve than the classic fully immersive virtual reality, or VR, where all you see are synthetic images, and in many ways MR is the more powerful of the two technologies.

Magic Leap is not the only company creating mixed-reality technology, but right now the quality of its virtual visions exceeds all others. Because of this lead, money is pouring into this Florida office park. Google was one of the first to invest. Andreessen Horowitz, Kleiner Perkins, and others followed. In the past year, executives from most major media and tech companies have made the pilgrimage to Magic Leap’s office park to experience for themselves its futuristic synthetic reality. At the beginning of this year, the company completed what may be the largest C-round of financing in history: $793.5 million. To date, investors have funneled $1.4 billion into it.

That astounding sum is especially noteworthy because Magic Leap has not released a beta version of its product, not even to developers. Aside from potential investors and advisers, few people have been allowed to see the gear in action, and the combination of funding and mystery has fueled rampant curiosity. But to really understand what’s happening at Magic Leap, you need to also understand the tidal wave surging through the entire tech industry. All the major players—Facebook, Google, Apple, Amazon, Microsoft, Sony, Samsung—have whole groups dedicated to artificial reality, and they’re hiring more engineers daily. Facebook alone has over 400 people working on VR. Then there are some 230 other companies, such as Meta, the Void, Atheer, Lytro, and 8i, working furiously on hardware and content for this new platform. To fully appreciate Magic Leap’s gravitational pull, you really must see this emerging industry—every virtual-reality and mixed-reality headset, every VR camera technique, all the novel VR applications, beta-version VR games, every prototype VR social world.

Like I did—over the past five months.

Then you will understand just how fundamental virtual reality technology will be, and why businesses like Magic Leap have an opportunity to become some of the largest companies ever created.

Even if you’ve never tried virtual reality, you probably possess a vivid expectation of what it will be like. It’s the Matrix, a reality of such convincing verisimilitude that you can’t tell if it’s fake. It will be the Metaverse in Neal Stephenson’s rollicking 1992 novel, Snow Crash, an urban reality so enticing that some people never leave it. It will be the Oasis in the 2011 best-selling story Ready Player One, a vast planet-scale virtual reality that is the center of school and work. VR has been so fully imagined for so long, in fact, that it seems overdue.

The Untold Story of Magic Leap, the World’s Most Secretive Startup

The most intense and complete sense of subconscious presence that I experienced occurred with a system called the Void, which debuted at the 2016 TED conference. The Void isn’t as advanced as Magic Leap technologically, but it integrates the best off-the-shelf parts available with custom gear to create an unforgettable experience. For several hours I watched a line of people enter the Void. Almost every person squealed with delight, screamed, laughed, and staggered away asking for more. I felt the same; I’d be happy to pay for an hour’s visit.

The Void grew out of stage magic, a theme park, and a haunted house. Every year, Ken Bretschneider, one of the three cofounders, stages a gonzo haunted house in Utah that draws 10,000 people in two days. It occurred to him that he could amplify the interactions of his house with VR. Curtis Hickman , the second cofounder, is a professional illusionist, designed tricks for big-name magicians, and is also a visual-effects producer. The third, James Jensen, started out developing special effects for film and unique experiences for theme parks. He came up with the idea of layering VR over a physical playground. The common factor among the three was their realization that VR was a new way to trick the mind into believing something imaginary is real.

The Void takes place in a large room. You wear a 12-pound vest that carries batteries, a processor board, and 22 haptic patches that vibrate and shake you at the right moments. Your headset or goggles and earphones are connected to your vest, so you’re free to roam without a cord. Untethered, you’re released from worrying about tripping over a cable or tangling or straying too far. That relief heightens the effect of being present in the VR. Inside, you navigate an Indiana Jones-like adventure that seems to take place over a large territory. The illusion of unbounded space, or, as Hickman describes it, “a magical space bigger inside than it is outside,” is achieved by a trick called redirected walking.

As an example, whenever you turn 90 degrees in the room your VR will show you the room turning only 80 degrees. You don’t notice the difference, but the VR accumulates those small 10-degree cheats on each turn until it redirects your route away from a wall or even gets you to walk in a circle while making you think you’ve walked a mile in a straight line. Redirected touching does a similar trick. A room could contain one real block but display three virtual blocks on a shelf—blocks A, B, and C. You see your hand grab block B, but the VR system will direct your hand to touch the only real block in the room. You can replace block B and pick up block C, but in reality you’re picking up the same real block.

It’s astounding how those tiny misdirections fool your gut into believing that what you’re seeing is real. Stairs can be made to feel endless if they drop down as you walk upward. In fact, at one point in the Void a decaying floor collapses while you’re walking across it, and you see, hear, and feel—in all your body—a plunge down to the floor below. But in fact the real floor only sinks 6 inches. You can easily imagine a room 60 by 60 feet packed with a minimal set of elemental shapes, ramps, and seats, all recycled and redirected for a variety of multihour adventures.

Seeing, it turns out, is not believing. We use all our senses to gauge reality. Most of the high-end VR rigs on sale this year include dynamic binaural—that is, 3-D—audio. This is more than just stereo, which is fixed in space. To be persuasive, the apparent location of a sound needs to shift as you move your head. Deep presence includes the sensations of motion from your inner ear; if the two are out of sync with what you see, you get motion sickness. Good VR also includes touch. Jason Jerald, a professor at the Waterford Institute of Technology who wrote the book on VR (called The VR Book), claims that much of our sense of presence in VR comes from our hands. Gloves are still not consumer-ready, so hardware makers are using simple controllers with a few easily operated buttons. When you wave them, their positions are tracked, so you can manipulate virtual objects. As primitive as these stick-hands are, they double the sense of being present. Touch, vision, and sound form the essential trinity of VR.

Virtual reality hacks the brain in dozens of ways.

 ANDY GILMORE

W

HILE MAGIC LEAP has yet to achieve the immersion of the Void, it is still, by far, the most impressive on the visual front—the best at creating the illusion that virtual objects truly exist. The founder of Magic Leap, Rony Abovitz, is the perfect misfit to invent this superpower. As a kid growing up in South Florida, he was enthralled by science fiction and robots. He gravitated toward robots as a career and got a degree in biomedical engineering from the University of Miami. While still a grad student, he started a company that built robots for surgery. Before the company got off the ground, his only income was $30 a week drawing cartoons for his college newspaper. Most people find Abovitz’s cartoons more weird than funny. They are stream-of-consciousness doodles featuring alien creatures, annotated by tiny inscriptions that include secret messages to girlfriends. They do not appear to come from the mind of an engineer. As it happens, though, good virtuality takes both fantasy and physics.

Abovitz is heavyset, bespectacled, and usually smiling. He is warm and casual, at ease with himself. But he vibrates. He hums with ideas. Overflowing. One idea unleashes two more. He whips his large head around as he speaks, sweeping up more ideas. It’s hard for him to throttle their escape, to slow down how fast they issue from his brain. As in his cartoons, a discussion can leap almost anywhere. Most of his ideas seem to combine physics and biology. In his Twitter bio, Abovitz describes himself as a “friend of people, animals, and robots,” which is pretty accurate. In his conversation and his work he exhibits a rare sensitivity to both the logic of machines and the soul of biology. If you’re making robot arms that help human doctors carve into living flesh, you have to obey the laws of physics, the laws of biology, and the minds of humans. Abovitz has a knack for all three realms, and his surgery robots sold well. In 2008 his company, Mako, went public. It was sold in 2013 for $1.65 billion.

That success sparked a new idea. Could you make a virtual knee good enough to help repair a real knee? Could you augment a knee operation with an overlay of a virtual knee? Abovitz began thinking about the technology that could match virtual worlds with complex real-life surgery.

At the same time he began to create a graphic novel.

Abovitz has a deep love of science fiction, and he invented a whole world on another planet—flying whales, men in dragonfly gear, a young girl with a pet monkey-bat, and an invading army of robots. Flush with cash from his robotics company, he hired Weta Workshop, the New Zealand special-effects house co-owned by movie director Peter Jackson, to create a detailed realization of that world. The Weta team created all the props and practical effects for The Lord of the Rings, and they helped invent the culture of the Na’vi in Avatar. For Abovitz they designed his world, called Hour Blue, and filled in the details of flying whales and monkey-bats. It quickly mutated from graphic novel into virtual-reality precursor. Because what alien world would not be better experienced in immersive 3-D? Abovitz was already pioneering MR for doctors; this would be an extension of his ideas.

The company Abovitz set up to develop this immersive world was Magic Leap. Its logo would be his totem animal, the leaping whale. The hardware to create the MR would have to be invented. By this time, 2012, the Oculus Kickstarter campaign had launched, and other prototypes with similar phone-based technology were in the works. Here Abovitz deviated off the main path. Because of his work in biomedicine, he realized that VR is the most advanced technology in the world where humans are still an integral part of the hardware. To function properly, VR and MR must use biological circuits as well as silicon chips. The sense of presence you feel in these headsets is created not by the screen but by your neurology. Tricks like redirected walking operate in our brain as much as in the Nvidia processor. Abovitz saw artificial reality as a symbiont technology, part machine, part flesh. “I realized that if you give the mind and body what they want, they’ll give you back much more,” he says.

Artificial reality exploits peculiarities in our senses. It effectively hacks the human brain in dozens of ways to create what can be called a chain of persuasion. In a movie, our brains perceive real motion in a sequence of absolutely still images. In the same way, you can scan a blue whale from many angles and then render it as a 3-D volumetric image that can be displayed on a headset screen and viewed from any position. Even if we know the object isn’t real—say it’s Godzilla instead of a whale—we feel subconsciously that its presence is real.

But if even one small thing is misaligned, that discrepancy can break the gut-level illusion of presence. Something as simple as having to

A GAGGLE OF GOGGLES

As virtual (and mixed) technology improves—and as companies start smelling profits—everyone from phone manufacturers to tech giants is getting into the game. Here’s the hardware that VR’s and MR’s biggest players are cooking up. —Chelsea Leu

availability.pngAVAILABILITY type-02.pngVR type-03.pngMR

ff_magic-leap-goggles-facebook.png

Facebookavailability.pngNOWtype-02.png

In 2014, Facebook bought Oculus, the company that dreamed up the Rift headset and (literally) kick-started the VR revolution. Read more

ff_magic-leap-goggles-htc.png

HTCavailability.pngNOWtype-02.png

The Taiwanese phone manufacturer teamed up with game maker Valve Software to launch a high-end headset, the HTC Vive. Read more

ff_magic-leap-goggles-sony.png

Sonyavailability.pngOCTOBER 2016type-02.png

Unlike the Rift and the Vive, Sony’s PlayStation VR is designed to work not with a PC but its own game console—which more than 36 million already own. Read more

ff_magic-leap-goggles-google.png

Googleavailability.pngNOWtype-02.png

Google created Cardboard, its cheap assemble-it-yourself viewer, to bring virtual reality to the masses via their smartphones. Read more

ff_magic-leap-goggles-samsung.png

Samsungavailability.pngNOWtype-02.png

The Gear VR straps a Galaxy smartphone (new models only) to your head to deliver games and apps—all powered by Oculus software. Read more

ff_magic-leap-goggles-osvr.png

OSVRavailability.pngDEV KIT AVAILABLEtype-04.png

An open source platform for VR and MR launched in 2015 and backed by a consortium of companies like Intel and gaming outfit Razer. Read more

ff_magic-leap-goggles-fove.png

Foveavailability.pngFALL 2016type-02.png

This startup proposes to use eye tracking to sharpen what you focus on and blur everything else, cutting down on processing power.

ff_magic-leap-goggles-microsoft.png

Microsoftavailability.pngDEV KIT AVAILABLEtype-03.png

The company’s HoloLens is a wireless, wearable, sensor-packed computer that aggregates its data to embed holograms in the user’s environment. Read more

ff_magic-leap-goggles-meta.png

Metaavailability.pngDEV KIT, FALL 2016type-03.png

Initially funded via Kickstarter in 2013, the startup has created a visor that projects virtual interactive displays in the wearer’s field of vision.

Cupertino has acquired VR-related companies and patented an iPhone-compatible headset—but hasn’t announced anything yet. Read more

worry about tripping over a tethering cable can seed our unconsciousness with doubt. It might look like it’s there, but it won’t feel there.

Following his hunch to exploit human biology, Abovitz set off to make an artificial-reality display in a more symbiont way. The phonelike screens used in the majority of head-mounted displays created a nagging problem: They were placed right next to your eyeballs. If the device is generating the illusion of a blue whale 100 feet away, your eyes should be focused 100 feet away. But they’re not; they’re focused on the tiny screen an inch away. Likewise, when you look at a virtual jellyfish floating 6 inches from your face, your eyes are not crossed as they would be in real life but staring straight ahead. No one is conscious of this optical mismatch, but over long use the subconscious misalignment may contribute to frequently reported discomfort and weaken the chain of persuasion. Magic Leap’s solution is an optical system that creates the illusion of depth in such a way that your eyes focus far for far things, and near for near, and will converge or diverge at the correct distances.

In trying out Magic Leap’s prototype, I found that it worked amazingly well close up, within arm’s reach, which was not true of many of the other mixed- and virtual-reality systems I used. I also found that the transition back to the real world while removing the Magic Leap’s optics was effortless, as comfortable as slipping off sunglasses, which I also did not experience in other systems. It felt natural.

Magic Leap’s competition is formidable. Microsoft is now selling development versions of its mixed-reality visor called the HoloLens. The technology is unique (so far) in that the entire contraption—processor, optics, and battery—is contained in the visor; it is truly untethered. Meta, another startup, has released an MR device that began, like Oculus, with a Kickstarter campaign. The headset is tethered to a computer, and dev kits should hit the market this fall—likely well before Magic Leap.

Abovitz realized that VR is the most advanced technology in the world where humans are still an integral part of the hardware.

All three major MR headsets rely on images that are projected edgeways onto a semitransparent material—usually glass with a coating of nanoscale ridges. The user sees the outside world through the glass, while the virtual elements are projected from a light source at the edge of the glass and then reflected into the user’s eyes by the beam-splitting nano-ridges. Magic Leap claims that its device is unique in the way it beams light into the eye, though the company declines to explain it further at this time.

However Magic Leap works, its advantage is that pixels disappear. Most screen-based, head-mounted VR displays exhibit a faint “screen door” effect that comes from a visible grid of pixels. Magic Leap’s virtual images, by contrast, are smooth and incredibly realistic. But in truth, the quality of displays in all alternative-reality gear—VR and MR alike—is improving rapidly. Month by month the resolution of all visors increases, the frame rate jumps, the dynamic range deepens, and the color space widens. Within two decades, when you look into a state-of-the-art virtual-reality display, your eye will be fooled into thinking you’re looking through a real window into a real world. It’ll be as bright and crisp as what you see out your window.

Once this small display perfects realism, it becomes the one display to rule them all. If a near-eye screen offers sufficient resolution, brightness, breadth, and color richness, it can display any number of virtual screens, of any size, inside it. While I was wearing the photonic spectacles of Magic Leap, I watched an HD movie on a virtual movie screen. It looked as bright and crisp as my 55-inch TV at home. With Microsoft’s HoloLens on, I watched a live football game on a virtual screen hovering next to a web browser window, alongside a few other virtual screens. I could fill my office with as many screens as I wanted, as big (or small) as I desired. I could click for a screen overlaid anywhere in the real world.

One of Microsoft’s ambitions for the HoloLens is to replace all the various screens in a typical office with wearable devices. The company’s demos envision workers moving virtual screens around or clicking to be teleported to a 3-D conference room with a dozen coworkers who live in different cities. I found virtual screens and virtual media within a virtual reality surprisingly natural and practical. At Magic Leap, the development team will soon abandon desktop screens altogether in favor of virtual displays. Meron Gribetz, founder of Meta, says that its new Meta 2 mixed-reality glasses will replace monitors in his company of 100 employees within a year. It’s no great leap to imagine such glasses also replacing the small screens we all keep in our pockets. In other words, this is a technology that can simultaneously upend desktop PCs, laptops, and phones. No wonder Apple, Samsung, and everyone else is paying attention. This is what disruption on a vast scale looks like.

Exclusive Footage of What It’s Like to See Through Magic Leap

 

    Meta Center
    Transforming realities one thought at a time.