As a teenager, Palmer Luckey was obsessed with building an unparalleled virtual-reality headset for gaming. At the age of 19, he succeeded, unveiling the first affordable system that would allow users to explore three-dimensional digital worlds. The Oculus Rift won support from an army of Kickstarter backers, attracted big-money venture capital and turned Luckey into a rising rock star in the tech world. And then Facebook came calling… By Jerry Beillnson
Palmer Luckey likes to go barefoot. However, when he met the men who would become his business partners in Oculus VR, a company that was recently sold to Facebook for $2 billion (about R20 billion), he decided to wear flipflops. It was an important meeting.
This was in 2012, and he was just 19 years old. Luckey was also wearing shorts and a T-shirt, and he was carrying a plastic bin. Inside were a tangle of wires, a small monitor mounted in a frame with a pair of inexpensive glass lenses attached, and a computer. It was a prototype for a head-mounted display, or HMD.
The men he was meeting – Brendan Iribe, Nate Mitchell and Michael Antonov – were older than Luckey. They were gaming industry veterans who had worked together at Scaleform, a company that Iribe and Antonov founded, then sold. Once Luckey got his equipment working, the men took turns peering into it.
What they saw was a bare-bones room with a short set of stairs. It appeared in 3D, and they could explore it by turning their heads to the right or left and by looking up or down. As they turned their heads, the picture moved, too. This was not the first head-mounted display for virtual reality, and it was not the best.
For one thing, the image wasn’t very clear. Users could see the individual pixels on the screen. If they moved their heads quickly, the image blurred. However, sophisticated virtual-reality systems were restricted to university, military and industrial settings. They cost tens of thousands of dollars. Luckey believed he could sell his HMD to the public for under R5 000 and he was planning to run a Kickstarter campaign to fund the launch. He already had a name for the device. He called it the Oculus Rift.
Last October, Palmer Luckey travelled to New York from his home in California as a winner in the annual Popular Mechanics Breakthrough Awards. Luckey was now famous, and his company had received millions in venture capital funding. By the time he arrived, night had fallen. The cocktail party was being held on the 44th floor of the Hearst Tower, overlooking Central Park and the pixelated lights of midtown. Inside the room was champagne and hors d’oeuvres, suits and cocktail dresses, and displays of robots and autonomous UAVs. There was a queue to try out the Oculus Rift demo.
Luckey walked in alone. He was wearing jeans, a wrinkled grey shirt and flip-flops. There are tech celebrities who deliberate over which ripped jeans and hoodie to wear to a formal event. Palmer Luckey is not one of those people.
He is not pretentious. Arrogant, perhaps, if you consider it arrogant for a 19-year-old to build something that upends an industry. Mainly, though, he is enthusiastic. He listens well. He is smart. He likes other people to feel smart, too.
“Virtual reality is pervasive throughout our culture,” Luckey told me. “Since we’ve had video games, people have dreamed of stepping into games. Movies like Tron and The Matrix and The Lawnmower Man are all based on this idea.” But an affordable virtual-reality system wouldn’t just benefit gamers.
Architects are already using VR to step inside buildings that haven’t yet been constructed. Doctors can perform virtual surgery before cutting a patient open.
“We’re focused on gaming to start because it’s very demanding and it’s a large market,” Luckey said. “If we can make it good enough for gamers, then it’s going to be good enough for a lot of other markets.” The first virtual-reality device, the Sensorama, was built in the 1950s. It included a 3D display, fans to create the sensation of wind, stereo speakers, a vibrating chair and even a module for emitting odours. Head-mounted displays and flight simulators appeared over the following 20 years. Motion-capture gloves and body suits were developed in the 1980s by pioneers that included Jaron Lanier, a computer scientist, musician and writer. (It was Lanier who popularised the term virtual reality.)
Today, car designers wear HMDs and motion-capture gloves to test ergonomics and sight lines inside virtual prototypes. And VR systems are common in medical and military settings. “Virtual reality is not even glamorous anymore,” Lanier told me recently. “In some applications, it’s almost commoditised.” But for ordinary people, the dream of virtual reality has been slow to materialise.
Palmer Luckey was home-schooled. This gave him time to break things. As a young teenager, he cannibalised DVD burners for their laser diodes. He used them to build etching equipment. He got into a hobby called portablising, rebuilding old video-game consoles to produce handheld gaming devices.
His family lived in Long Beach, in Southern California. His father sold cars; his mother stayed home with Palmer and his three younger siblings. Palmer had a workshop in the garage and a part-time job at the US Sailing Centre of Long Beach, scrubbing decks, repairing boats and doing grounds keeping. He raced dinghies. He briefly trained for a job rowing Venice-style gondolas for a company that provides canal tours.
By the time he was 16, Luckey was an adept junkyard engineer, sifting through the detritus of electronics that languished in online auctions and liquidation sites. He launched a Web forum, ModRetro, devoted to portablising, where he posted under the name PalmerTech. The members described their projects and traded tips on hardware hacks and where to source parts. In those old posts, Luckey comes across as an open-hearted and enthusiastic site manager. d never met face to face.
Some of the threads are highly technical. Others are charmingly adolescent. A few weeks after the site launched, Luckey was faced with a growing to-do list of functionality problems on the site. He wrote: I try, I try! IRL (in real life) can come down hard sometimes, and my current list of IRL stuff that needs to be done is:
– Extra Credit Math
– Finish cleaning garage/workshop
– Get all my materials for debate camp together
– Finish Gondolier training
Luckey’s other emerging passion was virtual reality. In August 2009, he joined an online forum called MTBS3D. The letters stand for Meant To Be Seen, and the site had been founded by 3D gaming enthusiast Neil Schneider, who also started a non-profi t organisation that is now called the Immersive Technology Alliance. Schneider hoped that if MTBS3D were successful, the gaming industry would be motivated to meet the unfilled demand.
This was not a practical thought. At the time, the worldwide community of VR enthusiasts numbered in the low hundreds. “Among the vocal people, there were maybe 150,” Schneider says. “And I’m being aggressive with that number.” Many of the members were engineers, 3D filmmakers or software developers. Online, they shared their frustrations over the poor quality of LCD displays and the fact that game developers were turning away from 3D.
With Schneider’s permission, Luckey started a section of the site devoted to DIY projects. “He was always very enthusiastic, very excited, like a big ball of energy,” Schneider says. The forum became a hub for people trying to Frankenstein together HMDs from spare parts.
There is no precise definition of virtual reality. “At the bare minimum,” Luckey says, “there’s some threshold you cross into a sense of presence, being in a space and forgetting that it is not a real space but a virtual one.” In 2011, Luckey got a dream job working as a technician at the Mixed Reality Lab (MxR) at the University of Southern California’s Institute for Creative Technologies (ICT). There, he had a chance to cross deep into VR territory. On 25 September, he posted the following on MTBS3D. (These excerpts are condensed and lightly edited.)
“I told you guys I would give a write-up on what it feels like to use the Wide5/body tracking/ Unity based engine setup we have at my work… It sounds crazy, I know, but The Matrix is so much closer than we all think… These are life changing experiences we are talking about, keeping it all to high-end government research labs is a travesty (!) as a gamer, as a DIYer, and as a person who dreams… I want these everywhere.”
He described the experience in a 1 500-word narrative titled “Truly immersive” (AKA “Holy crap, this is real”) VR simulation.
“You stand in the middle of a brightly lit warehouse. Cameras, blinking lights and expensive simulation gear litter the walls and rafters. Your focus is concentrated on the backpack being tightened around your waist and the headpiece that you are adjusting. You hear a mechanical ‘click’ as the backpack is switched on, and the cooling fans carry a slight vibration into your body. Seconds later, your field of view is engulfed in an enormous scene. You seem to be standing on a post-apocalyptic bridge.”
In the distance, Luckey could see a forest and mountains. He felt the wind. He walked most of the length of the bridge, which was rusty and scarred by battle.
“You try to convince yourself that you are looking at a screen, but staring into the blue sky, focusing into the vast distance, it is hard to believe that is the case.”
Luckey heard someone calling his name, and he sprinted back along the bridge to meet an avatar who had appeared, wearing full desert gear. They shook hands. “So, pretty cool, eh?” the soldier said.
“And then as quick as he came, the soldier thanks you for the help, and blips out of existence.” Luckey stayed in the virtual environment for 20 minutes, and then it was time to unplug.
“You take a look around, and marvel at how real the warehouse seems. Wow, the shadow effects on the bright lights are amazing, and the tracking is flawless! In the next moment, you remember: this IS the real world. The bridge that you were on? That was just a simulation.”
I visited the Mixed Reality Lab in February. It was hard to find. There is no signage outside and it is located on an industrial side street in the Playa Vista neighbourhood of Los Angeles. I arrived at about 4:30 on a rainy afternoon. Mark Bolas, the lab’s director, has an office just inside the front door. The facility is a joint effort between the ICT and USC’s School of Cinematic Arts, he explained, developed to train art and engineering students in virtual-reality design. His team also works on a number of military systems.
Bolas showed me the warehouse that Luckey had described in his post. A copy of the Wide5 HMD that Luckey wore was hanging there, clipped to a metal stand. Bolas and a business partner started developing the Wide5 in 2005.
It has a horizontal fi eld of view of about 150 degrees and it is studded with light-emitting diodes (LEDs). In the rafters of the room were 80 cameras, which track a user’s movements by picking up on those LEDs. (The Oculus Rift works in much the same way, using a single camera.)
When Luckey first showed up at the lab, Bolas says, he “had a passion in his eye that is rare to find”. Bolas, too, dreams of developing low-cost virtual-reality systems for consumers. He assigned Luckey to a team already at work on some of these projects. Bolas showed me tabletops littered with several generations of inexpensive, immersive head-mounted displays. They combined foam or 3D-printed head mounts with monitors, smartphones and tablets. The lenses came from cheap, handheld magnifiers.
Throughout his time at MxR, Luckey continued to tinker on his own head-mounted-display project at home. By the American spring of 2012, he was ready to make it public. He wanted to get it into the hands of tinkerers and programmers. Like Schneider several years before, he was hoping the gaming industry would take notice and start designing wide-fi eld-of-view, immersive displays. On 15 April, he wrote on MTBS3D: “Hey guys, I am making great progress on my HMD kit! All of the hardest stuff (optics, display panels, and interface hardware) is done. The goal is to start a Kickstarter project. I won’t make a penny of profit off this project, the goal is to pay for the cost of parts, manufacturing, shipping and credit card/Kickstarter fees with about $10 left over for a celebratory pizza and beer.”
Things turned out differently. John Carmack, one of the most respected game developers in the world, had joined MTBS3D, and Luckey sent him a prototype. In June, Carmack showed off the Oculus Rift at E3, a gaming convention held in Los Angeles each year. The Rift was a sensation. That summer, Luckey combined forces with Brendan Iribe, Nate Mitchell, Michael Antonov and Andrew Scott Reisse to form Oculus VR and launch a Kickstarter campaign featuring a slickly produced video. (Carmack eventually joined the company as chief technology officer.)
By 1 September, the campaign had raised over R24 million. Many contributors donated small sums to promote the technology and get a T-shirt or poster. But more than 7 000 paid at least R3 000, which entitled them to Rift headsets and access to the software developer’s tools needed to write code for the platform. (They also received a copy of the game Doom 3 BFG, optimised for the Rift.) The developer’s kits started shipping the following March.
Oculus began showing a new version of the Rift a few months ago under the name Crystal Cove. The resolution was sharp and the motion blur (technically, “judder”) was largely gone. Anticipation built as the Game Developers Conference – the world’s biggest gathering of video-game creators – approached in mid-March. It was rumoured that Oculus would be releasing a commercial headset before Christmas, and people were speculating about which gaming titles would be Rift-ready by then. Gamers and developers were rooting for Palmer Luckey. He was the garage-hacker kid with a dream and the passion to make VR happen. If the company did make a big announcement at GDC, it would be electrifying.
“You are a f***ing sellout. We had ONE CHANCE AND you f***ed it up.” – a Reddit poster, 26 March.
On 25 March, in a conference call with reporters, Facebook announced that it was buying Oculus VR for $2billion (about R20 billion), including R4 billion in cash and the rest in Facebook stock. The gaming-industry rollout would come first, the company said in a Facebook post under CEO Mark Zuckerberg’s name.
But, the statement said, that was just the beginning. “Imagine enjoying a courtside seat at a game, studying in a classroom of students and teachers all over the world or consulting with a doctor face to face – just by putting on goggles in your home. This is really a new communication platform.”
Within hours, many virtual reality and gaming enthusiasts turned on Palmer Luckey. Fans were simply outraged that Oculus would sell to a corporate giant. And Facebook wasn’t even cool. One poster on Reddit wrote: “I honestly thought Palmer had a chance to be the next Steve Jobs. Now he’ll most likely be relegated to a footnote in VR history.” Another replied, “He’ll be loathed. He is going to go from the man who resurrected VR with his vision, to the hated sellout who killed his own baby.”
Kickstarter supporters, in particular, felt betrayed. Those donors had helped launch the company in 2012, but to build the consumer product Oculus VR had decided it needed much more money. It took R160 million from venture capital firms in June 2013 and another R750 million in December 2013. Now, just four months later, those Silicon Valley VCs were cashing out with huge returns – before Oculus ever released a consumer product.
Joel Johnson, the editorial director of Gawker Media and a long-time tech journalist, had contributed to the Rift Kickstarter project and publicly hailed the technology. He doesn’t accuse Oculus VR of wrongdoing, but, he says, “Ultimately, I think this and (the controversy surrounding) the Veronica Mars movie will be the first steps in a movement away from pure crowdfunding”.
Luckey spent the early morning hours of 26 March explaining the sale on Reddit and other forums. He argued that the acquisition would provide the capital Oculus needed to hire more engineers and release the best possible product. A couple of days later, he texted me: “The big picture will become clear in the long run. I don’t blame the criticisers.” But he found few supporters that night. Around dawn, he went back to the old neighbourhood, the MTBS3D forum where his VR obsession had begun.
Some members were worried or angry, others were hopeful that Facebook would help the technology scale up quickly. The vitriol of the Reddit thread was largely absent. Finally, Luckey logged on to ModRetro, the portablising site he had begun as a 16-year-old. There, a user named Zero was defending him: “This is a good thing, even in the long term. Everyone here especially should know that Palmer wouldn’t publicly support this if it wasn’t.”
Besides, Zero pointed out, anyone worried that Facebook would now be calling the shots was ignoring the obvious: Luckey had already lost the freedom to do whatever he liked with the Rift. PalmerTech thanked his old friends for their support. Then he wrote:
“Zero is right about stakes in the company. We did not sell out control to FB, we did it a long time ago when we had to raise money to keep going.”
A few weeks before the Facebook announcement, I met Palmer Luckey for lunch. We were at the Oculus VR headquarters in Irvine, California. The company is housed in a dark glass-and-marble office building. It has short escalators, empty lobbies and skylights. Luckey and I walked to a parking lot next door and bought hamburgers and fish tacos from a pair of food trucks.
As we ate, I asked whether he would have succeeded in building the Rift if he hadn’t found MTBS3D. “Not a chance,” he said. “One guy can only go so far. Without Internet communities, you’d just have a few people scattered across the country who were still interested in VR but didn’t know where to go with it.”
That’s not a problem any longer. The company shipped about 75 000 of the original developer’s kits and plans to sell even more of the DK2s. Gaming companies are hustling to create Rift-ready versions of many titles. Motion-capture gloves, competitive headsets and other VR peripherals are being announced. And the innovation doesn’t stop at gaming. D J Roller, one of the world’s premier 3D filmmakers, is planning to supply VR capabilities for sporting events, news and movies through Next3D, a company he co-founded. “Up until now, the most immersive medium on the planet has been a five-storey IMAX screen,” he says. “Now it’s a phone-size head-mounted screen.”
Luckey feels certain the Rift will succeed in the videogame market. But he also has expansive ideas of what virtual reality might do some day. “If you look at sci-fi, virtual reality is almost always a plot device that leads to this broken dystopian world. But I think that virtual reality is going to end up being a huge positive for humanity.”
In the near term, he sees students taking virtual field trips to ancient Rome. People will manipulate data files with their hands. Users will be able to see in the infrared spectrum, bend the laws of physics and squeeze into impossibly small spaces. Decades from now, he thinks, many people will choose to spend their days in virtual environments, enjoying luxuries they could never afford in the real world.
Luckey does not claim to have invented these visions. “If you want to know about optics and low-persistence OLEDs, I’m a good person to talk to,” he said. “For the future of VR, there are smarter people.” Jaron Lanier is one of these. He has seen repeated cycles of expectation and disappointment over the past three decades. “Every two or three years there’s another wave of interest in VR,” he said in a phone interview. “What happens typically is that there is insane speculation that reality will be transcended or something like that.” He laughed. “I started that, so I apologise. It’s my fault.”
Yet this time could be different. The global boom in mobile phones has made display technology inexpensive and lightweight. Ordinary PCs can now run processor heavy graphics programs. The software and skills to build complex 3D worlds are well-established. These conditions have been in place for several years. They’ve just been waiting for a catalyst to come along and get the whole thing moving. The Rift, or rather Palmer Luckey himself, may prove to be that catalyst.
Media reports have tended to portray Luckey as a solitary genius. But rarely, if ever, does someone invent anything in isolation. Even as a precocious 16-year-old, Luckey excelled at recruiting people to his passions. He had collaborators on the MTBS3D forums and, especially, at the Mixed Reality Lab. That doesn’t diminish his accomplishment. No one else combined Luckey’s talent for tinkering and his obsession with making an affordable headset for gaming. Someone had to stay up all night, soldering iron in hand.
When he was finishing up one of his prototypes in September 2011, Palmer-Tech wrote on MTBS3D: “Sorry for the long ramble, been up for more than 30 hours without any sleep, hopefully I make some sense. Thanks for the support! I would never have been able to do any of this without the knowledge and inspiration of the members here.”
Luckey had to get back upstairs to the office. Our lunch had run long and he was a half-hour late for a meeting. I’d never asked him the most obvious question: Hey, Palmer, what’s your favourite game?
“Chrono Trigger,” he said. “It’s awesome.” Now there’s a game that will never be played on the Rift. It has blocky graphics and the same synthesised, 16-bit music from when it was introduced for Super Nintendo in 1995. The narrative is rich, though. You and your companions travel through time, overcoming danger, misunderstandings and rejection. (There’s a great essay online titled “Everything I Know About Love I Learned from Chrono Trigger.”)
It’s hard to imagine when the founder of a R20 billion peripherals company finds time to play Chrono Trigger. But I hope he does, because it’s Palmer Luckey’s kind of game. It isn’t about action or richly rendered graphics. It’s about building a virtual universe that’s actually worth living in.
How it works
The Oculus Rift enables users to explore virtual worlds in stereoscopic 3D. Rift inventor Palmer Luckey sketched the system for Popular Mechanics over lunch a frame with a pair of inexpensive glass lenses attached, at Oculus headquarters in Irvine, California.
The head-mounted display (HMD), straps on like ski goggles. The user’s field of view is entirely taken up by an OLED screen (1) that is divided in half, displaying a unique image for each eye. As in other 3D displays, the images appear to be offset from each other, creating the illusion of depth.
To track the movements of the user’s head, the system employs a camera (2) that scans the position of 38 infrared LEDs (3) fixed on the headset. The system’s computer (4) continually adjusts the onscreen image accordingly. A major challenge for Rift engineers has been to cut motion blur and judder, in which an image seems to stutter as users turn their heads. To address the problem, they are using a screen with a high refresh rate and low persistence – each pixel ‑ ashes its image and then instantly turns to black.
From virtual to reality… By Will Dietrich-Egensteiner
The Sensorama augments short films of motorcycle and helicopter rides with a vibrating chair, odour emitters, stereo speakers, and fans to simulate wind.
The first digital head-mounted display shows 3D images that shift with the user’s head movements. A mechanical arm is needed to support the heavy headset.
Neuromancer, a novel by William Gibson, launches the cyberpunk genre and introduces the concept of a cyberspace that humans can plug into.
The first surgery simulators model the lower leg, allowing doctors to practise tendon transfers and predict a patient’s post-surgery walking ability.
Two movies, eXistenZ and The Matrix, highlight VR’s potential to blur the line between reality and the digital world and to be exploited for nefarious ends.
Facebook buys the company that Palmer Luckey co-founded, Oculus VR, for R20 billion. Consumer versions of the headset are expected to go on sale by early 2015.
The US Air Force plans to flight-test a new generation of F-35 helmets that integrate video and infrared images, giving pilots a 360-degree view around the aircraft.