Ender’s Game is Already a Reality for the U.S. Military

Troops are being trained with military technology inspired by an iconic science fiction novel

The Player of Games: Asa Butterfield (right) plays Ender Wiggin, the titular hero of the 2013 film adaptation of the 1985 novel Ender’s Game, alongside Ben Kingsley (left) and Hailee Steinfeld (center).
“Guy with a gun right there!” shouts Army Staff Sgt. Dustin Hill, as we storm an enemy compound in Afghanistan.

Dressed in camouflage and weighed down with a backpack, I crouch with another soldier, Staff Sgt. Edwin Lopez, near a small concrete wall. I peer around the corner to see a Taliban fighter aiming at us from the doorway of a run-down two-story building. As bullets zip past my helmet, I look up into the bright blue sky to see a column of thick black smoke twisting up from a nearby rooftop. But before I can raise the heavy rifle in my hands, I hear the scream of a bullet, and everything goes black.

As someone helps me take off my helmet, I’m suddenly not in Afghanistan anymore. Motion-tracking sensors are strapped to my arms and legs. My backpack holds a computer processing unit. Virtual reality goggles dangle from my helmet. The gun in my hands is an exact replica of an M16—except for the tiny joystick on the back of the trigger to control my motion.

I’m in a convention center in Orlando, Fla., at the Interservice/Industry Training, Simulation and Education Conference, the largest military simulation conference in the world. And I’ve just experienced the Army’s most lifelike training simulation yet: the Dismounted Soldier Training System which, after two years and US $57 million in development, will arrive at bases this year.

10RWEndersGame f2 Tor Books
Photo: Tor Books
Yes, it’s the stuff of science fiction, most notably Ender’s Game, the best-selling 1985 novel by Orson Scott Card that has whiz-kid cadets fighting an alien space fleet through a video game (the book was based on a short story by Card of the same name, published in 1977 as Analog Science Fiction and Fact). Among military geeks, the book—which is being officially released as a $100 million film starring Harrison Ford and Ben Kingsley on 1 November —has been a blueprint for the future for years. As Michael Macedonia, the former director of the Army’s simulation technology center, told The New York Times in 2003, “Ender’s Game has had a lot of influence on our thinking. The intent is to build a simulation that allows people to play in that world for months or years, participate in different types of roles, and see consequences of their decisions.”

Ten years later, Macedonia’s vision is bearing fruit, driven in part by a strong appetite for the savings that simulations promise in a time of tight budgets. The University of Southern California Institute for Creative Technologies (ICT) is a U.S. government–funded research and development organization in Playa Vista, Calif., that makes some of the most convincing military simulations around. According to ICT, one of its training tools, the Joint Fires and Effects Training System, saved the military $3 million per year on what the fuel would have cost for the comparable training with real vehicles. And according to a recent report in National Defense magazine, the Air Force “estimates it could save about $1.7 billion over five years by reducing flying hours by 5 percent and shifting more of its pilot and crew training to simulators.”

The technology has also been buoyed by consumer trends. Although simulations have been used in the military for decades, advances in mobile devices and computer graphics cards are helping sims replace live training for a wider range of roles than ever before, both on and off the battlefield.

Officers refine counseling and interrogation skills on artificially intelligent virtual humans. Commanders execute complex battle drills as if they were a giant round of World of Warcraft. Soldiers dispose of improvised explosive devices in driving simulators. Immersive, portable, and tailor-made for the Xbox generation, these simulations are being used to do everything from treating post-traumatic stress disorder to familiarizing a soldier with an enemy base, as in the case of the Afghanistan scenario I experienced.

“We don’t give them a manual, we don’t send them home for three weeks to study,” says James Blake, head of the U.S. Army Program Executive Office for Simulation, Training and Instrumentation. “We just put them in the environment, put the device on them, and exercise.”

Once Upon a Time

The use of video games for military training began in 1980. Battlezone, a popular arcade hit that pitted players against three-dimensional (albeit wireframe) tanks, was modified for the Army to school Bradley Fighting Vehicle gunners. But when he was writing the Ender’s Game short story and novel, Card didn’t find much in his neighborhood arcade to inspire a vision for the virtual warriors of the future. “None of the games I had seen were remotely useful in preparing soldiers for either combat or leadership,” he says.

So he imagined the Battle School, and the simulated training environment that dominates the latter part of Ender’s Game (earlier in the narrative, training is conducted mostly in a series of rooms in which the cadets physically engage in zero-gravity combat). The book’s young hero, Ender Wiggin, proves himself an expert war gamer, masterfully maneuvering first his own digital fleet and then a fleet of fleets commanded by other students against an enemy race of alien “buggers.” The training program is ruthless and unrelenting, driving Ender to invent tactic after tactic. The Battle School became a paradigm of just how convincing simulations can be.

Today, the U.S. Army is leading the fight to bring such simulations to life, nowhere more than at the ICT. Spread out over an office park and launched by the Army in 1999 under the auspices of USC, the ICT leverages talent from the movie and video-game industries in nearby Hollywood [for insight into ICT’s early years, see “Games Soldiers Play,” IEEE Spectrum, March 2002]. The Institute now employs over 180 game developers, filmmakers, artists, and engineers, and it gets half of its $35 million annual budget from the Army. The other half comes from customers such as the Department of Defense and the Navy. Pocket change compared to the budget of many defense programs (the development of just the engine of the F-35 Joint Strike Fighter plane has been budgeted at $4 billion), ICT’s budget is evidence that simulation technology can provide a lot of bang for the buck. Nonetheless, the ICT is facing flat or decreasing funding levels due to defense cuts.

10RWEndersGame f4 USCICT Lightstage
Photo: USC Institute for Creative Technologies
Cybercamera: the Light Stage allows realistic 3-D virtual humans to be created from human models.
10RWEndersGame f3 ICT ELITE
Photo: USC Institute for Creative Technologies
Office Space: What happens off the battlefield can be critical to troop effectiveness. This simulation lets officers practice their interpersonal skills.
Visiting the ICT feels like going to an Ender’s Game version of Willy Wonka’s factory. Upstairs in a darkened room, a young man sits inside something called a Light Stage. A camera on a rotating arm around an actor captures a perfectly rendered, 3-D digital version of him, which will later be animated and incorporated into training simulations. (The technology has also been used in films such as Avatar and Spider-Man 2.) Paul Debevec, who did visual effects for The Matrix before becoming the ICT’s associate director of graphics research, calls the results “virtual humans.”

Downstairs in another studio, I get to interact with one of the ICT’s virtual soldiers in the Emergent Leader Immersive Training Environment (ELITE) system. The system has been deployed at facilities such as the Maneuver Center of Excellence at Fort Benning, Ga., and the Officer Training Command Newport.

Photo: USC Institute for Creative Technologies Office Space: What happens off the battlefield can be critical to troop effectiveness. This simulation lets officers practice their interpersonal skills.

The purpose of this war game is to train officers in counseling skills. Sitting before a large computer monitor, I see a life-size, lifelike camouflaged staff sergeant on screen fidgeting in a chair in a small cluttered office. I’m told he has been having conflict with his platoon sergeant. “It’s all good,” the virtual man tells me, when I ask how things are going on base. “It’s not all good,” I reply, choosing from three lines of possible dialogue on a small screen beside me. “Fix the attitude with your platoon sergeant.”

After completing my counseling session, I receive my evaluation from the computer program. A softer approach, I learn, would have gleaned me more information about the sergeant’s conflict.

The ICT’s creation and deployment of virtual humans serves several purposes. First, it’s cheaper: Hiring real-life role players is both expensive and limiting (only so many soldiers, after all, can interact with a real person at a time). Second, it’s part of what the Army has categorized as “the human dimension.” Randall Hill Jr., executive director of the ICT, says that in the past the military’s view on technology would “often focus on how you buy equipment and treat the soldier like a Christmas tree: hang armor on him and gadgets and not look at them holistically.” Now the Army uses technology to empower soldiers more as humans, not mere weapons carriers. “We’re trying to look at soldiers that way,” says Hill. “Our research into virtual humans plays well into that element of training.”

Julia Kim, an ICT project director who is a developing a counterterrorist intelligence training game for the Naval Post Graduate School, distances the ICT’s work from the violence of the simulations in Ender’s Game. “From a moral perspective I have profound issues with Ender’s Game,” she says, noting that her goal is not to turn soldiers into killing machines but rather equip them with survival skills. “Ender’s Game presents an ethical framework that is problematic,” she adds.

For his part, Card lauds the efforts of institutions such as the ICT to better equip the next generation of Americans entering warfare. He also cautions against using the book as a blueprint for how to desensitize the Enders of tomorrow to combat. “If, to any degree, Ender’s Game has helped the U.S. military to train its soldiers more effectively and to reduce both the dangers and economic costs of war, then I rejoice in that good effect,” he says. “However, my purpose in writing was to tell a story, and the primary effect of storytelling is behavioral: The story, once it dwells in the memory of the audience, helps shape the community’s view of which motives and actions are important, and, of the important ones, which are admirable and which despicable.”

Training Armies

Shortly after completing my simulated combat at the military conference in Orlando, I join my virtual war buddies, Hill and Lopez, in a small back room to discuss the experience. Dressed in real camouflage, their replica M16s resting against the wall, the two seem weary from several days demonstrating the Dismounted Soldier Training System. While organizations like the ICT spend millions creating war games, the application of the systems ultimately falls on drill sergeants such as these two.

Hill, a stoic 29-year-old with a fuzzy brown crew cut, and Lopez, an affable 27-year-old from Rhode Island, are lifelong gamers, just like the young enlistees they train to become soldiers at Fort Benning. Over the past decade, the Army has distributed a free online video game for personal computers, America’s Army, as a recruitment tool. But when recruits arrive at Fort Benning they have to check their flashy first-person-shooter skills—typically learned in more gung-ho video-game titles—at the door.

“We get kids who come in with unrealistic views of the Army because of Call of Duty,” Hill says, referring to Activision Blizzard’s popular video game franchise. “You’re not going to come in here and run around like you’re in a one-man war.”

Down the hall at the trade show, I previewed Virtual Battle Space 3 or VBS3, an upcoming sequel to the most widely used tactical training video game in the Army: Earlier versions are used on more than 70 bases (a variety of software licenses and tools are available at costs between $500 to $30 000). Though the game bears a resemblance to Call of Duty, with its brigade of pixelated soldiers infiltrating a dusty Middle Eastern village, the missions have a different purpose. “It’s teaching people how to think,” says Peter Morrison, the young CEO of the game’s Orlando-based developer, Bohemia Interactive Simulations. “You’re not using the game to teach people how to shoot.”

VBS3 is a virtual sandbox that can be used to create training exercises on the fly. The software contains thousands of vehicles, terrains, weapons, and characters. Using a mission-editing program, a commander can stage millions of possible scenarios for a trainee to navigate.

10RWEndersGame f5 Bohemia
Image: Bohemia Interactive
Virtually There: The third version of the Virtual Battle Space simulator will be widely deployed, and allow training scenarios to be rapidly developed.
Morrison demonstrates it for me. With a few clicks of his mouse, he drops some palm trees and buildings in a small town. “Now we’ll choose our IED,” he says, dragging an explosive soft drink can into the dirt. With another click and drag, he places a squad of soldiers on the street, as well as an African man in a T-shirt and jeans. In under a minute, he has created a scenario for a trainee: Find and disarm the IED before the civilian stumbles on it accidentally and dies.

Rehearsing missions in virtual environments saves not just money but time, says Blake. “It’s just like an amusement park,” he says. “We want a rich enough tool set so a commander giving a task can say, ‘I want some of this, some of this, and some of this, and I’d like the training next week.’ ”

The Army has just begun rolling out its most ambitious mission simulation project yet. Costing $27 million to develop and $490 per system to deploy, the Integrated Training Environment is a suite of government off-the-shelf hardware and software tools that allow a commander to combine virtual humans and real brigades together in what Blake calls a “blended environment.” A group of live soldiers on the ground could practice attacking a target on the grounds of a military base while they’re under mortar fire from a virtual enemy infantry division. The computer-generated soldiers behave realistically according to the behavioral parameters in the software. Though the live soldiers can’t see the virtual troops, they are visible to commanders and the actions in the exercises of the real and virtual troops can affect each other: During live training, soldiers use a system called I-Miles, which tracks their location, uses laser tagging to simulate gunshots and note hits, and can also notify soldiers that they have, for example, been “killed” by a mortar round. Crews operating helicopter and tank simulators in training facilities can also be added to the mix.

So instead of having a single brigade of, say, 20 soldiers in a field, a commander can have three giant brigades with tanks, helicopters, and air strikes—quickly and cheaply. Because the overhead action—from live troop positions, crews in vehicle simulators, and virtual units—is all displayed on the same computer screen, commanders get a bird’s-eye view that blurs the line between what’s real and what’s not.

But no matter how sophisticated a simulation becomes, or how convincing a virtual human, it has one fundamental limitation: “You can’t get the emotions from it,” as Dustin Hill tells me. “You don’t get the emotions from your battle buddy being shot.”

Beyond Ender’s Game

To get a glimpse of where our war games are going in the future, I head to the Mixed Reality Lab at the ICT. The lab is a giant, black-draped studio filled with the latest, greatest virtual reality gear. A rack of VR suits—used to track one’s body in a simulated environment—hangs on one wall. On another side, I slip on a giant red helmet and find myself inside a bright, vivid Afghanistan with a Humvee so realistic I step out of the way to avoid smacking into the fender.

When I slip off the gear, my eyes toggle back to reality, and a spry man in thick-framed black glasses is standing before me: the Willy Wonka of virtual reality himself, Mark Bolas. A pioneering engineer in VR since the 1980s, Bolas has a copy of Ender’s Game on his desk, but he says the book now feels dated. “I almost think it’s passé at this point,” he says. “Ender’s Game is happening. It’s already done.”

Bolas leads me past the rack of VR body suits, which can immerse a person’s entire body into a VR world. The suits have sensors that allow a virtual avatar to match the posture of a wearer. (Bolas is also experimenting with a Kinect sensor, the motion-capturing device developed for the Xbox 360, to detect the basic geometry of the wearer.) Tracking gestures in this way “provides the ability to see one’s own body in the environment,” Bolas says, “This is very important. If a user reaches out his or her hand or looks down at their feet, it is disconcerting not to see anything.”

He leads me to another area and hands me a small black cardboard box, into which I peer. Pressing my eyes against the viewfinder, I see a deep red desert landscape. But this isn’t a virtual Afghanistan; it’s Mars, created from pictures gathered from NASA probes. After hearing me marvel at the fidelity, Bolas effectively pulls the rabbit from the hat, reaching into the black box to reveal an iPhone with a Retina display. The box is a small, foldable viewer that collapses flat enough to be shipped in a manila envelope. Created with the help of students and researchers at the USC School of Cinematic Arts, the device is called FOV2GO and exemplifies the cheap and increasingly mobile new wave of off-the-shelf virtual reality hardware.

Bolas predicts that within a few years, a young soldier will be sitting in, say, California in a VR headset and navigating a drone halfway across the world. “Instead of having to go somewhere in the desert and do training and get deployed,” he says, “you just download the environment you’re going to go into, maybe with data they just took yesterday, and all of a sudden you’re really going through that environment.” Bolas calls this “just-in-time training.”

In fact, it seems Bolas is right about Ender’s Game actually being old news. Even as the movie hits screens, it will have to contend with a reality that’s already leaving the fantasy behind. Still, author Card cautions against putting too much stock in war games, no matter how impressive they become.

“Engineers and programmers change the playing field by constantly advancing the means of offense and defense,” he tells me, “But Ender Wiggin and his ilk take these tools and figure out ways to use them under combat conditions. In a sense, the tools do not matter; that is, whatever the tools are, the great commanders figure out new ways to use them…. It remains true, in the real world as in Ender’s Game, that there is no military component more important than the quality of command.”

source: David Kushner/http://www.spectrum.ieee.org/