Two armed soldiers stand behind a barrier, guarding a checkpoint in the road, watching for trouble.
A white truck turns toward them.
“See what he wants, guys,” says one of the guards.
“Sir, we have a military-aged male jumping out of the truck. He’s going behind that van.”
“Is he armed?”
“No visuals, sir.”
And so goes another training session in MIRAGE, an Iowa State University lab that mixes elements from virtual and real worlds to create unique training and research opportunities. (You can see the military checkpoint training simulation on a video here.) MIRAGE (Mixed Reality Adaptive Generalizable Environment) has been used by researchers since 2010. It’s part of Iowa State’s Virtual Reality Applications Center (VRAC).
The MIRAGE research team is led by Stephen Gilbert, associate director of VRAC and assistant professor of industrial and manufacturing systems engineering; Eliot Winer, associate director of VRAC and associate professor of mechanical engineering; and James Oliver, director of VRAC and University Professor of mechanical engineering. All three are also faculty in the human computer interaction graduate program.
The MIRAGE lab’s original project was a study for the U.S. Army Research Laboratory called “Veldt.” Gilbert said the project was designed to test whether military training that mixes physical and virtual objects could be more flexible and effective than, say, building a training facility that resembles a few city blocks of Afghanistan.
“We built MIRAGE as an ideal research lab for mixed reality,” Gilbert said. “It balances the real world with the technology we use in virtual reality.”
The lab’s technology includes:
- a 33-foot by 11-foot screen with six projectors capable of producing 3-D images
- 50-inch television screens that show 3-D images throughout the lab
- 24 infrared optical tracking devices mounted in the ceiling
- a surround-sound audio system
- a control center loaded with computing power, graphics card technology and the capability of connecting with other virtual reality labs
- walls, barriers and other props that can be moved around to create rooms, buildings, alleys or even a war zone checkpoint
“In this lab, we’re very good at integrating hardware from multiple systems and multiple vendors and tying it all together in novel ways,” Winer said.
In the checkpoint scenario, for example, armed guards are stationed behind barriers. The projectors show images of a road with parked cars and moving traffic. And then there’s the unidentified military-aged man jumping out of his truck and hiding behind a van.
The guards crouch behind their barrier and aim their rifles in case there’s trouble.
“It’s engaging your whole body,” Gilbert said. “It’s much more visceral.”
All movements by the soldiers-in-training – even virtual bullets from the fake rifles – are tracked to within a millimeter.
“Now we can go back and collect all this data and say, ‘The reason you made that mistake and got ‘killed’ that time was because you were looking in the wrong direction when this happened,” Oliver said in a video about the Veldt project. “We have a wealth of information we can mine to tune those training scenarios specifically to help.”
While it was originally built to study military training, MIRAGE can also help researchers with many different projects.
Jonathan Kelly, an Iowa State assistant professor of psychology and faculty member in human computer interaction, is using the MIRAGE lab to study how large-scale 3-D imaging can be improved so multiple users can experience the same virtual environment. Current technology can produce a good experience for a lead user, but others nearby see stretched, compressed or otherwise distorted images. A building, for example, might look like a parallelogram instead of a rectangle.
Kelly – who studies spatial cognition, or how the brain comprehends distances and navigates from place to place – said his research group will develop and test software solutions to the problem. Researchers have already experimented with software that calculates the average location of every person in a virtual 3-D environment, giving everybody a less distorted view.
Earlier studies have indicated the brain can somehow reduce some of the image distortions. Kelly wants to learn more about the brain’s ability to adjust how images are perceived.
Without the MIRAGE lab, “this research would be impossible, really,” Kelly said. That’s because most other virtual reality labs are much smaller, only allowing viewers to separate by a few feet. In MIRAGE, viewers can be separated by 20 to 30 feet.
Thanks to the lab, “we’re just starting to scratch the surface of these studies,” Kelly said.
Gilbert is working with other researchers who are considering the MIRAGE lab for studies of violence mediation, promotion of physical fitness and the training of emergency responders. The lab has even been home to VRAC’s annual Halloween haunted house organized by students.
Gilbert said part of the lab’s effectiveness is the ability to use props and technology to create all kinds of environments. In many ways, it’s the same kind of stagecraft that goes into creating a play or movie.
“This lab,” he said, “gives you the immersive experience of the real world but the flexibility of a video game.”