The Future of Flight Training is Virtual

Photo illustration by Fred Flerlage


Flight simulation trainers are about to change, as Naval Air Systems Command looks to virtual and augmented reality technology to build trainers that are deployable, improve fidelity, make better use of training flights and save money.

The Naval Aviation Training Systems program office is developing new prototype trainers leveraging virtual reality (VR) and augmented reality (AR) technology for the F/A-18 fighter jet, TH-57 Sea Ranger training helicopter and T-45 Goshawk jet trainer, said Cmdr. Chris Foster, the program office’s integrated product team lead for Air Warfare Training Development. Foster spoke as part of a Nov. 28 panel discussion on augmented, virtual and mixed reality in Navy training in Orlando during the 2017 Interservice/Industry Training, Simulation and Education Conference.

The F/A-18 project calls for two networked Deployable Mission Rehearsal Trainers (DMRT) that use VR and AR “to look at whether the fidelity of the cues provided are sufficient to support the training tasks that we need to be able to train,” Foster said. Thanks to VR technology, the trainer would deploy with a much smaller footprint than the platform’s current, dome-based flight simulators.

A key question is whether the VR technology can represent a trainee’s hand with enough accuracy for him or her to reliably manipulate a virtual, multi-function display and navigate its series of menus, Foster said.

Meanwhile, the two TH-57 VR part-task trainers (PTTs) will be aimed at making new trainees more comfortable before their first low-level familiarization flights.

“What we know is that low-level flights for students who are beginning their Navy training in TH-57s are quite overwhelming,” Foster said. “It’s very difficult to develop the sight picture that you need and to do it quickly enough when things move much faster on those first few flights, so there’s very little learning that’s actually taking place.”

The hope is that students who take a few VR familiarization flights on a desktop trainer will later perform better during their initial low-level flights in an actual TH-57, Foster said.

Finally, T-45 systems will include one Augmented Reality Visual System (ARVS) training device and two VR PTTs. This technology offers the potential to both improve training fidelity and reduce the high lifecycle costs of the T-45’s current dome-based simulators.

The ARVS will be integrated with an actual T-45 operation flight trainer (OFT), where the trainee will sit while wearing VR goggles.

“When you’re looking through the goggles, you’re going to be able to see outside the cockpit, but we also need there to be a virtual representation of the physical cockpit that the trainee would see, and they should then be able to interact accurately and reliably with the physical cockpit, but relying on its virtual representation.”

‑Chris Foster

The plan is to network the ARVS trainer with two VR PTTs with capabilities similar to the F/A-18 DMRT. If the new trainers’ cue fidelity proves sufficient, the intent is to use them to support training tasks that T-45 trainers currently cannot support, such as multi-ship training and tasks related to carrier qualification and basic fighter maneuvering, Foster said.

“The wonderful thing about VR right now is the entertainment industry is driving this technology further and faster,” said Courtney McNamara, a senior computer scientist and principle investigator and Advanced Gaming Interactive Learning Environment team lead at Naval Air Warfare Center Training Systems Division (NAWCTSD).

“They’re applying a lot of dollars, especially in the game environment, to make these accessible to all users at a very low cost point,” McNamara said. “The resolutions are getting much better and higher, the latency is getting much lower, so we’re able to see less sim sickness when you’re wearing these, and more accurate tracking so that users can move fully in space or be stationary, depending on the training task. Because industry is pushing these limits, the government does not have to invest a lot of its own money into the technology to see it move further and faster.”

In addition to the new flight trainers, VR could also play a key role in new immersive mishap awareness training, an effort managed by the NAWCTSD Battle Lab in collaboration with the program office and the Naval Survival Training Institute.

Spatial disorientation and loss of situational awareness remain a causal factor in many mishaps, but training to avoid and deal with such mishaps has changed little over the years, relying mostly on video or classroom training, Foster said.

“What we need is the capability to quickly and efficiently recreate mishaps based on information that we can pull from mishap reports and other data sources and use those data to update training but also provide a more immersive training environment for students,” he added.

To that end, VR could provide that higher level of immersion, Foster said.

“So there’s not just one way we might approach AR/VR/MR in trying to address known and future training capability gaps,” he said. “There are a variety of ways, and from our perspective, it’s important that we pursue all of those different avenues and make sure that, as we’re going through the process, we’re gathering feedback from the fleet to make sure we’re hitting the nail on the head in terms of the things we need.”

Written by Jeff Newman, a staff writer and contributing editor for Naval Aviation News.