Journey to the Virtual Procedure Trainer (VPT)

 

From concept to completion, the journey to the Virtual Procedure Trainer (VPT) took many months and involved several expertise within AXIS Flight Simulation to complete.

We spoke with product manager, Marcell Surányi and project manager for innovation management, Zsolt Koltai, about what inspired the VPT and how it was created.

1. How did the idea for the VPT originate?

The development of the VPT was driven by insights from flight training experts and observations from key aviation conferences.

Traditional training methods, while effective, often involve high costs, limited simulator availability and scheduling challenges. Industry professionals highlighted the need for a more accessible, scalable and cost-effective solution that would allow pilots to practice procedures anytime, anywhere.

Market trends also played a crucial role in shaping the VPT. The growing adoption of digital learning tools, advancements in virtual reality (VR) and augmented reality (AR), and the increasing demand for competency-based training models underscored the need for an innovative training platform. It was evident that the first adopters – just as with many other technological advancements – were in military aviation. Their early use of VR for mission rehearsal and procedural training validated the potential of the technology and provided valuable insights into its effectiveness.

The idea for the VPT was to leverage the latest technology to create an immersive and interactive training environment where pilots could develop muscle memory and procedural proficiency without the constraints of traditional simulators.

1. How did you approach the integration of virtual reality technologies? Is this something AXIS had done previously?

Integrating this VR technology was a new challenge for AXIS. To ensure success, we partnered with Propair Flight, a company with prior experience in VR technology, to co-develop the trainer.

Before joining forces with Propair, we conducted an extensive market analysis to better understand the technological landscape. This research helped us outline a trainer concept that we aimed to bring to life. To build on this foundation, we experimented in-house with various VR and MR solutions, exploring both high-end and more accessible technologies to gain practical, hands-on experience.

Once our initial ideas took shape, we worked closely with internal and partner subject matter experts to validate and refine our vision.

Although VR technology was new to AXIS, the company has a long-established history in flight training device development, manufacturing and flight testing. This deep expertise in training methodologies, aircraft system logic and architecture provided a strong foundation for developing the VPT.

1. Could you walk us through the key stages of the VPT’s design and development?

The design and development of the VPT followed a structured, iterative process to ensure it met the needs of pilots, instructors and training organisations.

• Research and concept development

The first stage involved extensive research, gathering insights from flight training experts, airline operators and regulatory bodies. We analysed existing training challenges, market trends and the growing role of VR in aviation. Observations from aviation conferences and early military aviation adopters further reinforced the need for a scalable and immersive training solution.

• Prototyping and technology selection

Once the first concept was defined, we focused on selecting the right technology stack. This included choosing VR headsets and interactive 3D modelling tools. We developed initial prototypes to test different approaches, ensuring that the VPT could provide a realistic cockpit environment, intuitive controls and high-fidelity interactions.

• Content development and procedure integration

A critical part of development was integrating aircraft-specific procedures. Working closely with subject-matter experts and operators, we mapped out standard operating procedures (SOPs) for the first aircraft type. This stage also involved developing interactive scenarios, ensuring pilots could practice flows, emergency procedures and checklist execution in a realistic virtual environment that was not possible before.

• Testing and iteration

We conducted multiple rounds of testing with pilots, instructors and even non-experts to refine the experience. Feedback on usability, realism and effectiveness led to several iterations, such as improving the interface design, enhancing feedback mechanisms and optimising performance.

• Continuous improvement and expansion

Development doesn’t stop at first deployment. We continue to refine the VPT based on user requests and feedback, technological advancements and regulatory updates. Future enhancements will include AI-driven coaching, expanded aircraft models and system familiarisation and deeper integration with customer learning management systems.

1. What were some of the major challenges encountered during the manufacturing process?

Our initial VR headset lacked hand tracking and relied on controllers, using the Unity game engine for development. However, after the first iterations and trials, we realised that this setup would not allow us to achieve the level of immersion and quality we aimed for.

This led us to switch to a headset with superior display quality, which also allowed us to eliminate the need for controllers. This introduced a new challenge, creating a seamless user experience using only hand gestures for navigation within the software.

Additionally, we found that the Unity engine did not provide the level of flexibility we required, so we transitioned to Unreal Engine for the second iteration of our software and expanded our team with specialists who could work seamlessly within this new environment.

1. How did you select the final VR headset for the VPT?

Choosing the right VR headset and tracking technology was crucial. We focused on display clarity, refresh rates and ergonomic design to provide users with a comfortable and immersive experience. Early iterations relied on controller-based interactions, but as we transitioned to hand and eye-tracking technology, we needed hardware that could precisely detect motion without compromising performance.

To ensure the quality and reliability of the final product, we selected one of the market leaders in VR technology. Varjo not only manufactures high-quality headsets that feature the capabilities we were seeking, but they are also already proved that with their head sets it is possible to achieve a level of quality that even aviation authorities approve.

For example, high fidelity displays, built-in hand and eye-tracking are critical features that aren't found in lower-tier hardware.

1. How does the implementation of hand and eye tracking technologies in the VPT contribute to the overall training effectiveness?

Hand and eye-tracking technologies play a crucial role in making the simulation more intuitive, realistic and effective. By integrating these advanced features, flight training programmes can achieve a more immersive, responsive and accurate training environment, which closely mimics real-world scenarios.

Hand-tracking technology allows the system to detect and interpret the movements and gestures of a pilot’s hand in real time. The technology relies on sensors or cameras to monitor hand positions, finger movements and gestures.

It enhances the realism of the virtual environment by enabling a more natural, intuitive interaction with the cockpit’s components. This allows the pilot to make precise, immediate adjustments to the control systems, throttle or switches with a shorter learning curve compared to traditional controller-based methods.

Eye-tracking technology tracks the movement and focus of the user’s eyes, providing insight into where attention is directed during different stages of the flight. By analysing patterns of what the trainee is focusing on, the system can assess if attention is being paid to critical aspects such as flight instruments, terrain, visual checks and air traffic.

1. How is the VPT customised for different customers and aircraft types?

Customisation is individual to each customer. The trainer is adapted to the aircraft and processes required. To meet regulations, the training procedures must be adjusted to the SOPs of the given operator. Also, the aircraft’s specific graphical modelling and simulation of the systems is needed.

1. The VPT includes a built-in AI instructor. What benefits does it offer to trainees during their simulation sessions?

The built in AI instructor was developed with experts in flight simulation training, including pilots and instructors. By asking questions around different technical domains and procedures, the instructor considers the natural way that students learn to fly, as well as through repeating the same workflows. The virtual co-pilot enables students to learn and practice in a multi-crew environment, without the need for additional personnel at the early stages of training. At the same time, the student can address questions directly to the AI instructor during training in a natural conversational way on procedures or even on details on the aircraft systems.

1. How do you envision the evolution of VR technologies in pilot training?

The use of VR offers a high level of customisation for both training organisations and the pilots they train. Our goal is to develop products that enhance pilot training by providing a highly autonomous and natural experience, comparable to having an instructor present. This allows training to take place not only in designated centres but also remotely, such as at home.

Read all about AXIS Flight Simulation’s Virtual Procedure Trainer (VPT).