Aircraft pilot training system, method and apparatus for theory, practice and evaluation

Patent metadata

Field	Value
Publication number	US-10529248-B2
Application number	US-201514741175-A
Country	US
Kind code	B2
Filing date	Jun 16, 2015
Priority date	Jun 19, 2014
Publication date	Jan 7, 2020
Grant date	Jan 7, 2020

Abstract

A system and method for training an aircraft pilot employs a station that delivers training lessons in the form of output and also receives input data from the trainee. An apparatus comprising a computational system is capable of displaying content from different types of media, including virtual reality, augmented reality, and variations. A mixed reality environment interacts with a human being, simulating the response of real objects, and also provides necessary information to achieve Learning Objectives for a specific part of training. A register or record of the interactions is used for evaluation.

First claim

What is claimed:

1. A virtual airplane cockpit system structured to receive inputs from first and second human operators and provide a simulation environment simulating real world behavior in response to the inputs, comprising:
a physical mockup of an aircraft cockpit including user-manipulable physical controls attached to sensors, the sensors providing output signals indicating the manipulation states of the physical controls;
at least one tracking device that tracks the position, orientation and/or movement of the head, torso, arms, hands and fingers of each of the first and second human operators;
an augmented reality generator that, in response to the tracked position, orientation and/or movement of the head, torso, arms, hands and fingers of the first and second human operators, generates an augmented reality simulation image for overlay onto the first and second human operators' real world image perception of the physical mockup, the augmented reality generator enabling the first human operator to view the hands of the second human operator and what the hands of the second human operator interact with; and
at least one processor operatively coupled to the augmented reality generator, the at least one processor being responsive to the tracked position, orientation and/or movement of the head, torso, arms and fingers of the first and second human operators, the at least one processor controlling the augmented reality generator to generate augmented reality simulation images and associated audio signals that present objective learning exercise and present simulations including augmented reality feedback, wherein the at least one tracking device tracks movement of the first and second users' head, torso, arms and fingers the same way as will be required in a real aircraft cockpit.

2. The system of claim 1 further including a simulation subsystem that allows the first and second human operators to learn from interaction with the control and other systems interfaces since the first contact.

3. The system of claim 1 further including an output interface that delivers information in different formats and ways, reaching the first and second human operators through various senses including vision, hearing, touch and biomechanical movement.

4. The system of delivering information of claim 3 , wherein the first and second human operators are able to learn theoretical contents while interacting with systems controllers and interfaces following predetermined procedures, wherein the theoretical and practical contents are learned together.

5. The system of claim 1 further including at least one processor that receives, logs and processes the inputs from the first and second human operators' interactions to evaluate the achievement of lesson goals.

6. The system of claim 1 wherein the at least one processor is configured to integrate theory, practice and evaluation, all together on the same phase of training.

7. The system of claim 6 wherein the at least one processor is further configured to provide contents of lessons that are oriented to real system operation and reflect the first and second human operators' visions of the controllers and interfaces.

8. The system according to claim 1 , configured to replace computer based training (CBT) and procedures training devices (FTD).

9. The system according to claim 1 , configured to reduce the time necessary to achieve specific Learning Objectives.

10. A virtual airplane simulator providing a simulation environment, the simulator comprising:
a physical mockup of an aircraft cockpit including at least one user-manipulable physical control attached to at least one sensor, the sensor providing an output signal that indicates the manipulation state of the physical control;
at least one tracking device that tracks the position, orientation and/or movement of the head, torso, arms, hands and fingers of first and second human operators;
at least one processor coupled to tracking device, the at least one processor including a sound subsystem, a 3D graphics subsystem, a simulation control subsystem, an augmented reality subsystem and a sensor subsystem;
a first head mounted display for use by the first human operator;
a second head mounted display for use by the second human operator;
the simulation control subsystem being coupled to at least one non-transitory memory device storing lesson plans providing Learning Objectives;
the augmented reality generator being configured to, in response to the tracked position, orientation and/or movement, generate an augmented reality simulation image for the first and second head mounted displays to overlay onto the first and second human operators' real world image perception of the physical mockup through the first and second head mounted displays, the augmented reality generator being further configured to enable the first and second human operators to see each others' head, torso, arms, hands and fingers and what the head, torso, arms, hands and fingers interact with,
wherein the system is configured to generate augmented reality simulation images and sounds for both a pilot and a co-pilot simultaneously as a team; and
the simulation control system is further configured to be responsive to the tracked position, orientation and/or movement of the head, torso, arms and fingers of the first and second human operators, the simulation control system controlling the augmented reality generator to generate augmented reality simulation images and associated audio signals that present the Learning Objectives and require augmented reality feedback, wherein the at least one tracking device tracks movement of the first and second users' head, torso, arms and fingers the same way as will be required in a real aircraft cockpit.

11. The simulator of claim 10 wherein the first head mounted display is structured to provide a transparent portion that permits the first human operator to view the real world physical mockup and the interactions of the hands of the second human operator with the real world physical mockup, and superimposes display of virtual objects the second human operator's hands interact with onto the first human operator's view through the transparent portion of the real world physical mockup.

12. The simulator of claim 10 wherein the tracking device includes sensors that capture, measure or evaluate the movements of arms, hands and fingers of the first and second human operators and also sense manipulation by the hand and finger movement and position of physical controls that are part of the physical mockup.

13. The simulator of claim 10 wherein the physical mockup comprising real lamps, alarms, buttons, levers and sliders that give, to the first and second human operators' hands and fingers, realistic impressions of reach, movement, touch and tactile sensing.

14. The simulator of claim 10 wherein the physical mockup comprises hand-manipulable elements selected from the group consisting of buttons, knobs, joysticks, touch screens, dials, levers, lights, indicators, steering wheels, log book stations, navigational displays, informational displays, instrument displays, status displays, numerical displays, text displays, video displays, gauges, microphones, jacks, or other devices that may be present and manipulated by hands on an actual flight deck of an aircraft.

15. The simulator of claim 10 wherein the control provides force feedback to the first and second operators' hands and fingers.

16. The simulator of claim 10 wherein the tracking device tracks at least one hand of each of the first and second human operators a

Assignees

• Embraer Sa

Inventors

• Chavez Nilson Emanuel Bezerra
• De Miranda Cara Guilherme
• De Macedo Nelson Jose Issa

Classifications

• G09B9/302 Primary

  the image being transformed by computer processing, e.g. updating the image to correspond to the changing point of view · CPC title

• G09B5/06

  with both visual and audible presentation of the material to be studied · CPC title

• G09B9/24

  including display or recording of simulated flight path · CPC title