Human machine interface system

Patent metadata

Field	Value
Publication number	US-9938010-B2
Application number	US-201514957067-A
Country	US
Kind code	B2
Filing date	Dec 2, 2015
Priority date	Dec 2, 2014
Publication date	Apr 10, 2018
Grant date	Apr 10, 2018

Abstract

An electronic human machine interface (HMI) module is configured to receive at least one input from an operator of a manually-operated vehicle. The electronic HMI module includes an electronic graphical display unit and an electronic control module. The electronic graphical display unit is configured to display information corresponding to the manually-operated vehicle and information corresponding to an autonomously operated unmanned vehicle (UV) located remotely from the manually-operated vehicle. The electronic control module is in signal communication with the UV and is configured to receive an electronic image signal from the UV. The electronic control module is further configured to display a real-time image captured by the UV on the electronic graphical display unit based on the image signal.

First claim

The invention claimed is:

1. An electronic human machine interface (HMI) module configured to receive at least one input from an operator of a manually-operated vehicle, the electronic HMI module comprising:
an electronic graphical display unit configured to display information corresponding to the manually-operated vehicle and information corresponding to an autonomously operated unmanned vehicle (UV) located remotely from the manually-operated vehicle, wherein the graphical display unit further displays at least one of a handover location associated with establishing a handover communication session and an estimated time of arriving at the handover location; and
an electronic control module in signal communication with the UV, the electronic control module configured to receive an electronic image signal from the UV and to display a real-time image captured by the UV on the electronic graphical display unit based on the image signal during the handover communication session;
wherein the electronic control module is operable to establish the handover communication session between the manually-operated vehicle and the UV when the manually-operated vehicle and the UV are within a distance threshold, wherein the manually-operated vehicle continuously obtains an up-to-date analysis of a point of interest from the UV while travelling to the point of interest, the continuous up-to-date analysis being initiated in response to establishing the handover communication.

2. The electronic HMI module of claim 1 , wherein the electronic control module selectively operates the electronic graphical display unit in a first mode and a second mode during the handover communication session.

3. The electronic HMI module of claim 2 , wherein the graphical display unit displays an interface of selectable actions to control operation of the manually-operated vehicle and the UV when operating in the first mode, and wherein the graphical display unit displays real-time image captured by the UV when operating in the second mode.

4. The electronic HMI module of claim 3 , wherein the interface of selectable actions include a first set of selectable actions to control at least one of the handover communication session and navigation of the UV, and a second set of selectable action to control operation of at least one UV image sensor installed on the UV.

5. The electronic HMI module of any one of claims 1 - 4 , wherein the electronic control module outputs an autonomous operation control signal that commands the UV to traverse to a geographical point of interest while simultaneously transmitting the image signal to the electronic control module.

6. The electronic HMI module of claim 5 , wherein the UV automatically initiates the at least one image sensor to capture the real-time image in response to establishing the handover communication session.

7. The electronic HMI module of claim 6 , wherein the electronic control module receives an input commanding the at least one UV sensor to selectively output the image signal as one of an infrared image signal or a video signal.

8. A method of visually enhancing a geographical point of interest using a human machine interface (HMI) system, the method comprising:
displaying at least one of a handover location associated with establishing a handover communication session and an estimated time of arriving at the handover location on a graphical display of the HMI module installed in the manually-operated vehicle:
establishing the handover communication session to electrically communicate at least one electrical signal between a manually-operated vehicle and an autonomously operated unmanned vehicle (UV) located remotely from the manually-operated vehicle when the manually-operated vehicle and the UV are within a distance threshold;
automatically navigating the UV to a point of interest designated using an electronic HMI module installed on the manually-operated vehicle;
capturing a real-time image of the point of interest using at least one image sensor installed on the UV, and electrically transmitting the real-time image to the manually-operated vehicle wherein the manually-operated vehicle continuously obtains an up-to-date analysis of the point of interest from the UV while travelling to the point of interest, the continuous up-to-date analysis being initiated in response to establishing the handover communication; and
displaying the real-time image captured by the UV on the graphical display.

9. The method of claim 8 , further comprising automatically establishing the handover communication session in response to satisfying the distance threshold between the manually-operated vehicle and the UV.

10. The method of claim 9 , wherein the UV automatically begins capturing the real-time image and transmitting the real-time image to the manually-operated vehicle in response to establishing the handover communication session.

11. The method of claim 8 , wherein the manually-operated vehicle is a manually-operated aerial vehicle and the UV is an unmanned aerial vehicle.

12. The method of claim 11 , wherein the point of interest is a landing area or evacuation area of the unmanned aerial vehicle.

13. The method of claim 12 , wherein the unmanned aerial vehicle automatically navigates to the point of interest in response to establishing the handover communication session.

14. The method of claim 13 , wherein the unmanned aerial vehicle continuously traverses about a flight path above the point of interest area while transmitting the real-time image to the manually-operated aerial vehicle in response to reaching the point of interest.

15. The method of claim 14 , further comprising displaying the real-time image of the point of interest on the graphical display unit before the manually-operated vehicle arrives at the point of interest.

Assignees

• Sikorsky Aircraft Corp

Inventors

• Lampazzi Margaret Macisaac
• Guido Thomas
• Howland Gary
• Bertuccelli Luca F
• Pupalaikis Robert

Classifications

• B64U2101/30

  for imaging, photography or videography · CPC title

• B64U2201/10

  autonomous, i.e. by navigating independently from ground or air stations, e.g. by using inertial navigation systems [INS] · CPC title

• G01C23/00

  Combined instruments indicating more than one navigational value, e.g. for aircraft; Combined measuring devices for measuring two or more variables of movement, e.g. distance, speed or acceleration · CPC title

• B64C2201/127

  Operations & Transport · mapped topic

• G08G5/0069

  Physics · mapped topic