Aiding maneuvering of obscured objects

Patent metadata

Field	Value
Publication number	US-2019344902-A1
Application number	US-201815974931-A
Country	US
Kind code	A1
Filing date	May 9, 2018
Priority date	May 9, 2018
Publication date	Nov 14, 2019
Grant date	—

Abstract

A system is provided. The system includes a camera on a refueling aircraft to obtain an image of a receiver aircraft having a receptacle for refueling the receiver aircraft. The system also includes a processor on the refueling aircraft to determine an orientation and position of the receiver aircraft relative to the camera, a separation distance and respective positions of a refueling boom of the refueling aircraft and the receptacle. The processor also generates a display of a side perspective view of 3D models of the refueling boom and the receiver aircraft having the receptacle. The display of the side perspective view illustrates a current position of the refueling boom relative to a current position of the receptacle and enables an operator on the refueling aircraft to observe and guide the refueling boom to the receptacle.

First claim

What is claimed is:

1 . A system for aiding maneuvering of a refueling boom of a refueling aircraft, the system comprising:
a camera on the refueling aircraft configured to obtain an image of a receiver aircraft having a receptacle for refueling the receiver aircraft; and a processor on the refueling aircraft, coupled to the camera and configured to at least:
determine, from points on the image of the receiver aircraft, an orientation and position of the receiver aircraft relative to the camera using a pose estimation algorithm and a known relationship between the points on the image of the receiver aircraft and corresponding points on a 3D model of the receiver aircraft;
determine, based on the orientation and position of the receiver aircraft, a separation distance and respective positions of the refueling boom of the refueling aircraft and the receptacle of the receiver aircraft; and
generate a display on a display device in the refueling aircraft, using the separation distance and respective positions of the refueling boom and the receptacle, the display being of a side perspective view of a 3D model of the refueling boom and the 3D model of the receiver aircraft having the receptacle, illustrating a current position of the refueling boom relative to a current position of the receptacle, from a viewpoint of a virtual camera positioned alongside the refueling boom,
wherein the display of the side perspective view illustrating the current position of the refueling boom relative to the current position of the receptacle enables an operator on the refueling aircraft to observe and guide the refueling boom to the receptacle.

2 . The system of claim 1 , wherein the processor being configured to generate the display includes being configured to retrieve and orient positions of the 3D model of the refueling boom and the 3D model of the receiver aircraft based on the separation distance and the respective positions of the refueling boom and the receptacle.

3 . The system of claim 1 , wherein the processor being configured to generate the display includes being configured to overlay indicia on the side perspective view indicating a point of collision between the refueling boom and the receiver aircraft.

4 . The system of claim 1 , further comprising a storage device including the 3D model of the refueling boom and the 3D model of the receiver aircraft.

5 . The system of claim 1 , wherein the respective positions of the refueling boom of the refueling aircraft include a position of the refueling boom, and the processor being configured to determine the separation distance and the respective positions includes being configured to determine the position of the refueling boom using one or more sensors on the refueling boom.

6 . The system of claim 1 , wherein the display of the side perspective view enables the operator on the refueling aircraft to observe and guide a tip of the refueling boom to the receptacle.

7 . A method of aiding maneuvering of a refueling boom of a refueling aircraft, the method comprising:
obtaining, by a camera on the refueling aircraft, an image of a receiver aircraft having a receptacle for refueling the receiver aircraft; and by a processor on the refueling aircraft; determining, from points on the image of the receiver aircraft, an orientation and position of the receiver aircraft relative to the camera using a pose estimation algorithm and a known relationship between the points on the image of the receiver aircraft and corresponding points on a 3D model of the receiver aircraft; determining, based on the orientation and position of the receiver aircraft, a separation distance and respective positions of the refueling boom of the refueling aircraft and the receptacle of the receiver aircraft; and generating a display on a display device in the refueling aircraft, using the separation distance and respective positions of the refueling boom and the receptacle, the display being of a side perspective view of a 3D model of the refueling boom and the 3D model of the receiver aircraft having the receptacle, illustrating a current position of the refueling boom relative to a current position of the receptacle, from a viewpoint of a virtual camera positioned alongside the refueling boom, wherein the display of the side perspective view illustrating the current position of the refueling boom relative to the current position of the receptacle enables an operator on the refueling aircraft to observe and guide the refueling boom to the receptacle.

8 . The method of claim 7 , wherein generating the display includes retrieving and orienting positions of the 3D model of the refueling boom and the 3D model of the receiver aircraft based on the separation distance and the respective positions of the refueling boom and the receptacle.

9 . The method of claim 7 , wherein generating the display includes overlaying indicia on the side perspective view indicating a point of collision between the refueling boom and the receiver aircraft.

10 . The method of claim 7 , further comprising retrieving the 3D model of the refueling boom and the 3D model of the receiver aircraft from a storage device.

11 . The method of claim 7 , wherein the respective positions of the refueling boom of the refueling aircraft include a position of the refueling boom, and wherein determining the separation distance and the respective positions includes determining the position of the refueling boom using one or more sensors on the refueling boom.

12 . The method of claim 7 , wherein the display of the side perspective view enables the operator on the refueling aircraft to observe and guide a tip of the refueling boom to the receptacle.

13 . A computer-readable storage medium for aiding maneuvering of a refueling boom of a refueling aircraft, the computer-readable storage medium being non-transitory and having computer-readable program code stored therein that in response to execution by a processor, causes a computer system to at least:
obtain an image of a receiver aircraft having a receptacle for refueling the receiver aircraft from a camera on the refueling aircraft; determine, from points on the image of the receiver aircraft, an orientation and position of the receiver aircraft relative to the camera using a pose estimation algorithm and a known relationship between the points on the image of the receiver aircraft and corresponding points on a 3D model of the receiver aircraft; determine, based on the orientation and position of the receiver aircraft, a separation distance and respective positions of the refueling boom of the refueling aircraft and the receptacle of the receiver aircraft; and generate a display on a display device in the refueling aircraft, using the separation distance and respective positions of the refueling boom and the receptacle, the display being of a side perspective view of a 3D model of the refueling boom and the 3D model of the receiver aircraft having the receptacle, illustrating a current position of the refueling boom relative to a current position of the receptacle, from a viewpoint of a virtual camera positioned alongside the refueling boom, wherein the display of the side perspective view illustrating the current position of the refueling boom relative to the current position of the receptacle enables an operator on the refueling aircraft to observe and guide the refueling boom to the receptacle.

14 . The computer-readable storage medium of claim 13 , wherein the computer system being caused to generate the display includes being caused to retrieve and orient positions of the 3D model of the refu

Assignees

• Boeing Co

Inventors

• Cramblitt Robert Merrill

Classifications

• B64D39/00 Primary

  Refuelling during flight · CPC title

• B64D43/00 Primary

  Arrangements or adaptations of instruments · CPC title

• G06T7/75

  involving models · CPC title

• G06T17/20

  Finite element generation, e.g. wire-frame surface description, {tesselation} · CPC title

• B64D39/02

  Means for paying-in or out hose · CPC title