Aerial drone utilizing pose estimation

Patent metadata

Field	Value
Publication number	US-10884433-B2
Application number	US-201816100479-A
Country	US
Kind code	B2
Filing date	Aug 10, 2018
Priority date	Aug 28, 2017
Publication date	Jan 5, 2021
Grant date	Jan 5, 2021

Abstract

A computer-implemented method, system, and computer program product are provided for a stabilization system utilizing pose estimation in an aerial drone. The method includes receiving, by a pose estimation system, a plurality of images from one or more cameras. The method also includes predicting, by the pose estimation system, a pose from the score map and a combined feature map, the combined feature map correlated from a pair of the plurality of images. The method additionally includes moving, by a propulsion system, the aerial drone responsive to the pose.

First claim

What is claimed is:

1. An aerial drone utilizing pose estimation, the aerial drone comprising:
one or more cameras;
a propulsion system that moves the aerial drone; and
a stabilization system that controls the propulsion system, the stabilization system includes a pose estimation system that receives a plurality of images from the one or more cameras and predicts a pose from a score map and a combined feature map, the combined feature map generated from a pair of the plurality of images, the pose estimation system including a pose estimation convolutional neural network (CNN) trained utilizing two-dimensional (2D) keypoint displacement loss employing Velodyne points.

2. The aerial drone as recited in claim 1 , wherein the pose estimation system generates feature maps for each of the pair of the plurality of images and generates the combined feature map from the feature maps.

3. The aerial drone as recited in claim 2 , wherein the feature maps are generated with a feature extraction CNN.

4. The aerial drone as recited in claim 1 , wherein the pose estimation system includes a pose estimation CNN trained utilizing three-dimensional (3D) pose regression loss.

5. The aerial drone as recited in claim 1 , wherein the pose estimation system includes a feature weighting network.

6. The aerial drone as recited in claim 5 , wherein the feature weighting network estimates the score map from a pair of the feature maps.

7. The aerial drone as recited in claim 1 , wherein the guidance control system steers the autonomous vehicle around obstacles.

8. The aerial drone as recited in claim 1 , wherein the propulsion system is selected from the group consisting of a one or more jet engine, hover technology, one or more motors, and one or more internal combustion engine.

9. The aerial drone as recited in claim 1 , wherein the combined feature map includes training utilizing motion loss.

10. The aerial drone as recited in claim 9 , wherein the motion loss is determined with an optical flow CNN.

11. The aerial drone as recited in claim 1 , wherein the combined feature map includes training utilizing semantics loss.

12. The aerial drone as recited in claim 11 , wherein the semantics loss is determined with a semantic segmentation CNN.

13. A computer program product for a stabilization system utilizing pose estimation in an aerial drone, the computer program product comprising a non-transitory computer readable storage medium having program instructions embodied therewith, the program instructions executable by a computer to cause the computer to perform a method comprising:
receiving, by a pose estimation system, a plurality of images from one or more cameras;
predicting, by the pose estimation system, a pose from the score map and a combined feature map, the combined feature map correlated from a pair of the plurality of images, the pose estimation system including a pose estimation convolutional neural network (CNN) trained utilizing two-dimensional (2D) keypoint displacement loss employing Velodyne points; and
moving, by a propulsion system, the aerial drone responsive to the pose.

14. A computer-implemented method for a stabilization system utilizing pose estimation in an aerial drone, the method comprising:
receiving, by a pose estimation system, a plurality of images from one or more cameras;
predicting, by the pose estimation system, a pose from the score map and a combined feature map, the combined feature map correlated from a pair of the plurality of images, the pose estimation system including a pose estimation convolutional neural network (CNN) trained utilizing two-dimensional (2D) keypoint displacement loss employing Velodyne points; and
moving, by a propulsion system, the aerial drone responsive to the pose.

15. The computer-implemented method as recited in claim 14 , wherein the combined feature map includes training utilizing motion loss.

16. The computer-implemented method as recited in claim 15 , wherein the motion loss is determined with an optical flow CNN.

17. The computer-implemented method as recited in claim 14 , wherein the combined feature map includes training utilizing semantics loss.

18. The computer-implemented method as recited in claim 17 , wherein the semantics loss is determined with a semantic segmentation CNN.

Assignees

• Nec Lab America Inc
• Nec Corp

Inventors

• Tran Quoc-Huy
• Chandraker Manmohan
• Kim Hyo Jin

Classifications

• B64U2201/10

  autonomous, i.e. by navigating independently from ground or air stations, e.g. by using inertial navigation systems [INS] · CPC title

• B64U2201/00

  UAVs characterised by their flight controls · CPC title

• G06N3/045

  Combinations of networks · CPC title

• G06N3/08

  Learning methods · CPC title

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title