Panoramic photographing method for unmanned aerial vehicle and unmanned aerial vehicle using the same

Patent metadata

Field	Value
Publication number	US-11042997-B2
Application number	US-201816212301-A
Country	US
Kind code	B2
Filing date	Dec 6, 2018
Priority date	Dec 6, 2017
Publication date	Jun 22, 2021
Grant date	Jun 22, 2021

Abstract

A panoramic photographing method for an unmanned aerial vehicle (UAV) and an UAV using the same are provided. The panoramic photographing method includes steps of: disposing a digital camera on an UAV; recording a flying altitude and a flying angle of the UAV of each of photographed pictures when a panoramic photographing process is performed; performing an image stitching process, determining overlapped regions of the two photographed pictures, and obtaining a feature operation region by way of division according to the flying altitude and the flying angle of the UAV; and performing a feature operation in the overlapped regions of the two photographed pictures to determine an image stitching location, and thus generate a panoramic picture.

First claim

What is claimed is:

1. An unmanned aerial vehicle (UAV), comprising:
a flying mechanism;
a wireless transceiver;
a digital camera;
an inertia measurement unit for detecting a flying altitude and a flying angle, and outputting a feedback signal; and
a control circuit coupled to the flying mechanism, the wireless transceiver, the digital camera and the inertia measurement unit, wherein the control circuit controls the flying mechanism to control the flying altitude and the flying angle according to a signal received by the wireless transceiver, and determines the flying altitude and the flying angle according to the feedback signal of the inertia measurement unit, wherein:
when the wireless transceiver receives a panoramic photographing instruction, the control circuit controls the digital camera to perform photographing, and records the flying altitude and the flying angle of the UAV of each of photographed pictures; and
when an image stitching process is performed, overlapped regions of the two photographed pictures photographed in order is determined according to the flying altitude and the flying angle, and a feature operation region is obtained by way of division, and then a feature operation is performed in the feature operation region of the two photographed pictures to determine an image stitching location, and thus generate a panoramic picture,
wherein the digital camera performs picture photographing at a first time to obtain a first photographed picture, and the digital camera performs picture photographing at a second time to obtain a second photographed picture,
wherein the image stitching process comprises:
obtaining a camera's focal length matrix according to focal lengths of the digital camera:
recording, by the control circuit, the flying altitude and the flying angle of the UAV at the first time, and then further calculating, by the control circuit, a first rotation matrix according to a sensor fusion algorithm and the flying altitude and the flying angle recorded at the first time;
recording, by the control circuit, the flying altitude and the flying angle of the UAV at the second time, and then further calculating, by the control circuit, a second rotation matrix according to the sensor fusion algorithm and the flying altitude and the flying angle recorded at the second time;
detecting a displacement of a center of a lens of the digital camera and obtaining a camera displacement matrix;
calculating, by the control circuit, a homograph matrix according to the camera's focal length matrix, the camera displacement matrix, the first rotation matrix and the second rotation matrix; and
substituting image coordinates of the second photographed picture according to the homograph matrix to obtain corresponding image coordinates in the first photographed picture.

2. The UAV according to claim 1 , wherein the inertia measurement unit is a nine-axis sensor comprising:
a triaxial magnetic field sensor;
a triaxial acceleration sensor; and
a triaxial gyroscope.

3. The UAV according to claim 1 , wherein the feature operation comprises:
a scale-invariant feature transform (SIFT) operation.

4. The UAV according to claim 1 , wherein the feature operation comprises:
a speeded up robust features (SURF) operation.

5. The UAV according to claim 1 , wherein the camera's focal length matrix comprises a first focal length matrix and a second focal length matrix, the digital camera photographs the first photographed picture through a first focal length, the control circuit calculates the first focal length matrix according to the first focal length, and the digital camera photographs the second photographed picture through a second focal length, and the control circuit calculates the second focal length matrix according to the second focal length,
where the first focal length is represented as f I , the second focal length is represented as f j , and the first focal length matrix is represented as V I having a mathematical equation of

V
I

=

[

f
I

0

0

0

f
I

0

0

0

f
I

]

where the second focal length matrix is represented as V j having a mathematical equation of

V
J

=

[

f
J

0

0

0

f
J

0

0

0

f
J

]

.

6. The UAV according to claim 5 , wherein the camera displacement matrix is represented as T j having a mathematical equation of

T
J

=

[

1

0

c
jx

0

1

c
jy

Assignees

• Generalplus Technology Inc

Inventors

• Wen Shao-Min
• Bei Jing Jo

Classifications

• H04N23/698

  for achieving an enlarged field of view, e.g. panoramic image capture · CPC title

• B64U2101/30

  for imaging, photography or videography · CPC title

• B64U2201/20

  Remote controls · CPC title

• H04N5/2624

  for obtaining an image which is composed of whole input images, e.g. splitscreen · CPC title

• G06T3/20

  Linear translation of whole images or parts thereof, e.g. panning · CPC title