Image processing method, and unmanned aerial vehicle and system

Patent metadata

Field	Value
Publication number	US-11070725-B2
Application number	US-202016802041-A
Country	US
Kind code	B2
Filing date	Feb 26, 2020
Priority date	Aug 31, 2017
Publication date	Jul 20, 2021
Grant date	Jul 20, 2021

Abstract

Image processing method, drone, and drone-camera system are provided. The method includes acquiring, according to a current environmental parameter of the drone, a target sky image that matches the current environmental parameter; and determining a direction parameter of the camera device when capturing a to-be-stitched image. The to-be-stitched image is an image captured under the current environmental parameter. The method further includes stitching the target sky image with the to-be-stitched image according to the direction parameter to obtain a panoramic image.

First claim

What is claimed is:

1. An image processing method, applied to a drone, comprising:
acquiring, according to a current environmental parameter of the drone, a meteorological satellite image that matches the current environmental parameter;
obtaining, according to the meteorological satellite image, a target sky image that matches the current environmental parameter;
determining a direction parameter of a camera device associated with the drone when capturing a to-be-stitched image, wherein the to-be-stitched image is an image captured under the current environmental parameter; and
stitching the target sky image with the to-be-stitched image according to the direction parameter to obtain a panoramic image, wherein:
the target sky image is obtained by one of: an imaging clipping, a selection from a material library, and an image-recognition process,
wherein obtaining the target sky image by the image clipping includes:
determining, according to a sky range value of the to-be-stitched image, a clipping range of the meteorological satellite image;
obtaining, based on a position parameter of the drone, a regional image from the meteorological satellite image according to the clipping range; and
determining, according to the regional image obtained through image clipping, the target sky image.

2. The method according to claim 1 , wherein:
the current environmental parameter of the drone includes at least one of a time parameter, a height parameter, or a position parameter.

3. The method according to claim 2 , wherein:
when the target sky image is obtained by the image clipping, prior to obtaining the target sky image according to the meteorological satellite image, the method further comprises:
determining a sky range value of the to-be-stitched image, according to a current height parameter and a position parameter of the drone and an internal parameter of the camera device.

4. The method according to claim 3 , wherein the obtaining, based on the position parameter of the drone, the regional image from the meteorological satellite image according to the clipping range includes:
taking a position of the drone as a clipping center and obtain, according to the clipping range, the regional image from the meteorological satellite image.

5. The method according to claim 4 , wherein the obtaining, based on the position parameter of the drone, the regional image from the meteorological satellite image according to the clipping range includes:
performing feature recognition processing on the meteorological satellite image to obtain a cloud layer image of the meteorological satellite image; and
obtaining the regional image from the cloud image according to the clipping range.

6. The method according to claim 3 , wherein the determining the target sky image according to the regional image obtained through the image clipping:
performing an enlargement process on the regional image obtained through the image clipping; and
determining a target image from the enlarged regional image, and obtaining the target sky image according to the target image.

7. The method according to claim 6 , wherein the performing the enlargement process on the regional image obtained through the image clipping includes:
determining a height parameter of the meteorological satellite image and a height parameter of the drone;
determining, according to the height parameter of the meteorological satellite image and the height parameter of the drone, a scale value for performing the enlargement process on the regional image obtained through the image clipping; and
enlarging the regional image obtained through image clipping according to the scale value.

8. The method according to claim 6 , further including:
performing resolution filling on the enlarged regional image to obtain the target sky image.

9. The method according to claim 1 , wherein:
when the target sky image is obtained by the selection from the material library, the obtaining the target sky image according to the meteorological satellite image includes:
matching the meteorological satellite image with the sky images in the material library, wherein the material library includes a pre-established set of sky images; and
selecting a sky image that matches the meteorological satellite image from the material library as the target sky image.

10. The method according to claim 9 , wherein the matching the meteorological satellite image with the sky images in the material library includes:
matching one or more of a visible light cloud map, an infrared light cloud map, a water vapor cloud map, a surface temperature map, and a sea surface temperature map in the meteorological satellite image with the sky images in the material library.

11. The method according to claim 1 , wherein:
when the target sky image is obtained by the image-recognition process, the obtaining the target sky image according to the meteorological satellite image includes:
performing the image-recognition process on the meteorological satellite image to obtain sky description information of the meteorological satellite image; and
generating a virtual sky image according to the sky description information of the meteorological satellite image, and using the virtual sky image as the target sky image.

12. The method according to claim 11 , wherein the sky description information includes:
one or more of description information obtained based on a visible light cloud map, description information obtained based on an infrared light cloud map, description information obtained based on a water vapor cloud map, description information obtained on a surface temperature map, and description information obtained based on a sea surface temperature map.

13. The method according to claim 1 , wherein the acquiring, according to the current environmental parameter, the target sky image that matches the current environmental parameter includes:
determining at least one user identifier according to the environmental parameter of the drone;
corresponding to each of the at least one user identifier, acquiring a sky image; and
selecting, from acquired sky images, a matching sky image that matches the environmental parameter as the target sky image.

14. The method according to claim 13 , wherein the selecting, from the acquired sky images, the matching sky image that matches the environmental parameter as the target sky image includes:
sorting matching sky images according to a degree of matching between the matching sky images and the environmental parameter, and pushing sorting results to a display interface; and
in response to a determining operation received at the display interface, obtaining the target sky image according to the matching sky image indicated by the determining operation.

15. The method according to claim 1 , wherein the stitching the target sky image with the to-be-stitched image according to the direction parameter to obtain the panoramic image includes:
determining a stitching direction of the target sky image according to the direction parameter;
extracting edge feature points of the target sky image; and
stitching the target sky image and the to-be-stitched image together according to the edge feature points and the stitching direction of the target sky image to obtain a panoramic image.

16. A drone, comprising:
a memory and a processor, wherein:
the memory is configured to store program instructions; and
the processor, is configured to execute the program instructions stored in the memory, and when the program instructions are executed, the processor

Assignees

• Sz Dji Technology Co Ltd

Inventors

• Zheng Hongyong
• Lu Zhenbo
• Hu Pan
• Cao Zisheng

Classifications

• H04N23/698 Primary

  for achieving an enlarged field of view, e.g. panoramic image capture · CPC title

• H04N23/80

  Camera processing pipelines; Components thereof · CPC title

• G06F18/22

  Matching criteria, e.g. proximity measures · CPC title

• G06V20/13 Primary

  Satellite images · CPC title

• B64U20/87

  Mounting of imaging devices, e.g. mounting of gimbals · CPC title