Low-cost, long-term aerial imagery

Patent metadata

Field	Value
Publication number	US-11153535-B2
Application number	US-201916731713-A
Country	US
Kind code	B2
Filing date	Dec 31, 2019
Priority date	Jan 21, 2017
Publication date	Oct 19, 2021
Grant date	Oct 19, 2021

Abstract

An imaging system that includes a camera mounted on an aerial platform, for example a balloon, allows a user to increase the longevity of the camera's battery by remote control. A user may capture imagery at a time scale of interest and desired power consumption by adjusting parameters for image capture by the camera. A user may adjust a time to capture an image, a time to capture a video, or a number of cycles per time period to capture one or more images as the aerial platform moves in a region of interest to change power consumption for imaging. The system also provides imaging alignment to account for unwanted movement of the aerial platform when moved in the region of interest. Additionally, a mounting device is provided that is simple and inexpensive, and that allows a camera to remain positioned in a desired position relative to the ground.

First claim

What is claimed is:

1. An apparatus comprising:
one or more processors and,
memory in communication with the one or more processors, the memory comprising code that, when executed, causes the one or more processors to control the apparatus to:
receive first data, the first data comprising information related to a region to be imaged and information related to a field of vision of a camera device positioned on an aerial platform above the region;
receive second data from the camera device, the second data including a plurality of images taken by the camera device;
extract a plurality of features from the plurality of images;
reject selected features of the plurality of features that do not match across the plurality of images;
determine a homography that maps features in each image of the plurality of images to features in another image of the plurality of images; and,
apply the homography to a current image of the plurality of images and align the current image to a previous image of the plurality of images taken previously to the current image,
wherein the aerial platform includes a steering device and the code, when executed, further causes the one or more processors to control the apparatus to:
determine, based at least on the first data, a path within the region and provide information on the path to a mobile entity;
receive the second data from the camera device during a time period as the mobile entity moves the aerial platform along the path using the steering device,

wherein the path comprises a first path and the code, when executed, further causes the one or more processors to control the apparatus to:
receive third data from the position sensor of the camera device as the mobile entity moves the aerial platform along the first path using the steering device;
determine, based at least on the third data, at least one unimaged area of the region that remains unimaged in the movement along the first path; and,
determine, based at least on the at least one unimaged area and the first data, a second path within the region and provide information on the second path to the mobile entity.

2. The apparatus of claim 1 , wherein the plurality of images comprises a series of images taken at discrete times over the time period and the second data further includes position data of the camera device at the time of capture of each of the series of images.

3. The apparatus of claim 1 , wherein the plurality of images comprises a video taken over the time period and the second data further includes position data of the camera device at the time of capture of each frame of the video.

4. The apparatus of claim 1 , wherein the apparatus further comprises a mobile device including a user interface in communication with the one or more processors, and the code, when executed, further causes the one or more processors to control the apparatus to display the current and previous images aligned at the user interface.

5. The apparatus of claim 1 , further comprising a balloon, wherein the balloon provides the aerial platform on which the camera device is mounted, and the steering device comprises a tether.

6. A method comprising:
receiving first data, the first data comprising information related to a region to be imaged and information related to a field of vision of a camera device positioned on an aerial platform above the region;
receiving second data from the camera device, the second data including a plurality of images taken by the camera device;
extracting a plurality of features from the plurality of images;
rejecting selected features of the plurality of features that do not match across the plurality of images;
determining a homography that maps features in each image of the plurality of images to features in another image of the plurality of images;
applying the homography to a current image of the plurality of images and align the current image to a previous image of the plurality of images taken previously to the current image;
determining, based at least on the first data, a path within the region and providing information on the path to a mobile entity;
receiving the second data from the camera device during a time period as the mobile entity moves the aerial platform along the path using a steering device;
receiving third data from the position sensor of the camera device as the mobile entity moves the aerial platform along the path using the steering device;
determining, based at least on the third data, at least one unimaged area of the region that remains unimaged in the movement along the path; and
determining, based at least on the at least one unimaged area and the first data, an additional path within the region and providing information on the additional path to the mobile entity.

7. The method of claim 6 , wherein the plurality of images comprises a series of images taken at discrete times over the time period and the second data further includes position data of the camera device at the time of capture of each of the series of images.

8. The method of claim 6 , wherein the plurality of images comprises a video taken over the time period and the second data further includes position data of the camera device at the time of capture of each frame of the video.

9. The method of claim 6 , further comprising displaying the current and previous images aligned on a user interface of a mobile device.

10. The method of claim 6 , wherein the aerial platform comprises a balloon to which the camera device is attached, and the steering device comprises a tether connected to the balloon.

11. A non-transitory computer-readable storage medium having stored thereon computer executable instructions, which when executed by a computer device, cause the computer device to:
receive first data, the first data comprising information related to a region to be imaged and information related to a field of vision of a camera device positioned on an aerial platform above the region;
receive second data from the camera device, the second data including a plurality of images taken by the camera device;
extract a plurality of features from the plurality of images;
reject selected features of the plurality of features that do not match across the plurality of images;
determine a homography that maps features in each image of the plurality of images to features in another image of the plurality of images;
apply the homography to a current image of the plurality of images and align the current image to a previous image of the plurality of images taken previously to the current image;
determine, based at least on the first data, a path within the region and provide information on the path to a mobile entity;
receive the second data from the camera device during a time period as the mobile entity moves the aerial platform along the path using the steering device;
receive third data from the position sensor of the camera device as the mobile entity moves the aerial platform along the path using the steering device;
determine, based at least on the third data, at least one unimaged area of the region that remains unimaged in the movement along the path; and
determine, based at least on the at least one unimaged area and the first data, an additional path within the region and providing information on the additional path to the mobile entity.

12. The non-transitory computer-readable storage medium of claim 11 , wherein the plurality of images comprises a series of images taken over the time period and the second data further includes position data of the camera device at the time of capture of each of the series of images.

13. The non-transit

Assignees

• Microsoft Technology Licensing Llc

Inventors

• Chandra Ranveer
• Swaminathan Manohar
• Swamy Vasuki Narasimha
• Kapetanovic Zerina
• Vasisht Deepak
• Kumar Akshit
• Badam Anirudh
• Ranade Gireeja
• Sinha Sudipta
• Patil Rohit

Classifications

• H04N7/185 Primary

  from a mobile camera, e.g. for remote control · CPC title

• H04N7/183 Primary

  for receiving images from a single remote source · CPC title

• H04N23/63

  by using electronic viewfinders · CPC title

• H04N23/64

  Computer-aided capture of images, e.g. transfer from script file into camera, check of taken image quality, advice or proposal for image composition or decision on when to take image · CPC title

• H04N23/62

  Control of parameters via user interfaces · CPC title