Estimating a height of a cloud depicted in an image

Patent metadata

Field	Value
Publication number	US-10902260-B2
Application number	US-201816203742-A
Country	US
Kind code	B2
Filing date	Nov 29, 2018
Priority date	Nov 29, 2018
Publication date	Jan 26, 2021
Grant date	Jan 26, 2021

Abstract

Methods, systems and computer program products for estimating a height of a cloud depicted in an image are provided. Aspects include extracting metadata from the image, the metadata including a latitude, a longitude, and heading of a camera that captured the image and a time at which the image was captured. Aspects also include creating a three-dimensional model of a landscape depicted in the image and inserting a simulated cloud into the three-dimensional model. Aspects further include creating a simulated image of the three-dimensional model, including the simulated cloud and determining a similarity score of the cloud depicted in the image to the simulated cloud in the simulated image. Based on a determination that the similarity score exceeds a threshold value, aspects include providing a height of the simulated cloud in the three-dimensional model as the height of the cloud depicted in the image.

First claim

What is claimed is:

1. A computer-implemented method for estimating a height of a cloud depicted in an image, the method comprising:
extracting metadata from the image, the metadata including a latitude, a longitude, and heading of a camera that captured the image and a time at which the image was captured;
creating a three-dimensional model of a landscape depicted in the image;
inserting a simulated cloud into the three-dimensional model;
creating a simulated image of the three-dimensional model, including the simulated cloud;
determining a similarity score of the cloud depicted in the image to the simulated cloud in the simulated image; and
based on a determination that the similarity score exceeds a threshold value, providing a height of the simulated cloud in the three-dimensional model as the height of the cloud depicted in the image.

2. The computer-implemented method of claim 1 , wherein the method further comprises iteratively repeating:
inserting a second simulated cloud into the three-dimensional model;
creating a second simulated image of the three-dimensional model, including the second simulated cloud; and
determining a second similarity score of the cloud depicted in the image to the second simulated cloud in the simulate image until the second similarity score is above the threshold value.

3. The computer-implemented method of claim 1 , wherein a type of the simulated cloud inserted into the three-dimensional model is determined at least in part based on a satellite image including the latitude and longitude at the time that the image was captured.

4. The computer-implemented method of claim 1 , wherein a type of the simulated cloud inserted into the three-dimensional model is determined at least in part based on a weather forecast for the latitude and longitude at the time that the image was captured.

5. The computer-implemented method of claim 1 , wherein the three-dimensional model is created based on data extracted from the image.

6. The computer-implemented method of claim 1 , wherein the three-dimensional model is created based on data extracted from a second image captured at the latitude, the longitude, and the heading, wherein the second image depicts a clear sky.

7. The computer-implemented method of claim 1 , further comprising creating a database of simulated images, each simulated image having simulated clouds with different characteristics.

8. The computer-implemented method of claim 1 , wherein the three-dimensional model is created by obtaining an appropriate digital elevation model for the latitude, the longitude, and the heading of the camera at the time at which the image was captured.

9. A system comprising:
a processor communicatively coupled to a memory, the processor configured to:
extract metadata from an image depicting a cloud, the metadata including a latitude, a longitude, and heading of a camera that captured the image and a time at which the image was captured;
create a three-dimensional model of a landscape depicted in the image;
insert a simulated cloud into the three-dimensional model;
create a simulated image of the three-dimensional model, including the simulated cloud;
determine a similarity score of the cloud depicted in the image to the simulated cloud in the simulated image; and
based on a determination that the similarity score exceeds a threshold value, provide a height of the simulated cloud in the three-dimensional model as the height of the cloud depicted in the image.

10. The system of claim 9 , wherein the processor is further configured to iteratively:
insert a second simulated cloud into the three-dimensional model;
create a second simulated image of the three-dimensional model, including the second simulated cloud; and
determine a second similarity score of the cloud depicted in the image to the second simulated cloud in the simulate image until the second similarity score is above the threshold value.

11. The system of claim 9 , wherein a type of the simulated cloud inserted into the three-dimensional model is determined at least in part based on a satellite image including the latitude and longitude at the time that the image was captured.

12. The system of claim 9 , wherein a type of the simulated cloud inserted into the three-dimensional model is determined at least in part based on a weather forecast for the latitude and longitude at the time that the image was captured.

13. The system of claim 9 , wherein the three-dimensional model is created based on data extracted from the image.

14. The system of claim 9 , wherein the three-dimensional model is created based on data extracted from a second image captured at the latitude, the longitude, and the heading, wherein the second image depicts a clear sky.

15. The system of claim 9 , wherein the processor is further configured to create a database of simulated images, each simulated image having simulated clouds with different characteristics.

16. A computer program product comprising a computer readable storage medium having program instructions embodied therewith the program instructions executable by a computer processor to cause the computer processor to perform a method comprising:
extracting metadata from an image depicting a cloud, the metadata including a latitude, a longitude, and heading of a camera that captured the image and a time at which the image was captured;
creating a three-dimensional model of a landscape depicted in the image;
inserting a simulated cloud into the three-dimensional model;
creating a simulated image of the three-dimensional model, including the simulated cloud;
determining a similarity score of the cloud depicted in the image to the simulated cloud in the simulated image; and
based on a determination that the similarity score exceeds a threshold value, providing a height of the simulated cloud in the three-dimensional model as the height of the cloud depicted in the image.

17. The computer program product of claim 16 , wherein the method further comprises iteratively repeating:
inserting a second simulated cloud into the three-dimensional model;
creating a second simulated image of the three-dimensional model, including the second simulated cloud; and
determining a second similarity score of the cloud depicted in the image to the second simulated cloud in the simulate image until the second similarity score is above the threshold value.

18. The computer program product of claim 16 , wherein a type of the simulated cloud inserted into the three-dimensional model is determined at least in part based on a satellite image including the latitude and longitude at the time that the image was captured.

19. The computer program product of claim 16 , wherein a type of the simulated cloud inserted into the three-dimensional model is determined at least in part based on a weather forecast for the latitude and longitude at the time that the image was captured.

20. The computer program product of claim 16 , wherein the three-dimensional model is created based on data extracted from the image.

Assignees

• IBM

Inventors

• Dow Eli Michael
• Henderson Michael E
• Watson Campbell D
• Auger Guillaume

Classifications

• G06V20/647

  by matching two-dimensional images to three-dimensional objects · CPC title

• G06V10/761

  Proximity, similarity or dissimilarity measures · CPC title

• G06V20/13

  Satellite images · CPC title

• G06T7/60 Primary

  Analysis of geometric attributes · CPC title

• G06F18/22

  Matching criteria, e.g. proximity measures · CPC title