Advanced cloud detection using machine learning and optimization techniques

Patent metadata

Field	Value
Publication number	US-10083354-B2
Application number	US-201615362254-A
Country	US
Kind code	B2
Filing date	Nov 28, 2016
Priority date	Nov 28, 2016
Publication date	Sep 25, 2018
Grant date	Sep 25, 2018

Abstract

Techniques for automatically determining, on a pixel by pixel basis, whether imagery includes ground images or is obscured by cloud cover. The techniques include training a cloud dictionary and a ground dictionary, determining whether a given pixel is best represented by “words” from the cloud dictionary or “words” from the ground dictionary to make an initial determination of cloud or ground, and performing a max-flow, min-cut operation on the image to determine whether each pixel is a cloud or ground imagery.

First claim

What is claimed is:

1. A computer-implemented process for determining whether given imagery in an overhead image is cloud imagery or ground imagery, comprising:
for multiple portions of an image, making an initial determination about whether each of the multiple portions primarily contains cloud imagery or primarily contains ground imagery by:
with a processor, utilizing words from a cloud dictionary of picture elements to describe a first portion of the image;
with a processor, utilizing words from a ground dictionary of picture elements to describe the first portion of the image;
with a processor, determining whether the cloud dictionary words or the ground dictionary words best describe the first portion of the image; and

with a processor, performing an optimization technique on the multiple portions of the overhead image using the initial determination to determine which portions of the overhead image include cloud imagery or ground imagery.

2. A computer-implemented process as defined in claim 1 , wherein the optimization technique includes identifying adjacent pixels and calculating a capacity between the identified adjacent pixels.

3. A computer-implemented process as defined in claim 2 , wherein the optimization technique further includes creating a score for each pixel to represent the likelihood that the pixel does or does not contain a cloud and creating a grid-graph of the scores of the pixels with adjacency information associated with each set of adjacent pixels.

4. A computer-implemented process as defined in claim 3 , wherein the optimization technique further includes connecting the pixels of the grid-graph to both a source and a sink using the pixel score as the capacity, wherein one of the source and the sink represents cloud and one represents ground, and performing a min-cut/max-flow segmentation on the image.

5. A computer-implemented process as defined in claim 1 , wherein the optimization technique includes applying a window, having a height and width that are less than a height and width of the image and that are the same as that of the cloud dictionary and ground dictionary words, to various portions of the image, the various portions partially overlapping adjacent portions, in order to determine if each portion most likely contains cloud imagery or ground imagery, and incrementing or decrementing a score for each pixel in the portion based on whether the determination was of cloud imagery or ground imagery, respectively.

6. A computer-implemented process as defined in claim 1 , wherein the overhead image is a satellite-based image.

7. A computer-implemented process as defined in claim 1 , further including adding to metadata associated with each pixel an indication of whether each such pixel includes cloud imagery.

8. A computer-implemented process as defined in claim 7 , further including using the indication of cloud imagery in the metadata to select pixels for an orthomosaic image free of clouds.

9. A computer-implemented process as defined in claim 1 , wherein the cloud dictionary includes multiple words that represent portions of cloud imagery and the ground dictionary includes multiple words that represent portions of ground imagery.

10. A computer-implemented process as defined in claim 1 , wherein the determining includes creating a first linear combination of the words from the cloud dictionary and a second linear combination of the words from the ground dictionary, and comparing the first and second linear combinations.

11. A computer-implemented process for determining whether given imagery in an overhead image is cloud imagery or ground imagery, comprising:
with a processor, utilizing words from a cloud dictionary of picture elements to describe a portion of an image;
with a processor, utilizing words from a ground dictionary of picture elements to describe the portion of the image; and
with a processor, determining whether the cloud dictionary words or the ground dictionary words best describe the portion of the image.

12. A computer-implemented process as defined in claim 11 , further including:
with a processor, performing an optimization technique on a plurality of portions of the overhead image including the portion of the overhead image to determine which portions of the overhead image include cloud imagery or ground imagery.

13. A computer-implemented process for determining whether given imagery in an overhead image is cloud imagery or ground imagery, comprising:
for multiple portions of an image, making an initial determination about whether each of the multiple portions primarily contains cloud imagery or primarily contains ground imagery by:
with a processor, utilizing words from a cloud dictionary of picture elements to describe a first portion of the image;
with a processor, utilizing words from a ground dictionary of picture elements to describe the first portion of the image; and
with a processor, determining whether the cloud dictionary words or the ground dictionary words best describe the first portion of the image;

applying a window, having a height and width that are less than a height and width of the image and that are the same as that of the cloud dictionary and ground dictionary words, to various portions of the image, the various portions partially overlapping adjacent portions, in order to determine if each portion most likely contains cloud imagery or ground imagery, and incrementing or decrementing a score for each pixel in the portion based on whether the determination was of cloud imagery or ground imagery, respectively;
identifying adjacent pixels and calculating a capacity between the identified adjacent pixels;
creating a score for each pixel to represent the likelihood that the pixel does or does not contain a cloud;
creating a grid-graph of the scores of the pixels with adjacency information associated with each set of adjacent pixels;
connecting the pixels of the grid-graph to both a source and a sink using the pixel score as the capacity, wherein one of the source and the sink represents cloud and one represents ground; and
performing a min-cut/max-flow segmentation on the image to define portions of the overhead image which are believed to include cloud imagery and portions of the overhead image which are believed to include ground imagery.

14. A computer-implemented process as defined in claim 13 , wherein the overhead image is a satellite-based image.

15. A computer-implemented process as defined in claim 13 , further including adding to metadata associated with each pixel an indication of whether each such pixel includes cloud imagery.

16. A computer-implemented process as defined in claim 15 , further including using the indication of cloud imagery in the metadata to select pixels for an orthomosaic image free of clouds.

17. A computer-implemented process as defined in claim 13 , wherein the cloud dictionary includes multiple words that represent portions of cloud imagery and the ground dictionary includes multiple words that represent portions of ground imagery.

18. A computer-implemented process as defined in claim 13 , wherein the determining includes creating a first linear combination of the words from the cloud dictionary and a second linear combination of the words from the ground dictionary, and comparing the first and second linear combinations.

Assignees

• Digitalglobe Inc

Inventors

• Aschenbeck Michael

Classifications

• G06T7/162 Primary

  involving graph-based methods · CPC title

• G06V10/7635

  based on graphs, e.g. graph cuts or spectral clustering · CPC title

• G06V10/84

  using probabilistic graphical models from image or video features, e.g. Markov models or Bayesian networks · CPC title

• G06V10/7715

  Feature extraction, e.g. by transforming the feature space, e.g. multi-dimensional scaling [MDS]; Mappings, e.g. subspace methods · CPC title

• G06V20/13

  Satellite images · CPC title