Technologies for using image data analysis to assess and classify hail damage

Patent metadata

Field	Value
Publication number	US-12026786-B2
Application number	US-202318307254-A
Country	US
Kind code	B2
Filing date	Apr 26, 2023
Priority date	Oct 30, 2018
Publication date	Jul 2, 2024
Grant date	Jul 2, 2024

Abstract

Systems and methods for analyzing image data to assess property damage are disclosed. According to certain aspects, a server may analyze segmented digital image data of a roof of a property using a convolutional neural network (CNN). The server may extract a set of features from a set of regions output by the CNN. Additionally, the server may analyze the set of features using an additional image model to generate a set of outputs indicative of a confidence level that actual hail damage is depicted in the set of regions.

First claim

What is claimed is:

1. A computer-implemented method, comprising:
accessing a digital image;
identifying, by a processor and using a convolutional neural network (CNN), a region of potential damage depicted in the digital image;
identifying, by the processor, a color feature indicative of potential damage and illustrated within the region of potential damage by:
determining a first section of the digital image within the region of potential damage and a second section of the digital image outside of the region of potential damage; and
identifying the color feature within at least one of the first section or the second section based at least in part on a color skewness determined using a histogram that represents color statistics for colors depicted within the first section and the second section; and

generating, by the processor, using a classification model, and based at least in part on the color feature, an output indicating a presence of damage associated with the region of potential damage, wherein the output is used to automatically determine an estimated damage amount.

2. The computer-implemented method of claim 1 , wherein the color statistics represent one or more of a color mean value, a color skewness value, or a color variation value.

3. The computer-implemented method of claim 1 , further comprising:
training the CNN using training data comprising training images and training labels, wherein the trained CNN is configured to:
determine a classification for the digital image based at least in part on at least one of the training images or the training labels, and
generate an image depicting the region of potential damage based at least in part on the classification.

4. The computer-implemented method of claim 3 , wherein:
the training images comprise at least a first image that depicts damage and a second image that does not depict the damage; and
the training labels comprise data identifying a portion of the first image as a first region depicting the damage and identifying a remaining portion of the first image as a second region not depicting the damage, the remaining portion excluding the first region depicting the damage.

5. The computer-implemented method of claim 1 , wherein identifying the color feature comprises determining that the color feature is associated with a color variation that meets or exceeds a threshold value.

6. The computer-implemented method of claim 1 , wherein the digital image is determined based at least in part on a set of digital images using a sliding window technique.

7. The computer-implemented method of claim 1 , wherein the output comprises a set of binary outputs indicating whether damage is represented in the color feature.

8. The computer-implemented method of claim 1 , wherein generating the output using the classification model comprises:
inputting, by the processor, the color feature into the classification model; and
assigning a confidence level to the output based at least in part on a likelihood of the color feature indicating the presence of damage in the digital image.

9. A system, comprising:
a memory configured to store non-transitory computer executable instructions; and
a processor interfacing with the memory, and configured to execute the non-transitory computer executable instructions, wherein execution of the instructions causes the processor to:
access a digital image;
identify, using a convolutional neural network (CNN), a region of potential damage depicted in the digital image;
identify a color feature indicative of potential damage and illustrated within the region of potential damage by:
determining a first section of the digital image within the region of potential damage and a second section of the digital image outside of the region of potential damage; and
identifying the color feature within at least one of the first section and the second section based at least in part on color skewness determined using a histogram that represents color statistics for colors depicted within the first section and the second section; and

generate, using a classification model and based at least in part on the color feature, an output indicating a presence of damage associated with the region of potential damage, wherein the output is used to automatically determine an estimated damage amount.

10. The system of claim 9 , wherein the color statistics represent one or more of a color mean value, a color skewness value, or a color variation value.

11. The system of claim 9 , wherein to generate the output using the classification model, the processor is configured to:
input the color feature into the classification model to generate a set of binary outputs indicating whether damage is represented in the color feature.

12. The system of claim 9 , wherein identifying the color feature is based at least in part on determining that the color feature is associated with a color variation that meets or exceeds a threshold value.

13. The system of claim 9 , wherein the output is further based at least in part on a shape feature extracted from the digital image.

14. The system of claim 9 , wherein to generate the output using the classification model, the processor is configured to:
input the color feature into the classification model; and
generate a confidence level based on a likelihood of the color feature indicating the presence of damage in the digital image.

15. A non-transitory computer-readable storage medium configured to store instructions, the instructions when executed by a processor causing the processor to perform operations comprising:
accessing a digital image;
identifying, using a convolutional neural network (CNN), a region of potential damage depicted in the digital image;
identifying, a color feature indicative of potential damage and illustrated within the region of potential damage by:
determining a first section of the digital image within the region of potential damage and a second section of the digital image outside of the region of potential damage; and
identifying the color feature within at least one of the first section and the second section based at least in part on a color skewness determined using a histogram that represents color statistics for colors depicted within the first section and the second section; and

generating, using a classification model, and based at least in part on the color feature, an output indicating a presence of damage associated with the region of potential damage, wherein the output is used to automatically determine an estimated damage amount.

16. The non-transitory computer-readable storage medium of claim 15 , wherein the digital image is determined based at least in part on a set of digital images segmented using a sliding window technique.

17. The non-transitory computer-readable storage medium of claim 15 , wherein the output comprises a confidence level associated with the presence of damage in the digital image.

18. The non-transitory computer-readable storage medium of claim 15 , wherein the output is further based at least in part on a shape feature determined from the digital image.

19. The non-transitory computer-readable storage medium of claim 15 , wherein identifying the color feature is further based at least in part on determining that the color skewness meets or exceeds a threshold value.

20. The non-transitory computer-readable storage medium of claim 15 , wherein the output indicating the presence of damage comprises an indication of hail damage to

Assignees

• State Farm Mutual Automobile Insurance Co

Inventors

• Bokshi-Drotar Marigona
• Wan Jing
• Kane Sandra
• Li Yuntao

Classifications

• G06N3/09

  Supervised learning · CPC title

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06V10/26

  Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion · CPC title

• G06V10/82

  using neural networks · CPC title

• G06V10/54

  relating to texture · CPC title