Technologies for using image data analysis to assess and classify hail damage

Patent metadata

Field	Value
Publication number	US-10977490-B1
Application number	US-201816175126-A
Country	US
Kind code	B1
Filing date	Oct 30, 2018
Priority date	Oct 30, 2018
Publication date	Apr 13, 2021
Grant date	Apr 13, 2021

Abstract

Systems and methods for analyzing image data to assess property damage are disclosed. According to certain aspects, a server may analyze segmented digital image data of a roof of a property using a convolutional neural network (CNN). The server may extract a set of features from a set of regions output by the CNN. Additionally, the server may analyze the set of features using an additional image model to generate a set of outputs indicative of a confidence level that actual hail damage is depicted in the set of regions.

First claim

What is claimed is:

1. A computer-implemented method of analyzing image data to automatically assess hail damage to a property, the method comprising:
accessing digital image data depicting a roof of the property;
segmenting, by a processor, the digital image data into a set of digital images depicting portions of the roof of the property;
identifying, by the processor using a first trained model, a set of regions of potential hail damage from the set of digital images, the first trained model including a convolutional neural network (CNN) configured to:
classify first individual pixels in a digital image of the set of digital images as indicating potential hail damage, and a remainder of individual pixels in the digital image as indicating an absence of hail damage, and
generate an output set of images, the output set of images including a binary image depicting:
the first individual pixels indicating potential hail damage with a first color, and
the remainder of individual pixels with a second color different from the first color;

extracting, by the processor, a set of features from individual regions of the set of regions of potential hail damage, the set of features comprising a set of texture features indicating at least one of contrast, homogeneity, and entropy; and
analyzing, by the processor using a second trained model, the set of features to generate a set of outputs indicating a presence of hail damage in the output set of images, the second trained model including a classification model configured to classify the set of features from the individual regions as depicting hail damage.

2. The computer-implemented method of claim 1 , wherein segmenting the digital image data into the set of digital images comprises:
segmenting the digital image data into the set of digital images using a sliding window technique.

3. The computer-implemented method of claim 1 , further comprising:
training the convolutional neural network (CNN) using a set of training data comprising a set of training images and a set of training labels.

4. The computer-implemented method of claim 1 , wherein analyzing the set of features using the classification model comprises:
analyzing, by the processor, the set of features using the classification model to generate a set of binary outputs respectively indicating whether hail damage is present in the set of features.

5. The computer-implemented method of claim 1 , further comprising:
calculating, by the processor based on the set of outputs, an estimated damage amount to the roof of the property.

6. The computer-implemented method of claim 1 , wherein extracting the set of features from the individual regions of potential hail damage further comprises:
extracting, by the processor from the individual regions, at least one of a set of color features and a set of shape features, wherein a shape feature of the set of shape features includes at least one of an aspect ratio, an area, and a contour curvature.

7. The computer-implemented method of claim 1 , wherein analyzing the set of features using the classification model comprises:
inputting, by the processor, each of the set of features into the classification model; and
after inputting each of the set of features into the classification model, generating the set of outputs, each of which comprises a confidence level indicating the presence of hail damage in the set of digital images.

8. A system for analyzing image data to automatically assess hail damage to a property, comprising:
a memory configured to store non-transitory computer executable instructions; and
a processor interfacing with the memory, and configured to execute the non-transitory computer executable instructions to cause the processor to:
access digital image data depicting a roof of the property;
segment the digital image data into a set of digital images depicting a respective set of portions of the roof of the property;
identify, using a first trained model, a set of regions of potential hail damage from the set of digital images, the first trained model including a convolutional neural network (CNN) configured to:
classify first individual pixels in a digital image of the set of digital images as indicating potential hail damage, and a remainder of individual pixels in the digital image as indicating an absence of hail damage, and
generate an output set of images, the output set of images including a binary image depicting:
the first individual pixels indicating potential hail damage with a first color, and
the remainder of individual pixels with a second color different from the first color;

extract a set of features from individual regions of the set of regions of potential hail damage, the set of features comprising a set of texture features indicating at least one of contrast, homogeneity, and entropy; and
analyze, using a second trained model, the set of features to generate a set of outputs indicating a presence of hail damage in the output set of images, the second trained model including a classification model configured to classify the set of features from the individual regions as depicting hail damage.

9. The system of claim 8 , wherein the processor is further configured to:
train the convolutional neural network (CNN) using a set of training data comprising a set of training images and a set of training labels; and
store, in the memory, the CNN that was trained.

10. The system of claim 8 , wherein to analyze the set of features using the classification model, the processor is configured to:
analyze the set of features using the classification model to generate a set of binary outputs respectively indicating whether hail damage is present in the set of features.

11. The system of claim 8 , wherein the processor is further configured to:
calculate, based on the set of outputs, an estimated damage amount to the roof of the property.

12. The system of claim 8 , wherein to extract the set of features from the individual regions of potential hail damage, the processor is further configured to:
extract, from the individual regions, at least one of a set of color features and a set of shape features, wherein a shape feature of the set of shape features includes at least one of an aspect ratio, an area, and a contour curvature.

13. The system of claim 8 , wherein to analyze the set of features using the classification model, the processor is configured to:
input each of the set of features into the classification model, and
after inputting each of the set of features into the classification model, generate the set of outputs, each of which comprises a confidence level indicating the presence of hail damage in the set of digital images.

14. A non-transitory computer-readable storage medium configured to store instructions, the instructions when executed by a processor causing the processor to perform operations comprising:
accessing digital image data depicting a roof of a property;
segmenting the digital image data into a set of digital images depicting a respective set of portions of the roof of the property;
identifying, using a first trained model, a set of regions of potential hail damage from the set of digital images, the first trained model including a convolutional neural network (CNN) configured to:
classify first individual pixels in a digital image of the set of digital images as indicating potential hail damage, and a remainder of individual pixels in the digital image as indicating an absence of hail damage, and
generate an output set of images, the output set of images including a binary image depicting:
the firs

Assignees

• State Farm Mutual Automobile Insurance Co

Inventors

• Bokshi-Drotar Marigona
• Wan Jing
• Kane Sandra
• Li Yuntao

Classifications

• G06V20/10

  Terrestrial scenes (scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59) · CPC title

• G06V10/764

  using classification, e.g. of video objects · CPC title

• G06V20/176

  Urban or other man-made structures · CPC title

• G06Q40/08 Primary

  Insurance · CPC title

• G06N3/045

  Combinations of networks · CPC title