Assessing damages on vehicles

Patent metadata

Field	Value
Publication number	US-12033218-B2
Application number	US-202217589453-A
Country	US
Kind code	B2
Filing date	Jan 31, 2022
Priority date	Jan 31, 2022
Publication date	Jul 9, 2024
Grant date	Jul 9, 2024

Abstract

Methods, devices, apparatus, systems and computer-readable storage media for assessing damages on vehicles are provided. In one aspect, a method includes: accessing an image of a vehicle showing shapes each indicating at least one damage area on the vehicle; providing the image as input to a first model to identify the shapes in the image and obtaining shape data describing a position of each shape identified in the image; providing the image as input to a second model to identify one or more panels of the vehicle and obtaining panel data describing a position of each panel identified in the image; automatically correlating the one or more shapes and the one or more panels based on the shape data and the panel data to determine a number of shapes present on each panel; and generating a damage assessment report describing the number of shapes present on each panel.

First claim

What is claimed is:

1. A computer-implemented method for assessing damages on a vehicle, the computer-implemented method comprising:
accessing an image of at least one section of the vehicle, the image showing a plurality of shapes that each have been applied to indicate at least one damage area present in the at least one section of the vehicle;
providing the image as input to a first model that has been trained, through a first machine learning algorithm, to identify the plurality of shapes in the image and, in response, obtaining shape data that is generated by the first model, the shape data describing a position of each of one or more shapes identified in the image;
providing the image as input to a second model that has been trained, through a second machine learning algorithm, to identify one or more panels of the vehicle that are present in the at least one section shown in the image, and in response, obtaining panel data that is generated by the second model, the panel data describing a position of each of the one or more panels identified in the image;
automatically correlating the one or more shapes and the one or more panels based on the shape data with the panel data to determine, for each of the one or more panels of the vehicle, a number of shapes that are present on the panel; and
generating a damage assessment report that describes, for each of the one or more panels of the vehicle, the number of shapes that are present on the panel.

2. The computer-implemented method of claim 1 , wherein the first model comprises at least one of:
You Only Look Once (YOLO), single-shot detector (SSD), Faster Region-based Convolutional Neural Network (Faster R-CNN), or a computer vision algorithm.

3. The computer-implemented method of claim 1 , wherein the plurality of shapes have at least two different shape types, each of the different shape types corresponding to a different damage category, and
wherein the shape data comprises a corresponding shape type for each of the one or more shapes identified in the image.

4. The computer-implemented method of claim 3 , comprising:
for each of the one or more panels,
classifying one or more identified shapes correlated with the panel according to one or more corresponding shape types for the one or more identified shapes; and
for each of the one or more corresponding shape types, counting a respective number of identified shapes that are correlated with the panel and have a same corresponding shape type.

5. The computer-implemented method of claim 4 , wherein generating the damage assessment report comprises:
generating shape-damage correlation data based on the one or more panels, the one or more corresponding shape types for each of the one or more panels, and the respective number for each of the one or more corresponding shape types.

6. The computer-implemented method of claim 5 , wherein generating the damage assessment report comprises:
accessing estimate cost data for damage repair that is associated with at least one of damage categories, a number of damage areas in a same damage category, different panels, or vehicle models; and
generating the damage assessment report based on the shape-damage correlation data and the estimated cost data for damage repair.

7. The computer-implemented method of claim 6 , comprising:
providing the damage assessment report to at least one of a repair shop representative or a vehicle insurance company representative.

8. The computer-implemented method of claim 3 , wherein the shape data comprises: for each of the one or more shapes, a corresponding label for the corresponding shape type of the shape, and
wherein the computer-implemented method comprises:
for each of the one or more panels, counting, based on the corresponding labels, a number of identified shapes that are correlated with the panel and have a same corresponding shape type.

9. The computer-implemented method of claim 1 , wherein the first model is trained to process the image to enclose a corresponding bounding box for each of the one or more shapes identified in the image and to determine the position of the shape based on a position of the corresponding bounding box, and
wherein automatically correlating the one or more shapes and the one or more panels based on the shape data with the panel data comprises:
correlating each of the one or more shapes with a respective panel of the one or more panels based on the position of the shape and the position of the respective panel.

10. The computer-implemented method of claim 1 , wherein the second model is trained to process the image to segment the at least one section of the vehicle into the one or more panels by
masking one or more segments of the image, and
isolating the masked one or more segments of the image as the one or more panels, each of the masked one or more segments being associated with a corresponding one of the one or more panels.

11. The computer-implemented method of claim 10 , wherein automatically correlating the one or more shapes and the one or more panels based on the shape data with the panel data comprises:
correlating each of the one or more shapes with a respective panel of the one or more panels based on the position of the shape and a masked segment associated with the respective panel.

12. The computer-implemented method of claim 1 , wherein the second model comprises at least one of:
masked R-CNN, thresholding segmentation, edge-Based segmentation, region-based segmentation, watershed segmentation, or clustering-based segmentation.

13. The computer-implemented method of claim 1 , comprising:
receiving the image from a remote communication device configured to capture images of the vehicle.

14. The computer-implemented method of claim 1 , comprising:
providing an initial image as input to a third model that has been trained, through a third machine learning algorithm, to reduce surface glare of the vehicle in the image and, in response, obtaining the image that is generated by the third model.

15. The computer-implemented method of claim 1 , comprising:
generating the image based on multiple sectional images of the vehicle, each of the multiple sectional images being associated with a different corresponding section of the vehicle.

16. The computer-implemented method of claim 1 , comprising:
generating the image based on at least one frame of a video stream for the at least one section of the vehicle.

17. The computer-implemented method of claim 1 , comprising:
displaying an instruction on a display for capturing a sectional image of a section of the vehicle; and
in response to obtaining a captured sectional image of the section of the vehicle, determining whether the captured sectional image reaches an image criteria for the section of the vehicle.

18. The computer-implemented method of claim 17 , wherein determining whether the captured sectional image reaches the image criteria for the section of the vehicle comprises:
processing the captured sectional image to detect information of glare or bright spot on the section of the vehicle; and
determining whether the detected information of glare or bright spot is below a predetermined threshold.

19. The computer-implemented method of claim 17 , comprising one of:
in response to determining that the captured sectional image fails to reach the image criteria for the section of the vehicle, displaying an indication on the display for retaking the sectional image of the section of the vehicle, or
in response to deter

Assignees

• Vehicle Service Group Llc

Inventors

• Dev Bodhayan
• Patnala Sreedhar
• Johannes Michael
• Swaroop Prem

Classifications

• G06Q40/08 Primary

  Insurance · CPC title

• G06V10/225

  based on a marking or identifier characterising the area · CPC title

• G06V10/255

  Detecting or recognising potential candidate objects based on visual cues, e.g. shapes · CPC title

• G06T2207/30156

  Vehicle coating · CPC title

• G06T2207/20084

  Artificial neural networks [ANN] · CPC title