Automatic Annotation for Vehicle Damage

Patent metadata

Field	Value
Publication number	US-2025200101-A1
Application number	US-202519069624-A
Country	US
Kind code	A1
Filing date	Mar 4, 2025
Priority date	Feb 10, 2020
Publication date	Jun 19, 2025
Grant date	—

Abstract

Aspects described herein may allow an automated generation of an interactive multimedia content with annotations showing vehicle damage. In one method, a server may receive vehicle-specific identifying information of a vehicle. Image sensors may capture multimedia content showing aspects associated with exterior regions of the vehicle, and may send the multimedia content to the server. For each of the exterior regions of the vehicle, the server may determine, using a trained classification model, instances of damage. Furthermore, the server may generate an interactive multimedia content that shows images with annotations indicating instances of damage. The interactive multimedia content may be displayed via a user interface.

First claim

1 . A method comprising:
receiving, from one or more sensors, one or more first images showing one or more exterior aspects of a vehicle and vehicle-specific identifying information corresponding to the vehicle; receiving one or more second images showing one or more interior aspects of the vehicle; determining, using a machine learning model and based on the one or more first images and based on the one or more second images, one or more instances of damage to the vehicle and an extent of damage at each of the one or more instances of damage; generating, based on the one or more instances of damage and based on the extent of damage, at least one annotation indicating the one or more instances of damage and the extent of damage at each of the one or more instances of damage; generating, based on the one or more first images and based on the one or more second images and based on the at least one annotation, an interactive multimedia content associated with the vehicle; and causing, through a user interface and responsive to a request, display of the interactive multimedia content of the vehicle.

2 . The method of claim 1 , wherein the interactive multimedia content comprises an interior view of the vehicle and an exterior view of the vehicle.

3 . The method of claim 1 , wherein the one or more first images and the one or more second images are received from a staging system comprising one or more sensors and one or more light sources.

4 . The method of claim 1 , wherein the interactive multimedia content comprises a 360-degree image of the vehicle and a rotation feature such that a user may rotate the 360-degree image horizontally and vertically.

5 . The method of claim 4 , wherein the 360-degree image of the vehicle comprises a 360-degree image of an interior view of the vehicle and a 360-degree image of an exterior view of the vehicle.

6 . The method of claim 1 , further comprising:
generating, using the machine learning model and based on the one or more instances of damage and based on the extent of damage, a cost to repair each of the one or more instances of damage; and causing, based on generating the cost, display of the cost alongside the at least one annotation indicating the one or more instances of damage and the extent of damage at each of the one or more instances of damage.

7 . The method of claim 1 , wherein the one or more first images and the one or more second images are received from a mobile device.

8 . A device comprising:
one or more processors; and memory storing instructions that, when executed by the one or more processors, cause the device to:
receive vehicle-specific identifying information corresponding to the vehicle;
determine, based on one or more first images received from one or more sensors, one or more first images corresponding to one or more exterior aspects of the vehicle;
determine, based on one or more second images, one or more second images corresponding to one or more interior aspects of the vehicle;
determine, using a machine learning model and based on the one or more first images and based on the one or more second images, one or more instances of damage to the vehicle and an extent of damage at each of the one or more instances of damage;
generate, based on the one or more instances of damage and based on the extent of damage, at least one annotation indicating the one or more instances of damage and the extent of damage at each of the one or more instances of damage;
generate, based on the one or more first images and based on the one or more second images and based on the at least one annotation, an interactive multimedia content associated with the vehicle; and
cause, through a user interface and responsive to a request, display of the interactive multimedia content of the vehicle.

9 . The device of claim 8 , wherein the interactive multimedia content comprises an interior view of the vehicle and an exterior view of the vehicle.

10 . The device of claim 8 , wherein the one or more first images and the one or more second images are received from a staging system comprising one or more sensors and one or more light sources.

11 . The device of claim 8 , wherein the interactive multimedia content comprises a 360-degree image of the vehicle and a rotation feature such that a user may rotate the 360-degree image horizontally and vertically.

12 . The device of claim 11 , wherein the 360-degree image of the vehicle comprises a 360-degree image of an interior view of the vehicle and a 360-degree image of an exterior view of the vehicle.

13 . The device of claim 8 , wherein the instructions, when executed by the one or more processors, cause the device to:
generate, based on the one or more instances of damage and the extent of damage, a cost to repair each of the one or more instances of damage; and cause, based on generating the cost, display of the cost alongside the at least one annotation indicating the one or more instances of damage and the extent of damage at each of the one or more instances of damage.

14 . The device of claim 8 , wherein the one or more first images and the one or more second images are received from a mobile device.

15 . A non-transitory computer-readable medium storing instructions that, when executed, cause a server to:
receive, from one or more sensors, one or more images showing one or more aspects of a vehicle and vehicle-specific identifying information corresponding to the vehicle; determine, based on the one or more images showing one or more aspects of a vehicle, one or more first images corresponding to one or more exterior aspects of the vehicle and one or more second images corresponding to one or more interior aspects of the vehicle; determine, using a machine learning model and based on the one or more first images and based on the one or more second images, one or more instances of damage to the vehicle and an extent of damage at each of the one or more instances of damage; generate, based on the one or more instances of damage and based on the extent of damage, at least one annotation indicating the one or more instances of damage and the extent of damage at each of the one or more instances of damage; generate, based on the one or more first images and based on the one or more second images and based on the at least one annotation, an interactive multimedia content associated with the vehicle; and cause, through a user interface and responsive to a request, display of the interactive multimedia content of the vehicle.

16 . The non-transitory computer-readable medium of claim 15 , wherein the interactive multimedia content comprises an interior view of the vehicle and an exterior view of the vehicle.

17 . The non-transitory computer-readable medium of claim 15 , wherein the one or more images are received from a staging system comprising one or more sensors and one or more light sources.

18 . The non-transitory computer-readable medium of claim 15 , wherein the interactive multimedia content comprises a 360-degree image of the vehicle and a rotation feature such that a user may rotate the 360-degree image horizontally and vertically.

19 . The non-transitory computer-readable medium of claim 15 , wherein the instructions, when executed, cause the server to:
generate, using the machine learning model and based on the one or more instances of damage and based on the extent of damage, a cost to repair each of the one or more instances of damage; and cause, based on generating the cost, display of the cost a

Assignees

• Capital One Services Llc

Inventors

• Ghamsari Avid
• Tang Qiaochu
• Dagley Geoffrey
• Price Micah

Classifications

• G06V10/764

  using classification, e.g. of video objects · CPC title

• G06F18/2413

  based on distances to training or reference patterns · CPC title

• G06F16/48

  Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually · CPC title

• G06N20/00

  Machine learning · CPC title

• G06F16/45

  Clustering; Classification · CPC title