Guided vehicle capture for virtual mode generation

Patent metadata

Field	Value
Publication number	US-12299871-B2
Application number	US-202418594331-A
Country	US
Kind code	B2
Filing date	Mar 4, 2024
Priority date	Nov 22, 2017
Publication date	May 13, 2025
Grant date	May 13, 2025

Abstract

Methods and system for guiding user data capture during a scan of a vehicle using a mobile device are disclosed. A user may scan a vehicle using a camera or other sensors of the mobile device to capture data from which a three-dimensional virtual model may be generated. During the scanning process, models may be generated and evaluated according to quality metrics. Visual cues may be determined and presented to the user during scanning to indicate areas sufficiently scanned or areas requiring additional scanning to meet data quality requirements for model generation. Damage to vehicle components may be identified by analysis of the generated model, and addition data capture or user annotation entry may be directed based upon the identified damage.

First claim

What is claimed is:

1. A method, comprising:
causing a computing device to capture first data points associated with a surface of an object represented in a first virtual model;
generating, based at least in part on the first data points, a second virtual model in which at least a portion of the object, including the surface, is represented;
identifying, based at least in part on the first data points and using the second virtual model, an area of the surface for which additional data is needed;
causing the computing device to capture second data points associated with the area of the surface;
updating the second virtual model based on the second data points; and
determining, based on the representation of the surface in the first virtual model and on the representation of the surface in the second virtual model, a damaged section of the object, the damaged section including the surface.

2. The method of claim 1 , further comprising:
presenting, via a display of the computing device and based at least in part on a configuration of the surface, a plurality of scanning options for capturing the area of the surface; and
receiving, from the computing device, a selection from the plurality of scanning options.

3. The method of claim 2 , wherein the scanning options include a wavelength band in which to scan, the wavelength band being selected based at least in part on a color of the damaged section.

4. The method of claim 1 , further comprising:
generating instructions to capture third data points associated with the area of the surface,
wherein the instructions comprises at least one of:
a visual indication of the surface to be recaptured;
an indication of a viewing angle from which the third data points are to be captured;
an indication of a distance from which the third data points are to be captured; or
a movement sequence for capturing the third data points.

5. The method of claim 1 , wherein the object is a vehicle, and the second virtual model is generated based at least in part on a type of the vehicle.

6. The method of claim 5 , further comprising:
identifying a component of the vehicle associated with the damaged section;
receiving an annotation related to the component; and
determining a damage estimate based at least in part on the annotation and the component of the vehicle.

7. The method of claim 5 , further comprising:
receiving telematics data captured by sensors associated with the vehicle,
wherein the damaged section is determined based at least in part on telematics data.

8. The method of claim 1 , further comprising:
determining an estimate of error for at least a section of the second virtual model including the surface,
wherein the area of the surface for which additional data is needed is identified based on the estimate of error.

9. The method of claim 8 , further comprising:
presenting, via a display of the computing device, a visual representation of the second virtual model indicating:
first areas of the surface for which the estimate of error is less than or equal to a threshold, and
second areas of the surface for which the estimate of error is greater than the threshold.

10. The method of claim 1 , further comprising:
generating a quality metric indicating an accuracy of the second virtual model;
determining that the quality metric satisfies a quality threshold;
determining, based at least in part on the quality metric satisfying the quality threshold, that capture of data points is complete; and
presenting, via a display of the computing device, information indicating that capture of the data points is complete.

11. A system, comprising:
a processor;
a sensor communicatively connected to the processor; and
a non-transitory memory communicatively coupled to the processor and storing executable instructions that, when executed by the processor, cause the processor to perform operations comprising:
causing the sensor to capture first data points associated with a surface of an object represented in a first virtual model;
generating, based at least in part on the first data points, a second virtual model in which at least a portion of the object including the surface is represented;
identifying, based at least in part on the first data points and using the second virtual model, an area of the surface for which additional data is needed;
causing the sensor to capture second data points associated with the area of the surface;
updating the second virtual model based on the second data points; and
determining, based on the representations of the surface in the first virtual model and on the representation of the surface in the second virtual model, a damaged section of the object, the damaged section including the surface.

12. The system of claim 11 , wherein:
the processor and the non-transitory memory are disposed within a mobile device,
the sensor comprises a camera of the mobile device, and
the system comprises one or more additional sensors communicatively connected to the mobile device.

13. The system of claim 12 , the operations further comprising:
presenting, via a display of the mobile device, a prompt associated with the damaged section,
wherein the prompt is indicative of the damaged section to be recaptured or an annotation to be entered.

14. The system of claim 11 , the operations further comprising:
identifying a component of the object associated with the damaged section;
receiving an annotation related to the component; and
determining a damage estimate based upon the annotation and the component.

15. The system of claim 11 , the operations further comprising:
determining an estimate of error for at least a section of the second virtual model including the surface,
wherein the area of the surface for which additional data is needed is identified based on the estimate of error.

16. The system of claim 11 , wherein the first data points are captured at a first resolution, and the second data points are captured at a second resolution higher than the first resolution.

17. A tangible, non-transitory computer-readable medium storing instructions that, when executed by a processor, causes the processor to:
cause a computing device to capture first data points associated with a surface of an object represented in a first virtual model;
generate, based at least in part on the first data points, a second virtual model in which at least a portion of the object including the surface is represented;
identify, based at least in part on the first data points and using the second virtual model, an area of the surface for which additional data is needed;
cause the computing device to capture second data points associated with the area of the surface;
update the second virtual model based on the second data points; and
determine, based on the representations of the surface in the first virtual model and on the representation of the surface in the second virtual model, a damaged section of the object, the damaged section including the surface.

18. The tangible, non-transitory computer-readable medium of claim 17 , the instructions further causing the processor to:
generate instructions to capture third data points associated with the area of the surface, wherein the instructions comprises at least one of:
a visual indication of the surface to be recaptured;
an indication of a viewing angle from which the third data points are to be captured;
an indication of a distance from which the third data points are to be captured; or
a movement sequen

Assignees

• State Farm Mutual Automobile Insurance Co

Inventors

• Nussbaum Bryan R
• Little Rebecca A
• Mitchell Kevin L
• Summers Nathan C
• Ho An

Classifications

• G06Q40/08 Primary

  Insurance · CPC title

• G06T2207/30252

  Vehicle exterior; Vicinity of vehicle · CPC title

• G06T2207/10028

  Range image; Depth image; 3D point clouds · CPC title

• G06T2200/08

  involving all processing steps from image acquisition to 3D model generation · CPC title

• G06T7/50

  Depth or shape recovery · CPC title