Guided vehicle capture for virtual model generation

Patent metadata

Field	Value
Publication number	US-11922618-B2
Application number	US-202217728092-A
Country	US
Kind code	B2
Filing date	Apr 25, 2022
Priority date	Nov 22, 2017
Publication date	Mar 5, 2024
Grant date	Mar 5, 2024

Abstract

Methods and system for guiding user data capture during a scan of a vehicle using a mobile device are disclosed. A user may scan a vehicle using a camera or other sensors of the mobile device to capture data from which a three-dimensional virtual model may be generated. During the scanning process, models may be generated and evaluated according to quality metrics. Visual cues may be determined and presented to the user during scanning to indicate areas sufficiently scanned or areas requiring additional scanning to meet data quality requirements for model generation. Damage to vehicle components may be identified by analysis of the generated model, and addition data capture or user annotation entry may be directed based upon the identified damage.

First claim

What is claimed is:

1. A computer-implemented method, comprising:
identifying, using an existing first virtual model in which a physical object is represented, a configuration of a surface of the object;
causing a computing device to capture first data points associated with the surface of the object, for which additional information is needed;
generating, based at least in part on the first data points, a second virtual model in which at least a portion of the object including the surface is represented;
identifying, based at least in part on the first data points and using the second virtual model, an area of the surface for which additional data is needed;
causing the computing device to capture second data points associated with the area of the surface;
updating the second virtual model based on the second data points;
determining, based on the representations of the surface in the first virtual model and in the second virtual model, an abnormality associated with the second virtual model; and
determining, based on the abnormality, a damaged section of the object, the damaged section including the surface.

2. The computer-implemented method of claim 1 , further comprising:
generating a quality metric indicating an accuracy of the second virtual model;
determining that the quality metric satisfies a quality threshold;
determining, based at least in part on the quality metric satisfying the quality threshold, that capture of data points is complete; and
presenting, via a display of the computing device, information indicating the completion.

3. The computer-implemented method of claim 1 , further comprising:
presenting, via a display of the computing device, a prompt associated with the damaged section,
wherein the prompt is indicative of the damaged section to be recaptured or an annotation to be entered.

4. The computer-implemented method of claim 3 , further comprising:
receiving, from the computing device, an annotation in response to the prompt;
identifying, based on the annotation and the abnormality, a component of the object associated with the damaged section; and
determining, a damage estimate based at least in part on the annotation and the identified component.

5. The computer-implemented method of claim 1 , further comprising:
presenting, via a display of the computing device and based at least in part on the damaged section, a plurality of scanning options for recapturing the damaged section,
wherein the scanning options include two or three-dimensional scanning, and a wavelength band in which to scan.

6. The computer-implemented method of claim 5 , wherein the wavelength band in which to scan is selected based at least in part on a color of the damaged section.

7. The computer-implemented method of claim 1 , wherein the object is a vehicle, and the damaged section is determined based at least in part on telematics data captured by sensors associated with the vehicle.

8. The computer-implemented method of claim 1 , further comprising:
identifying, based at least in part on the second virtual model, a surface of the object to be recaptured;
generating instructions to capture third data points associated with the surface; and
updating the second virtual model based on the third data points.

9. The computer-implemented method of claim 8 , wherein generating the instructions to capture the third data points comprises at least one of:
generating a visual indication of the surface to be recaptured;
generating an indication of a viewing angle from which the third data points are to be captured;
generating an indication of a distance from which the third data points are to be captured; or
generating a movement sequence for capturing the third data.

10. The computer-implemented method of claim 8 , wherein the first data points are captured at a first resolution, and the third data points are captured at a second resolution higher than the first resolution.

11. The computer-implemented method of claim 8 , wherein the object is a vehicle, and identifying the surface to be recaptured further comprises:
receiving telematics data captured by sensors associated with the vehicle;
determining, based at least in part on the telematics data, a likelihood that the surface is damaged; and
identifying the surface to be recaptured based on the likelihood exceeding a threshold value.

12. A system, comprising:
a processor;
a sensor communicatively connected to the processor; and
a non-transitory memory communicatively coupled to the processor and storing executable instructions that, when executed by the processor, cause system to:
identify, using an existing first virtual model representing a physical object, a configuration of a surface of the object;
cause the sensor to capture first data points associated with the surface of the object, for which additional information is needed;
generate, based on the first data points, a second virtual model representing at least a portion of the object including the surface;
identify, based at least in part on the first data points and using the second virtual model, an area of the surface for which additional data is needed;
cause the sensor to capture second data points associated with the area of the surface;
update the second virtual model based on the second data points;
determine, based on the representations of the surface in the first virtual model and in the second virtual model, an abnormality associated with the second virtual model; and
determining, based on the abnormality, a damaged section of the object, the damaged section including the surface.

13. The system of claim 12 , further comprising a display communicatively connected to the processor, wherein the instructions further cause the processor to:
generate instructions indicating position of the sensor to capture the second data points; and
present, via the display, a visual representation of the second virtual model together with the instructions.

14. The system of claim 13 , wherein the sensor comprises a camera of a mobile computing device, the mobile computing device including the display and the non-transitory memory.

15. The system of claim 12 , wherein the instructions, when executed, further cause the system to:
determine, based on a difference between the representation of the surface in the first virtual model and the second virtual model, a likelihood that the surface is damaged; and
determine, based at least in part of the likelihood exceeding a threshold value, a damage estimate corresponding to a component of the object associated with the surface.

16. The system of claim 12 , wherein the sensor is a first sensor configured to capture radiation in a first wavelength band, and the system further comprises a second sensor configured to capture radiation in a second wavelength band different from the first wavelength band, and wherein the instructions, when executed, further cause the system to:
identify, based at least in part on the second virtual model, a section of the object to be recaptured;
cause the second sensor to capture third data points associated with the section; and
update the second virtual model based on the third data points.

17. A tangible, non-transitory computer-readable medium storing instructions that, when executed by a processor of a system, cause the processor to:
identify, using an existing first virtual model in which a physical object is represented, a configuration of a surface of the object;
cause a sensor of a computing device to capture first data points

Assignees

• State Farm Mutual Automobile Insurance Co

Inventors

• Nussbaum Bryan R
• Little Rebecca A
• Mitchell Kevin L
• Summers Nathan C
• Ho An

Classifications

• G06T7/001 Primary

  using an image reference approach · CPC title

• G06Q40/08 Primary

  Insurance · CPC title

• G06T2207/10028

  Range image; Depth image; 3D point clouds · CPC title

• G06T2207/30252

  Vehicle exterior; Vicinity of vehicle · CPC title

• G06T7/50

  Depth or shape recovery · CPC title