Guided vehicle capture for virtual model generation

Patent metadata

Field	Value
Publication number	US-10699404-B1
Application number	US-201715820519-A
Country	US
Kind code	B1
Filing date	Nov 22, 2017
Priority date	Nov 22, 2017
Publication date	Jun 30, 2020
Grant date	Jun 30, 2020

Abstract

Methods and system for guiding user data capture during a scan of a vehicle using a mobile device are disclosed. A user may scan a vehicle using a camera or other sensors of the mobile device to capture data from which a three-dimensional virtual model may be generated. During the scanning process, models may be generated and evaluated according to quality metrics. Visual cues may be determined and presented to the user during scanning to indicate areas sufficiently scanned or areas requiring additional scanning to meet data quality requirements for model generation. Damage to vehicle components may be identified by analysis of the generated model, and addition data capture or user annotation entry may be directed based upon the identified damage.

First claim

What is claimed is:

1. A computer-implemented method for guiding a user to capture data for virtual model generation, comprising:
receiving, at one or more processors of a mobile device, an indication of a vehicle type of a vehicle to model;
presenting, via a display of the mobile device and based at least in part on the vehicle type, an instruction to the user to capture a plurality of data points associated with a portion of the vehicle;
causing one or more sensors of the mobile device to capture the plurality of data points, the plurality of data points indicating positions on a surface of the portion of the vehicle;
generating, by the one or more processors of the mobile device and based at least in part of the vehicle type, a virtual model of the portion of the vehicle in a three-dimensional virtual space based upon the plurality of data points;
presenting, via the display of the mobile device, a visual representation of the virtual model together with instructions to capture additional further data points based upon the visual representation to the user, the visual representation indicating: (i) first areas of the surface of the vehicle for which sufficient data points for the virtual model generation have been captured and (ii) second areas of the surface of the vehicle for which sufficient data points for the virtual model generation have not been captured;
causing the one or more sensors of the mobile device to capture the additional data points for the second areas;
determining, by the one or more processors of the mobile device, completion of data capture for the portion of the vehicle; and
storing, in a memory of the mobile device, the virtual model.

2. The computer-implemented method of claim 1 , wherein determining the completion of data capture for the portion of the vehicle comprises:
generating, by the one or more processors of the mobile device, a quality metric of the virtual model; and
determining, by the one or more processors of the mobile device, the quality metric meets a corresponding quality threshold.

3. The computer-implemented method of claim 2 , wherein generating the quality metric includes determining an estimate of error for at least a section of the virtual model based upon the plurality of data points.

4. The computer-implemented method of claim 1 , wherein determining the completion of data capture for the portion of the vehicle comprises:
receiving, at the one or more processors of the mobile device, a completion indication from the user;
presenting, via the display of the mobile device, information regarding the virtual model of the portion of the vehicle; and
determining, by the one or more processors of the mobile device, the completion of data capture for the portion of the vehicle based at least in part on a user response to the information.

5. The computer-implemented method of claim 1 , further comprising:
identifying, by the one or more processors of the mobile device, a section of the portion of the vehicle to be recaptured;
presenting, via the display of the mobile device, an additional instruction to the user to capture other additional data points associated with the section; and
updating, by the one or more processors of the mobile device, the virtual model based upon the other additional data points.

6. The computer-implemented method of claim 1 , further comprising:
presenting, via the display of the mobile device, a plurality of scanning options associated with vehicle components,
wherein receiving the indication from the user includes a selection by the user of one of the plurality of scanning options associated with the portion of the vehicle.

7. The computer-implemented method of claim 1 , further comprising:
receiving, at the one or more processors of the mobile device, a user annotation associated with the portion of the vehicle, wherein the user annotation includes an additional indication of a location within the virtual model;
transmitting, to a server via a communication network, information associated with the virtual model and the user annotation for insurance claim processing;
identifying, by the one or more processors of the mobile device or by one or more processors of the server, a component of the vehicle associated with the user annotation based upon the location within the virtual model; and
determining, by the one or more processors of the mobile device or by the one or more processors of the server, a damage estimate based upon the user annotation and the component of the vehicle for insurance claim processing.

8. The computer-implemented method of claim 1 , further comprising:
identifying, by the one or more processors of the mobile device or by one or more processors of a remote server, one or more abnormalities within the virtual model based upon computational geometric analysis of one or more virtual surfaces of the virtual model;
determining, by the one or more processors of the mobile device or by the one or more processors of the remote server, a damaged section of the portion of the vehicle based upon the one or more abnormalities; and
presenting, via the display of the mobile device, a prompt to the user to enter a user annotation associated with the damaged section.

9. The computer-implemented method of claim 8 , further comprising:
receiving, at the one or more processors of the mobile device or by the one or more processors of the remote server, the user annotation entered by the user;
identifying, by the one or more processors of the mobile device or by the one or more processors of the remote server, a component of the vehicle associated with the damaged section; and
determining, by the one or more processors of the mobile device or by the one or more processors of the remote server, a damage estimate based upon the user annotation and the component of the vehicle for insurance claim processing.

10. The computer-implemented method of claim 1 , wherein:
presenting the instruction to the user to capture the plurality of data points associated with the portion of the vehicle further comprises presenting a plurality of visual cues indicating areas of the portion of the vehicle to capture;
capturing the plurality of data points associated with the portion of the vehicle further comprises receiving user indications of completion of data capture corresponding to the plurality of visual cues; and
determining the completion of data capture for the portion of the vehicle comprises determining that the user indications of completion have been received for each of the plurality of visual cues.

11. A computer system for guiding a user to capture data for virtual model generation, comprising:
one or more processors;
one or more sensors communicatively connected to the one or more processors;
a display communicatively connected to the one or more processors; and
a non-transitory program memory communicatively coupled to the one or more processors and storing computer-readable instructions that, when executed by the one or more processors, cause the computer system to:
receive an indication of a vehicle type of a vehicle to model;
present, based at least in part on the vehicle type, an instruction to the user to capture a plurality of data points associated with a portion of the vehicle via the display;
cause the plurality of data points to be captured, the plurality of data points indicating positions on a surface of the portion of the vehicle using the one or more sensors;
generate, based on the plurality of data points and the vehicle type, a virtual model of the portion of the vehicle in a three-dimensional virtual space;
present, via the display, a visual representation

Assignees

• State Farm Mutual Automobile Insurance Co

Inventors

• Nussbaum Bryan R
• Little Rebecca
• Mitchell Kevin
• Summers Nathan C
• Ho An

Classifications

• G06T2200/08

  involving all processing steps from image acquisition to 3D model generation · CPC title

• G06T7/50

  Depth or shape recovery · CPC title

• G06Q40/08 Primary

  Insurance · CPC title

• G06T2207/30252

  Vehicle exterior; Vicinity of vehicle · CPC title

• G06T2207/10028

  Range image; Depth image; 3D point clouds · CPC title