Vehicle component installation preview image generation

Patent metadata

Field	Value
Publication number	US-9928544-B1
Application number	US-201514644030-A
Country	US
Kind code	B1
Filing date	Mar 10, 2015
Priority date	Mar 10, 2015
Publication date	Mar 27, 2018
Grant date	Mar 27, 2018

Abstract

Systems and processes described herein can identify interface points for an item in an image and generate a preview image that illustrates one or more additional items interfacing with at least one of the identified interface points. The system described herein can use augmented reality to present the preview image of an additional item connected to the item at the at least one identified interface point. Further, the system described herein can use a three-dimensional model created based at least in part on measurements of the additional item and a determination of the size of the item from context or reference information within the image to confirm that the additional item will fit the interface point of the item within a threshold degree of fit.

First claim

What is claimed is:

1. A computer-implemented method comprising:
as implemented by one or more computing devices configured with specific computer-executable instructions,
accessing an image from a continuous stream of images that depicts at least a portion of a vehicle;
analyzing the image to determine spatial characteristics of the portion of the vehicle;
identifying a vehicle interface depicted in the image, the vehicle interface comprising a connection location at which one or more items can be connected to the vehicle;
annotating the image with an identification of a location of the vehicle interface to obtain an annotated image;
providing the annotated image to a user device for presentation to a user while the user device is capturing subsequent images from the continuous stream of images;
receiving a selection of an item for potential connection with the vehicle interface;
accessing a three-dimensional (3D) model for the item;
accessing a 3D model for the vehicle, wherein the 3D model for the vehicle represents at least the portion of the vehicle;
transforming the 3D model for the item and the 3D model for the vehicle based at least in part on the spatial characteristics of the portion of the vehicle;
creating a 3D image scene by positioning the 3D model for the item with respect to the 3D model for the vehicle such that an item interface of the item is aligned with respect to the vehicle interface;
identifying a second vehicle interface, wherein creating the 3D image scene further comprises positioning a second instance of the 3D model for the item with respect to the 3D model for the vehicle based at least in part on the second vehicle interface; and
generating a display image based at least in part on a rendering of the 3D image scene.

2. The computer-implemented method of claim 1 , wherein the vehicle interface is located between at least two components of the vehicle.

3. The computer-implemented method of claim 2 , wherein creating the 3D image scene further comprises removing a portion of the 3D model for the vehicle corresponding to a component from the at least two components.

4. The computer-implemented method of claim 1 , wherein the vehicle interface is identified based at least in part on the selection of the item.

5. The computer-implemented method of claim 1 , further comprising identifying a vehicle component included in the portion of the vehicle based at least in part on context information included in the image.

6. The computer-implemented method of claim 5 , wherein the spatial characteristics of the portion of the vehicle are determined based on the identified vehicle component and without using a reference marker.

7. The computer-implemented method of claim 5 , further comprising:
determining whether the vehicle component comprises an original component, wherein the original component comprises a component included by the manufacturer of the vehicle; and
in response to determining that the vehicle component is not an original component:
identifying the vehicle component; and
accessing a 3D model for the vehicle component, wherein creating the 3D image scene further comprises positioning the 3D model for the vehicle component with respect to the 3D model for the vehicle based at least in part on a location of a depiction of the vehicle component in the image.

8. The computer-implemented method of claim 7 , wherein the vehicle interface is on the vehicle component.

9. The computer-implemented method of claim 1 , wherein transforming the 3D model for the item and the 3D model for the vehicle comprises rotating, translating, and/or scaling the 3D model for the item and/or the 3D model for the vehicle.

10. The computer-implemented method of claim 1 , wherein vehicle interface dimensions of the vehicle interface are determined based on an identity of the vehicle, and wherein a set of items available for selection by a user are filtered based at least in part on the vehicle interface dimensions.

11. A system comprising:
an electronic data store configured to at least store specific computer-executable instructions; and
an interactive computing system comprising computer hardware in communication with the electronic data store, the interactive computing system configured to execute the specific compute-executable instructions to at least:
access an image that depicts at least a portion of a vehicle;
analyze the image to identify the vehicle;
receive a selection of an item for potential addition to the portion of the vehicle;
determine an item type of the selected item;
identify a plurality of vehicle connection interfaces based at least in part on the item type, wherein the item type is designed to attach to the vehicle at one or more of the plurality of vehicle connection interfaces;
access a three-dimensional (3D) model for the selected item;
access a 3D model for the vehicle;
scale the 3D model for the vehicle and the 3D model for the selected item based at least in part on the scale of the portion of the vehicle depicted in the image;
create an annotated image corresponding to the image, wherein a location of two or more of the plurality of vehicle connection interfaces are annotated in the annotated image;
provide the annotated image to a user device for presentation to a user;
receive a selection of at least one vehicle connection interface from the annotated vehicle connection interfaces;
create a 3D image scene based at least in part on a placement of the 3D model for the selected item with respect to the 3D model for the vehicle, wherein creating the 3D image scene further comprises placing one or more instances of the 3D model of the selected item with respect to the 3D model for the vehicle based at least in part on the received selection of the at least one vehicle connection interface; and
render a display image based at least in part on the 3D image scene.

12. The system of claim 11 , wherein the interactive computing system is further configured to execute the specific computer-executable instructions to at least determine whether the selected item fits the vehicle by determining whether the 3D model for the selected item mates with the 3D model for the vehicle at the at least one vehicle connection interface within a threshold degree of error.

13. The system of claim 12 , wherein the threshold degree of error is based at least in part on the item type and/or the material properties of the selected item.

14. The system of claim 12 , wherein the interactive computing system is further configured to execute the specific computer-executable instructions to at least annotate the at least one vehicle connection interface within the display image to indicate whether the selected item fits the vehicle.

15. The system of claim 12 , wherein, in response to determining that the selected item does not fit the vehicle, the interactive computing system is further configured to execute the specific computer-executable instructions to at least:
identify an alternative item that does fit the vehicle; and
output a recommendation of the alternative item.

16. The system of claim 11 , wherein the interactive computing system is further configured to execute the specific computer-executable instructions to at least modify a view of the display image corresponding to movement of the user device that captures a continuous stream of images of one or more portions of the vehicle, wherein movement of the user device is detected based at least in part on a comparison of the image with a subsequent image from the continuous stream of images.

Assignees

• Amazon Tech Inc

Inventors

• Hasan Shiblee Imtiaz

Classifications

• G06T2219/2016

  Rotation, translation, scaling · CPC title

• G06T11/00

  Two-dimensional [2D] image generation · CPC title

• G06T2219/004

  Annotating, labelling · CPC title

• G06T2219/2004

  Aligning objects, relative positioning of parts · CPC title

• G06T19/20

  Editing of three-dimensional [3D] images, e.g. changing shapes or colours, aligning objects or positioning parts · CPC title