Vehicle inventory search recommendation using image analysis driven by machine learning

Patent metadata

Field	Value
Publication number	US-11068549-B2
Application number	US-201916685385-A
Country	US
Kind code	B2
Filing date	Nov 15, 2019
Priority date	Nov 15, 2019
Publication date	Jul 20, 2021
Grant date	Jul 20, 2021

Abstract

A device receives user interface interaction data that identifies an interaction that a user has with an interface that displays an image of a vehicle. The device generates, by using a data model to process at least a portion of the image, an array of vectors that includes one or more vectors relating to vehicle characteristics of the vehicle. The device assigns one or more weights to the one or more vectors based on the user interface action data. The device determines, based on a similarity analysis, similarity scores that indicate similarities between the array of vectors that include the one or more vectors that have been weighted and other arrays relating to the vehicles depicted in the images. The device selects a subset of the images based on the similarity scores and causes the subset of the images to be displayed.

First claim

What is claimed is:

1. A method, comprising:
receiving, by a device, user interface interaction data that identifies one or more interactions that a user has with one or more interfaces of a vehicle search engine that display images of a set of vehicles,
wherein an interaction, of the one or more interactions, is with a specific part of an image of a vehicle;

identifying, by the device and by using at least one model to process the image of the vehicle, a set of vehicle characteristics of the vehicle,
wherein the at least one model outputs one or more image crops that include labeled vehicle characteristics of the vehicle;

generating, by the device and by using the at least one model that has been trained using machine learning to process image data that depicts at least a portion of the vehicle, an array of vectors that includes one or more vectors that represent the set of vehicle characteristics of the vehicle,
wherein the image data includes the one or more image crops;

assigning, by the device, one or more weights to the one or more vectors based on the user interface interaction data;
identifying, by the device, a first orientation of the vehicle within the image;
filtering, by the device, particular arrays of vectors from a set of arrays of vectors associated with the set of vehicles to create a subset of arrays of vectors,
wherein the particular arrays of vectors are associated with a second orientation, that is different from the first orientation, of vehicles within the images of the set of vehicles;

determining, by the device, a set of similarity scores that indicate similarities between the array of vectors that include the one or more vectors that have been weighted and the subset of arrays of vectors,
wherein the set of similarity scores are determined based on a similarity analysis of the array of vectors that include the one or more vectors that have been weighted and the subset of arrays of vectors;

selecting, by the device, a subset of the images of the set of vehicles based on the set of similarity scores; and
causing, by the device, the subset of the images to be displayed via at least one of the one or more interfaces of the vehicle search engine.

2. The method of claim 1 , wherein the one or more interactions identified by the user interface interaction data are a sequence of interactions that the user has with the one or more interfaces over a given time period; and
wherein assigning the one or more weights to the one or more vectors comprises:
assigning the one or more weights to the one or more vectors based on the sequence of interactions.

3. The method of claim 1 , wherein the set of vehicle characteristics includes a first subset of vehicle characteristics that are of a first level of granularity and one or more other subsets of vehicle characteristics that are of one or more other levels of granularity;
wherein the method further comprises:
filtering, based on the first subset of vehicle characteristics that are of the first level of granularity, a master set of vehicle characteristics to identify a filtered master set of vehicle characteristics; and

wherein determining the set of similarity scores comprises:
determining the set of similarity scores by performing the similarity analysis on the array of vectors that include the one or more vectors that have been weighted and a particular set of vectors that correspond to the filtered master set of vehicle characteristics.

4. The method of claim 1 , wherein the set of vehicle characteristics include at least one of:
a trim of the vehicle,
a style of the vehicle,
a shade of a color of the vehicle,
a sub-component of a component of the vehicle,
an attribute of the component of the vehicle, or
an attribute of the sub-component of the component.

5. The method of claim 1 , wherein the one or more interactions identified by the user interface interaction data are a sequence of interactions that the user has with the one or more interfaces over a given time period;
wherein the method further comprises:
generating user preferences data that identifies one or more user preferences of the user based on the sequence of interactions that the user has with the one or more interfaces; and

wherein assigning the one or more weights comprises:
assigning the one or more weights to the one or more vectors based on the user interface interaction data and the user preferences data.

6. The method of claim 1 , wherein the set of vehicle characteristics are a first set of vehicle characteristics that are detectable by the at least one model;
wherein the method further comprises:
identifying a second set of vehicle characteristics that are not detectable by the at least one model;
filtering the set of arrays of vectors based on the second set of vehicle characteristics; and
wherein determining the set of similarity scores comprises:
determining the set of similarity scores between the array of vectors that include the one or more vectors that have been weighted and the filtered set of arrays of vectors filtered based on the second set of vehicle characteristics.

7. A device, comprising:
one or more memories; and
one or more processors, operatively coupled to the one or more memories, configured to:
receive, over a given time period, user interface interaction data for a sequence of interactions that a user has with one or more interfaces of a vehicle search engine that display images of a set of vehicles,
wherein an interaction, of the sequence of interactions, is with an image of a vehicle;

output, by using at least one model, one or more image crops that include labeled vehicle characteristics of the vehicle;
generate an array of vectors by using the at least one model that has been trained using machine learning to process image data that depicts at least a portion of the image of the vehicle,
wherein the image data includes the one or more image crops, and
wherein the array of vectors includes one or more vectors that represent a set of vehicle characteristics of the vehicle;

assign one or more weights to the one or more vectors based on the user interface interaction data;
identify a first orientation of the vehicle within the image;
filter particular arrays of vectors from a set of arrays of vectors associated with the set of vehicles to create a subset of arrays of vectors,
wherein the particular arrays of vectors are associated with a second orientation, that is different from the first orientation of vehicles within the images of the set of vehicles;

determine a set of similarity scores that indicate similarities between the one or more vectors that have been weighted and one or more other vectors that are part of the subset of arrays of vectors,
wherein the set of similarity scores are determined based on a similarity analysis of the one or more vectors that have been weighted and the one or more other vectors;

select a subset of the images of the set of vehicles based on the set of similarity scores; and
cause the subset of the images to be displayed via at least one of the one or more interfaces of the vehicle search engine.

8. The device of claim 7 , wherein the one or more processors are further configured to:
identify, before generating the array of vectors, the set of vehicle characteristics of the vehicle by using the at least one model to process the image of the vehicle.

9. The device of claim 7 , wherein the interaction with the image of the vehicle is a particular interaction with a specific part of the image of vehicle; and
wherein the one or more processors, when assigning the one or more weights to the one or more vectors, are configured to:
assign the one or more we

Assignees

• Capital One Services Llc

Inventors

• Duan Yue
• Vasisht Sunil
• Ho Chi-San
• Price Micah

Classifications

• G06N3/045

  Combinations of networks · CPC title

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06N3/09

  Supervised learning · CPC title

• G06N3/084

  Backpropagation, e.g. using gradient descent · CPC title

• G06F16/5854

  using shape and object relationship · CPC title