Photograph driven vehicle identification engine

Patent metadata

Field	Value
Publication number	US-2019278994-A1
Application number	US-201816151280-A
Country	US
Kind code	A1
Filing date	Oct 3, 2018
Priority date	Mar 8, 2018
Publication date	Sep 12, 2019
Grant date	—

Abstract

Disclosed herein are systems and methods for a photograph driven vehicle identification system. In some embodiments, a system for image-based vehicle identification includes a database, an image processor, and a vehicle search engine. The database can include vehicle information. The image processor may apply one or more machine learning models on images received by a user device. The user device can include a camera that obtains the images. The user device can provide a display having images of a vehicle and information associated with the vehicle through a user interface (UI) of the user device. The display can include a first portion at a first location of the UI, and a second portion at a second location of the UI. The first portion and the second portion may be provided at a single instance. The vehicle search engine may identify one or more vehicles in the images received.

First claim

What is claimed is:

1 . A system for image-based vehicle identification, the system comprising:
a database comprising a plurality of vehicle information; an image data processor configured to apply one or more machine learning models on one or more images received by a user device, wherein the user device comprises a camera configured to obtain one or more images, wherein the user device is configured to provide a display comprising one or more images of a vehicle and information associated with the vehicle through a user interface of the user device, wherein the display comprises a first portion provided at a first location of the user interface, and a second portion provided at a second location different from the first location, each of the first portion and the second portion provided at a single instance; and a vehicle search engine configured to identify one or more vehicles in the images received from the user device.

2 . The system of claim 1 , wherein each of the one or more machine learning models identify a plurality of objects in the received images, at least one of the plurality of objects is a vehicle.

3 . The system of claim 1 , wherein the vehicle search engine is configured to identify a plurality of vehicle image co-ordinates corresponding to the one or more vehicles in the images received from the user device using a Single Shot Detector Inception machine learning model.

4 . The system of claim 1 , wherein the image data processor is configured to generate a detailed vehicle information based on the vehicle information retrieved from the database for each of the identified vehicles.

5 . The system of claim 4 , wherein the detailed vehicle information comprises at least one of: a mileage information, a pricing information, a vehicle stock information, a location of a vehicle dealer, a color information, one or more customer rating information, and a body style information.

6 . The system of claim 4 , wherein the image data processor is configured to generate an augmented image for each of the identified vehicles by overlaying the detailed vehicle information upon an image of at least one of the identified vehicles.

7 . The system of claim 6 , wherein the user device is configured to display the augmented image for each of the identified vehicles through the user interface of the user device.

8 . The system of claim 1 , wherein the image data processor is configured to:
receive image data for the one or more images obtained by the camera, wherein the image data is received in a system comprising a convolutional neural network (CNN), the CNN comprising an input layer, a first convolutional layer coupled to the input layer, a last convolutional layer, a fully connected layer coupled to the last convolution layer, and an output layer; extract multi-channel data from the output of the last convolutional layer; sum the extracted data to generate a general activation map; detect a location of an object within the one or more images by applying the general activation map to the received image data; receive one or more classifications of the output layer; and display the one or more images and a content overlay, wherein a position of the content overlay relative to the one or more images is determined using the detected object location, wherein the content overlay comprises information determined by the one or more classifications.

9 . The system of claim 1 , wherein the image data processor is configured to:
identify a plurality of vehicle image co-ordinates for each identified vehicle; perform a cropping of each of the one or more received images in accordance with the identified vehicle image co-ordinates; generate one or more cropped images from the one or more received images; and store the generated cropped images of the identified vehicle in the database.

10 . The system of claim 9 , wherein the image data processor is configured to perform the cropping of each of the one or more received images based on a scaling of the identified vehicle image co-ordinates in accordance with a plurality of parameters associated with the one or more received images.

11 . A method for image-based vehicle identification, the method comprising:
receiving one or more images from a user device; extracting one or more parameters corresponding to at least one of the received images; providing the determined one or more parameters as input to one or more machine learning models; obtaining, as an output from the one or more machine learning models, a prediction of one or more vehicle information, each vehicle information corresponding to a vehicle in the obtained one or more images, at least one of the one or more machine learning models being a Single Shot Detector Inception machine learning model; identifying, from the one or more predicted vehicle information obtained from the one or more machine learning models, one or more vehicles matching the vehicle in the obtained one or more images; and presenting a display with the one or more identified vehicles to the user device.

12 . The method of claim 11 , further comprising, for each of the vehicles identified from the one or more predicted vehicle information:
generating a detailed vehicle information based on a vehicle information retrieved from a database.

13 . The method of claim 12 , wherein the detailed vehicle information comprises at least one of: a mileage information, a pricing information, a vehicle stock information, a location of a vehicle dealer, a color information, one or more customer rating information, and a body style information.

14 . The method of claim 12 , further comprising:
generating an augmented image for each of the identified vehicles by overlaying the detailed vehicle information upon an image of at least one of the one or more identified vehicles.

15 . The method of claim 14 , further comprising:
displaying the augmented image for each of the identified vehicles through an user interface of the user device.

16 . The method of claim 11 , further comprising:
receiving image data for the one or more images obtained by the camera, wherein the image data is received in a system comprising a convolutional neural network (CNN), the CNN comprising an input layer, a first convolutional layer coupled to the input layer, a last convolutional layer, a fully connected layer coupled to the last convolution layer, and an output layer; extracting multi-channel data from the output of the last convolutional layer; summing the extracted data to generate a general activation map; detecting a location of an object within the one or more images by applying the general activation map to the received image data; receiving one or more classifications of the output layer; and displaying the one or more images and a content overlay, wherein a position of the content overlay relative to the one or more images is determined using the detected object location, wherein the content overlay comprises information determined by the one or more classifications.

17 . The method of claim 11 , further comprising:
identifying a plurality of vehicle image co-ordinates for each identified vehicle matching the vehicle in the obtained one or more images; performing a cropping of each of the one or more received images in accordance with the identified vehicle image co-ordinates; generating one or more cropped images from the one or more received images; and storing the generated cropped images of the identified vehicle in a database.

18 . The metho

Assignees

• Capital One Services Llc

Inventors

• Bumpas Derek
• Youngblood Stewart
• Venuraju Mithra Kosur
• Deshpande Amit
• Hoover Jason
• Martinez Daniel
• Hardin William
• Chikkaveerappa Satish
• Bass Majaliwa
• Guiles Jacob
• Solbrook Sona
• Colon Valerie
• Ha Khai
• Price Micah
• Tang Qiaochu
• Wylie Stephen
• Dagley Geoffrey
• Huang Jeremy
• Parcha Venkata Satya

Classifications

• G06N3/08

  Learning methods · CPC title

• G06N3/045

  Combinations of networks · CPC title

• G06F16/55 Primary

  Clustering; Classification · CPC title

• G06T11/60

  Creating or editing images; Combining images with text · CPC title

• G06F16/583

  using metadata automatically derived from the content · CPC title