Systems and methods for car shopping using messaging framework

Patent metadata

Field	Value
Publication number	US-10909599-B2
Application number	US-201815915863-A
Country	US
Kind code	B2
Filing date	Mar 8, 2018
Priority date	Mar 8, 2018
Publication date	Feb 2, 2021
Grant date	Feb 2, 2021

Abstract

Disclosed embodiments provide devices, methods, and computer-readable storage media for determining information of an object from a captured image. Further, the disclosed device responds to a user input by capturing, with the camera, image data representing an object. The device then transmits, via a messaging application through a network interface to a server, the image data and a request for information relating to the object. The object information is received via the messaging application through the network from the server and is displayed on the display screen.

First claim

What is claimed is:

1. A mobile device for determining object information, the device comprising:
at least one memory device storing instructions, the instructions comprising an operating system with a text messaging application;
a camera;
a network interface;
a display screen; and
at least one processor executing the instructions to perform operations comprising:
responding to a user input by capturing, with the camera, image data representing a captured image of an object,
preparing a text message compatible with a text message service for communication,
transmitting, via the text messaging application through the network interface to a server, the text message, the image data, and a request for information relating to the object, wherein the server is configured to prepare the information relating to the object in response to the request, by:
generating temporary images of an entirety of the captured image by executing at least one modification on the captured image, wherein the modification is one of flipping the captured image along x-axis and y-axis, skewing the captured image horizontally and vertically, or changing color characteristics of the captured image by manipulating a histogram;
comparing each of the temporary images to a plurality of external images of known objects from a database of reference images;
assigning scores to the reference images based on similarity to each of the temporary images;
aggregating the scores assigned to the reference images;
extracting available exchangeable image file format (EXIF) information in the image data;
identifying one of the reference images with a highest aggregated score as an identity of the object;
determining information related to the object based on the identity of the object and the extracted EXIF information, wherein the determining includes sending at least one of the identity of the object and the extracted EXIF information to a third-party server; and
preparing a response text message compatible with the text message service based on the determined object information, comprising at least one of:
an identification of an entity offering the object for sale,
an address of a website offering the object for sale,
a physical location where the object is sold, or
information about the object based on physical location of the extracted EXIF information;

receiving the response text message via the text messaging application through the network interface from the server, and
displaying the object information on the display screen via the text messaging application.

2. The device of claim 1 , wherein the object information comprises an identification of the object.

3. The device of claim 2 , wherein the object information of the object further comprises review information of the object.

4. The device of claim 1 , wherein the camera comprises a video camera.

5. The device of claim 4 , wherein:
responding to the user input comprises capturing video data representing the object by the mobile device; and
the operations further comprise extracting the image data of the object from the video data by the server.

6. The device of claim 5 , wherein the server is further configured to extract multiple image frames from the video data based on a characteristic of the video data.

7. The device of claim 1 , wherein the scores assigned to the reference images are computed based on at least one of a linear regression algorithm, a boosted tree algorithm, or a convolution neural network.

8. The device of claim 1 , wherein the server is further configured to:
capture text within the captured image; and
perform a search of the text to identify the object in the captured image.

9. The device of claim 1 , wherein:
the object is a vehicle; and
the plurality of external images of known objects are images of vehicles of known makes and models.

10. A non-transitory computer-readable storage medium storing instructions that, when executable by at least one processor of a mobile device, cause the mobile device to perform a method for determining vehicle information, the method comprising:
responding to a user input by capturing, with a camera, image data representing a captured image of a vehicle;
preparing a text message compatible with a text message service for communication;
transmitting via a text messaging application through a network interface to a server, the text message and a request for information relating to the vehicle, the server being configured to prepare the information relating to the vehicle in response to the request, by:
generating temporary images of an entirety of the captured image by executing at least one modification on the captured image,
comparing each of the temporary images to a plurality of extern al reference images of known makes and models from a database of reference images,
assigning scores to the known makes and models of the reference images based on similarity to the temporary images,
aggregating the scores assigned to the reference images of the known makes and models, to create an aggregated make and model score,
extracting available exchangeable image file format (EXIF) information in the image data,
determining, based on the EXIF information, a physical location where the image data was captured,
identifying one of the known makes and models with a highest aggregated make and model score as an identity of the vehicle,
determining information related to the vehicle based on the identity of the vehicle and the physical location where the image data was captured, and
preparing a response text message compatible with the text message service based on the determined vehicle information, the vehicle information comprising at least one of:
an identification of an entity offering the vehicle for sale;
an address of a website offering the vehicle for sale;
a physical location where the vehicle is sold; or
information about the vehicle based on the physical location where the image data was captured,

receiving the response text mess age via the text mess aging application through the network interface from the server; and
displaying the vehicle information on a display screen.

11. The non-transitory computer-readable medium of claim 10 , wherein displaying the vehicle information comprises displaying an identification of the vehicle.

12. The non-transitory computer-readable medium of claim 11 , wherein displaying the vehicle information further comprises displaying review information of the vehicle.

13. The non-transitory computer-readable medium of claim 10 , wherein the scores assigned to the known makes and models are computed by applying at least one of: a linear regression algorithm, a boosted tree algorithm, or a convolution neural network.

14. The non-transitory computer-readable medium of claim 10 , wherein the server is further configured to:
capture text within the captured image; and
perform a search of the text to identify the vehicle in the captured image.

15. The non-transitory computer-readable medium of claim 10 , wherein:
responding to the user input comprises capturing video data representing the vehicle by the mobile device; and
the server is further configured to extract multiple image frames from the video data based on a characteristic of the video data.

16. A method performed by a mobile device, comprising:
responding to a user input by capturing, with a camera, image data representing a captured image of a vehicle;
preparing a text message compatible with a text message service for communication;
transmitting via a text mess a

Assignees

• Capital One Services Llc

Inventors

• Reddy Sean
• Vasisht Sunil Subrahmanyam
• Price Micah
• Dugal Arjun
• Dagley Geoffrey
• Bajaj Raman
• Wylie Stephen Michael
• Hoover Jason Richard
• Tang Qiaochu

Classifications

• G06V10/17

  using hand-held instruments · CPC title

• G06V10/761

  Proximity, similarity or dissimilarity measures · CPC title

• G06F18/22

  Matching criteria, e.g. proximity measures · CPC title

• G06V10/751

  Comparing pixel values or logical combinations thereof, or feature values having positional relevance, e.g. template matching · CPC title

• H04L51/222

  using geographical location information, e.g. messages transmitted or received in proximity of a certain spot or area · CPC title