Utilizing machine learning to generate augmented reality vehicle information for a vehicle captured by cameras in a vehicle lot

Patent metadata

Field	Value
Publication number	US-12236473-B2
Application number	US-202117304043-A
Country	US
Kind code	B2
Filing date	Jun 14, 2021
Priority date	Apr 25, 2019
Publication date	Feb 25, 2025
Grant date	Feb 25, 2025

Abstract

A device receives, from multiple cameras, streaming video data associated with multiple vehicles and provides the streaming video data to a user device associated with a user. The device receives, from the user device, a request to control a first camera and provides, to the user device, first streaming video data associated with the first camera. The device receives, from the user device, a camera control command for the first camera and causes the first camera to perform an action. The device receives updated first streaming video data from the first camera and provides, to the user device, the updated first streaming video data. The device receives, from the user device, information identifying a first vehicle in the updated first streaming video data and identifies first augmented reality vehicle information associated with the first vehicle. The device provides the first augmented reality vehicle information to the user device.

First claim

What is claimed is:

1. A method, comprising:
receiving, by a device and from a plurality of cameras, streaming video data associated with a plurality of vehicles in a vehicle lot,
wherein the streaming video data includes first camera streaming video data associated with a first camera of the plurality of cameras;

providing, by the device, the streaming video data to a user device based on the user device accessing a web page associated with the plurality of vehicles;
receiving, by the device and from the user device, a first camera control command to control a first movement of the first camera;
causing, by the device, the first camera to move based on the first camera control command;
receiving, by the device, updated first camera streaming video data from the first camera,
wherein the updated first camera streaming video data includes streaming video data captured by the first camera associated with a first updated field of view after moving the first camera;

providing, by the device and to the user device, the updated first camera streaming video data;
receiving, by the device and from the user device, information identifying a first vehicle, of the plurality of vehicles, in the updated first camera streaming video data;
training, by the device, a first machine learning model using historical vehicle information including information indicating vehicle features, vehicle logos, vehicle makes, and vehicle models,
wherein training the first machine learning model comprises:
performing dimensionality reduction to reduce the historical vehicle information, and
training the first machine learning model using an artificial neural network processing technique to perform pattern recognition with regard to patterns of the historical vehicle information;

processing, by the device and using image processing and using the first machine learning model, an image associated with the information identifying the first vehicle to identify a make and model of the first vehicle;
identifying, by the device, first augmented reality vehicle information associated with the first vehicle based on the information identifying the first vehicle,
wherein identifying the first augmented reality vehicle information comprises:
using a second machine learning model to determine cost information, associated with the first vehicle, personalized to a user associated with the user device; and

providing, by the device, the first augmented reality vehicle information to the user device to enable the user device to overlay the first augmented reality vehicle information, including the cost information, on the updated first camera streaming video data.

2. The method of claim 1 , wherein the first camera control command includes a command to cause the first camera to one or more of: tilt, rotate, pan, or zoom.

3. The method of claim 1 , wherein the information identifying the first vehicle comprises one or more of:
an image of the first vehicle, or
a selection of a portion of an image of the first vehicle.

4. The method of claim 1 , wherein identifying the first augmented reality vehicle information comprises:
processing the information identifying the first vehicle using a third machine learning model,
wherein the third machine learning model is trained on other historical vehicle information.

5. The method of claim 1 , wherein identifying the first augmented reality vehicle information is based on image processing the information identifying the first vehicle to identify a vehicle type associated with the first vehicle.

6. The method of claim 1 , wherein identifying the first augmented reality vehicle information comprises:
identifying the first vehicle based on:
a location of the first camera, or
a location of the first vehicle.

7. The method of claim 1 , further comprising:
receiving a second camera control command to control a second movement of a second camera of the plurality of cameras; and
causing the second camera to move based on the second camera control command; and
receiving, by the device, updated second camera streaming video data from the second camera,
wherein the updated second camera streaming video data includes streaming video data captured by the second camera associated with a second updated field of view after moving the second camera.

8. A device, comprising:
one or more memories; and
one or more processors, coupled to the one or more memories, configured to:
receive, from a plurality of cameras, streaming video data associated with a plurality of vehicles in a vehicle lot,
wherein the streaming video data includes first camera streaming video data associated with a first camera of the plurality of cameras;

provide the streaming video data to a user device based on the user device accessing a web page associated with the plurality of vehicles;
receive, from the user device, a first camera control command to control a first movement of the first camera;
cause the first camera to move based on the first camera control command;
receive updated first camera streaming video data from the first camera,
wherein the updated first camera streaming video data includes streaming video data captured by the first camera associated with a first updated field of view after moving the first camera;

provide, to the user device, the updated first camera streaming video data;
receive, from the user device, information identifying a first vehicle, of the plurality of vehicles, in the updated first camera streaming video data;
train a first machine learning model using historical vehicle information including information indicating vehicle features, vehicle logos, vehicle makes, and vehicle models,
wherein the one or more processes, to train the first machine learning model, are configured to:
perform dimensionality reduction to reduce the historical vehicle information, and
train the first machine learning model using an artificial neural network processing technique to perform pattern recognition with regard to patterns of the historical vehicle information;

process, using image processing and using the first machine learning model, an image associated with the information identifying the first vehicle to identify a make and model of the first vehicle;
identify first augmented reality vehicle information associated with the first vehicle based on the information identifying the first vehicle,
wherein the one or more processors, to identify the first augmented reality vehicle information, are configured to:
use a second machine learning model to determine cost information, associated with the first vehicle, personalized to a user associated with the user device; and

provide the first augmented reality vehicle information to the user device to enable the user device to overlay the first augmented reality vehicle information, including the cost information, on the updated first camera streaming video data.

9. The device of claim 8 , wherein the first camera control command includes a command to cause the first camera to one or more of: tilt, rotate, pan, or zoom.

10. The device of claim 8 , wherein the information identifying the first vehicle comprises one or more of:
an image of the first vehicle, or
a selection of a portion of an image of the first vehicle.

11. The device of claim 8 , wherein the one or more processors, when identifying the first augmented reality vehicle information, are configured to:
process the information identifying the first vehicle using a third machine learning model,
wherein the third machine learning model is trained on other historical vehicle information.

12. The device of claim 8 ,

Assignees

• Capital One Services Llc

Inventors

• Tang Qiaochu
• Wylie Stephen
• Dagley Geoffrey
• Awoyemi Olalekan
• Duckworth Staevan
• Hoover Jason
• Price Micah

Classifications

• G06Q30/0623

  by investigating goods or services · CPC title

• H04N7/181

  for receiving images from a plurality of remote sources · CPC title

• G06T19/006

  Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00) · CPC title

• G06V2201/08

  Detecting or categorising vehicles · CPC title

• G06V20/20

  in augmented reality scenes · CPC title