System and method for providing an augmented reality vehicle interface

Patent metadata

Field	Value
Publication number	US-9550419-B2
Application number	US-201414160389-A
Country	US
Kind code	B2
Filing date	Jan 21, 2014
Priority date	Jan 21, 2014
Publication date	Jan 24, 2017
Grant date	Jan 24, 2017

Abstract

A method and system for providing an augmented reality vehicle interface. The method and system include providing an augmented reality user interface. The method and system additionally include receiving an image of a vehicle with an image capturing device. The method and system additionally include identifying a user classification category of the user that is capturing an image of the vehicle. Additionally, the method and system include presenting an augmented reality image of the vehicle by overlaying one or more virtual user interface objects on the points of interest. The method and system also includes controlling vehicle features via the one or more virtual user interface objects.

First claim

The invention claimed is:

1. A method for providing an augmented reality vehicle interface comprising:
receiving an image of a vehicle with an image capturing device;
identifying points of interest within portions of the image of the vehicle that correspond with vehicle features;
identifying a user classification category of the user that is capturing the image of the vehicle, wherein identifying the user classification category of the user that is capturing the image of the vehicle includes determining whether a driver or a passenger is operating the image capturing device while capturing the image of the vehicle, wherein determining whether the driver or the passenger is operating the image capturing device includes determining a cardinal and ordinal position of the image capturing device with respect to the vehicle while the user is capturing the image of the vehicle, wherein a guidance point is assigned at a location of a front portion of the vehicle that is based on the cardinal position of the image capturing device, and the guidance point is utilized to determine the ordinal position of the image capturing device, wherein determining whether the driver or the passenger is operating the image capturing device includes determining a location within the vehicle where the user capturing the image of the vehicle is located, wherein determining the location within the vehicle where the user capturing the image of the vehicle is located includes determining directional degrees of the location of the image capturing device with respect to the guidance point, wherein the location of the vehicle includes at least one of: a left front seat of the vehicle, a center front seat of the vehicle, a right front seat of the vehicle, a left rear seat of the vehicle, a center rear seat of the vehicle, and a center right seat of the vehicle, and wherein the user classification category is identified based on the location within the vehicle where the user capturing the image of the vehicle is located;
presenting an augmented reality image of the vehicle by overlaying one or more virtual user interface objects on the points of interest, wherein the one or more virtual user interface objects are overlaid as one or more circular points of focus over the points of interest, wherein the one or more virtual user interface objects are presented based on whether the driver or the passenger is operating the image capturing device while capturing the image of the vehicle; and
controlling the vehicle features via the one or more virtual user interface objects.

2. The method of claim 1 , wherein identifying points of interest within portions of the image of the vehicle that correspond with the vehicle features includes determining a directional orientation and positional location of the image capturing device with respect to the vehicle and identifying the vehicle features being captured by the image capturing device based on the direction and position of the image capturing device with respect to the vehicle.

3. The method of claim 1 , wherein identifying points of interest within portions of the image of the vehicle that correspond with the vehicle features includes identifying the location of a predetermined marker as a focal point within the image at a portion of the vehicle, and identifying points of interests based on the location of the predetermined marker by determining an orientation of the image capturing device.

4. The method of claim 1 , wherein presenting the augmented reality image of the vehicle includes providing the one or more virtual user interface objects shown on an augmented reality user interface that includes a plurality of vehicle feature interface icons that correspond to the vehicle features, wherein the plurality of vehicle feature interface icons are presented based on whether the driver or the passenger is operating the image capturing device when capturing the image of the vehicle.

5. The method of claim 4 , wherein presenting the augmented reality image of the vehicle includes providing the one or more virtual user interface objects shown on the augmented reality user interface that includes a plurality of vehicle feature settings menus icons on a vehicle feature settings menu that are utilized to determine the settings associated with the vehicle features, wherein the plurality of vehicle feature settings menus icons are presented based on whether the driver or the passenger is operating the image capturing device when capturing the image of the vehicle.

6. The method of claim 5 , wherein presenting the augmented reality image of the vehicle includes providing the one or more virtual user interface objects shown on the augmented reality user interface that includes a plurality of vehicle feature settings sub-menu icons on a vehicle feature settings sub-menu that are utilized to manipulate the settings associated with the vehicle features, wherein the plurality of vehicle feature settings sub-menus icons are presented based on whether the driver or the passenger is operating the image capturing device when capturing the image of the vehicle.

7. The method of claim 1 , wherein controlling one or more types of the vehicle features includes communicating with an electronic control unit of the vehicle via peer to peer communication between the image capturing device and the electronic control unit of the vehicle and receiving settings data associated with the vehicle features associated with the points of interest.

8. The method of claim 7 , wherein the controlling one or more types of the vehicle features includes receiving one or more user inputs on the plurality of vehicle feature settings sub-menu icons on the vehicle feature settings sub-menu associated with a corresponding vehicle feature and communicating with the electronic control unit of the vehicle to change settings data associated with the corresponding vehicle features.

9. A system for providing an augmented reality vehicle interface, comprising:
a memory storing instructions that when executed by a processor cause the processor to:
receiving an image of a vehicle with an image capturing device;
identify points of interest that correspond with vehicle features within portions of an image of the vehicle;
identify a user classification category of the user that is capturing the image of the vehicle, wherein identifying the user classification category of the user that is capturing the image of the vehicle includes determining whether a driver or a passenger is operating the image capturing device while capturing the image of the vehicle, wherein determining whether the driver or the passenger is operating the image capturing device includes determining a cardinal and ordinal position of the image capturing device with respect to the vehicle while the user is capturing the image of the vehicle, wherein a guidance point is assigned at a location of the front portion of the vehicle that is based on the cardinal position of the image capturing device, and the guidance point is utilized to determine the ordinal position of the image capturing device, wherein determining whether the driver or the passenger is operating the image capturing device includes determining a location within the vehicle where the user capturing the image of the vehicle is located, wherein determining the location within the vehicle where the user capturing the image of the vehicle is located includes determining directional degrees of the location of the image capturing device with respect to the guidance point, wherein the location of the vehicle includes at least one of: a left front seat of the vehicle, a center front seat of the vehicle, a right front seat of the vehicle, a left rear seat of the vehicle, a center rea

Assignees

• Honda Motor Co Ltd

Inventors

• Habashima Yoshiyuki
• Kurosawa Fuminobu
• Alaniz Arthur
• Gleeson-May Michael

Classifications

• G06T19/006 Primary

  Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00) · CPC title

• B60K35/211

  producing three-dimensional [3D] effects, e.g. stereoscopic images · CPC title

• B60K35/81

  for controlling displays · CPC title

• G08G1/00

  Traffic control systems for road vehicles (arrangement of road signs or traffic signals E01F9/00 {; automatic vehicle control B62D}) · CPC title

• G06F3/017

  Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883) · CPC title