System and method for in-vehicle interaction

Patent metadata

Field	Value
Publication number	US-9613459-B2
Application number	US-201314135036-A
Country	US
Kind code	B2
Filing date	Dec 19, 2013
Priority date	Dec 19, 2013
Publication date	Apr 4, 2017
Grant date	Apr 4, 2017

Abstract

A system and method for interaction in a vehicle includes establishing a bi-directional interaction session between a first display and a second display via a communication link, wherein each of the first display and the second display are communicatively coupled to the vehicle and the first display is operated by a first user and the second display is operated by a second user. Upon receiving a first interaction input associated generating a first augmented reality object based on the first interaction input and transmitting via the communication link the first augmented reality object to the second display. Upon receiving a second interaction input, generating a second augmented reality object based upon the second interaction input and transmitting via the communication link the second augmented reality object to the first display.

First claim

The invention claimed is:

1. A method for interaction in a vehicle, comprising:
establishing a bi-directional interaction session via a communication link between a heads-up display operated by a driver and a passenger display operated by a passenger, wherein the heads-up display and the passenger display are communicatively coupled to the vehicle, and wherein the passenger display is physically separate from the heads-up display;
upon receiving a first interaction input from the driver associated with a point of interest in a real-time point of view of the driver viewed through the heads-up display, generating a first augmented reality object based on the first interaction input, the point of interest, and the real-time point of view of the driver, and displaying the first augmented reality object on the heads-up display;
transmitting via the communication link the first augmented reality object to the passenger display for displaying the first augmented reality object on the passenger display;
upon receiving a second interaction input from the passenger at the passenger display, determining if the point of interest is visible by the driver in an updated real-time point of view of the driver viewed through the heads-up display;
generating a second augmented reality object based upon the second interaction input, the first augmented reality object, and the determination of whether the point of interest is visible by the driver; and
transmitting via the communication link the second augmented reality object to the heads-up display for displaying the second augmented reality object on the heads up display in accordance with the determination of whether the point of interest is visible by the driver;
wherein upon determining the point of interest is visible by the driver in the updated real-time point of view of the driver viewed through the heads-up display, transmitting via the communication link the second augmented reality object includes transmitting via the communication link the second augmented reality object to the heads-up display for displaying the second augmented reality object on the heads-up display at a location corresponding to the point of interest in the updated real-time point of view of the driver;
wherein upon determining the point of interest is not visible by the driver in the updated real-time point of view of the driver viewed through the heads-up display, generating the second augmented reality object includes generating the second augmented reality object based on a local view perceivable to the passenger on the passenger display and transmitting via the communication link the second augmented reality object to the heads-up display.

2. The method of claim 1 , wherein establishing the bi-directional interaction session includes establishing the bi-directional interaction session based on an initialization event.

3. The method of claim 1 , wherein establishing the bi-directional interaction session includes displaying a request for interaction as an augmented reality object on the heads-up display or the passenger display, the request for interaction indicating one other display communicatively coupled to the vehicle for interaction.

4. The method of claim 3 , wherein establishing the bi-directional interaction session includes receiving an input in response to the request for interaction from the heads-up display or the passenger display.

5. The method of claim 4 , wherein establishing the bi-directional interaction session includes establishing the bi-directional interaction session based on the input.

6. The method of claim 1 , wherein transmitting via the communication link the second augmented reality object to the heads-up display for displaying the second augmented reality object includes displaying the second augmented reality object as a standalone image on the heads-up display.

7. A non-transitory computer-readable storage medium storing instructions that, when executed by a vehicle computer, causes the vehicle computer to perform a method including:
establishing a bi-directional interaction session via a communication link between a heads-up display operated by a driver and a passenger display operated by a passenger, wherein the heads-up display and the passenger display are communicatively coupled to the vehicle, and wherein the passenger display is physically separate from the heads-up display;
upon receiving a first interaction input from the driver associated with a point of interest in a real-time point of view of the driver viewed through the heads-up display, generating a first augmented reality object based on the first interaction input, the point of interest, and the real-time point of view of the driver, and displaying the first augmented reality object on the heads-up display;
transmitting via the communication link the first augmented reality object to the passenger display for displaying the first augmented reality object on the passenger display;
upon receiving a second interaction input from the passenger at the passenger display, determining if the point of interest is visible by the driver in an updated real-time point of view of the driver viewed through the heads-up display;
generating a second augmented reality object based upon the second interaction input, the first augmented reality object, and the determination of whether the point of interest is visible by the driver; and
transmitting via the communication link the second augmented reality object to the heads-up display for displaying the second augmented reality object on the heads up display in accordance with the determination of whether the point of interest is visible by the driver;
wherein upon determining the point of interest is visible by the driver in the updated real-time point of view of the driver viewed through the heads-up display, transmitting via the communication link the second augmented reality object includes transmitting via the communication link the second augmented reality object to the heads-up display for displaying the second augmented reality object on the heads-up display at a location corresponding to the point of interest in the updated real-time point of view of the driver;
wherein upon determining the point of interest is not visible by the driver in the updated real-time point of view of the driver viewed through the heads-up display, generating the second augmented reality object includes generating the second augmented reality object based on a local view perceivable to the passenger on the passenger display and transmitting via the communication link the second augmented reality object to the heads-up display.

8. The non-transitory computer-readable storage medium of claim 7 , wherein establishing the bi-directional interaction session includes displaying a request for interaction as an augmented reality object on the heads-up display or the passenger display, the request for interaction indicating one other display communicatively coupled to the vehicle for interaction.

9. The non-transitory computer-readable storage medium of claim 8 , wherein establishing the bi-directional interaction session includes receiving an input in response to the request for interaction from the heads-up display or the passenger display, and wherein establishing the bi-directional interaction session includes establishing the bi-directional interaction session based on the input.

10. The method of claim 7 , wherein transmitting via the communication link the second augmented reality object to the heads-up display for displaying the second augmented reality object includes displaying the second augmented reality object as a standalone image on the heads-up display.

11. A

Assignees

• Honda Motor Co Ltd

Inventors

• Ng-Thow-Hing Victor
• Bark Karlin
• Tran Cuong

Classifications

• B60K35/50

  Instruments characterised by their means of attachment to or integration in the vehicle (B60K35/231 takes precedence) · CPC title

• B60K35/22

  Display screens · CPC title

• B60K35/60

  Instruments characterised by their location or relative disposition in or on vehicles (arrangements of lighting devices on dashboards B60Q3/10) · CPC title

• B60K35/23

  Head-up displays [HUD] (optical aspects of head-up displays G02B27/01) · CPC title

• G06F3/147

  using display panels · CPC title