Vehicle systems and methods for determining a target based on selecting a virtual eye position or a pointing direction

Patent metadata

Field	Value
Publication number	US-2019227635-A1
Application number	US-201815877911-A
Country	US
Kind code	A1
Filing date	Jan 23, 2018
Priority date	Jan 23, 2018
Publication date	Jul 25, 2019
Grant date	—

Abstract

Vehicle systems and methods for determining a final target position based on either a first target position and a second target position are disclosed. A vehicle includes a user detection system configured to output a gesture signal in response to a hand of a user performing at least one gesture to indicate a final target position. The vehicle also includes a user gaze monitoring system configured to output an eye location signal that indicates an actual eye position of the user. The vehicle also includes one or more processors and one or more non-transitory memory modules communicatively coupled to the processors. The processors store machine-readable instructions that, when executed, cause the one or more processors to select either the first target position or the second target position as the final target position based on a first accuracy and a second accuracy.

First claim

1 . A vehicle, comprising:
a user detection system configured to output a gesture signal in response to a hand of a user performing at least one gesture to indicate a final target position; a user gaze monitoring system configured to output an eye location signal that indicates an actual eye position of the user; one or more processors; and one or more non-transitory memory modules communicatively coupled to the one or more processors and storing machine-readable instructions that, when executed, cause the one or more processors to perform at least the following:
determine a first point and a second point located on the hand of the user based at least in part on the gesture signal from the user detection system, wherein the first point and the second point define a pointing direction of the hand of the user;
calculate a virtual eye position based at least in part on the first point located on the hand of the user and the actual eye position;
calculate a first target position based on the virtual eye position, wherein the first target position includes a first accuracy;
calculate a second target position based on the pointing direction of the hand of the user, wherein the second target position includes a second accuracy;
select either the first target position or the second target position as the final target position based on the first accuracy and the second accuracy; and
control at least one vehicle system based at least in part on the final target position.

2 . The vehicle of claim 1 , wherein the machine-readable instructions further cause the one or more processors to:
determine a midpoint measured between a pair of virtual eyes; and determine a vector that originates at the midpoint and intersects the first point located on the hand of the user, wherein the vector represents a virtual gaze direction of the user.

3 . The vehicle of claim 2 , wherein the machine-readable instructions further cause the one or more processors to:
determine a presence of an object located in an environment surrounding the vehicle that intersects with the vector representing the virtual gaze direction; and identify the object that intersects with the vector as the first target position.

4 . The vehicle of claim 1 , wherein the machine-readable instructions further cause the one or more processors to:
determine a vector by extending a line segment representing a pointing axis beyond the hand of the user, wherein the pointing direction is determined based on the pointing axis; determine a presence of an object located in an environment surrounding the vehicle that intersects with the vector representing the pointing direction; and identify the object that intersects with the vector as the second target position.

5 . The vehicle of claim 1 , wherein the machine-readable instructions further cause the one or more processors to:
compare the first accuracy associated with the first target position with the second accuracy associated with the second target position; in response to determining the first accuracy is greater than the second accuracy, select the first target position to be the final target position; and in response to determining the second accuracy is greater than the first accuracy, select the second target position as the final target position.

6 . The vehicle of claim 1 , wherein the machine-readable instructions further cause the one or more processors to:
determine a lateral side of a body that is connected to the hand of the user based on the gesture signal; and determine the second accuracy associated with the second target position based on the lateral side of the body that is connected to the hand of the user, the pointing direction of the hand, and the second target position.

7 . The vehicle of claim 6 , wherein the machine-readable instructions further cause the one or more processors to:
determine a relative direction of the pointing direction with respect to the body of the user; compare the relative direction of the pointing direction with the lateral side of the body that is connected to the hand of the user; and in response to determining that the relative direction of the pointing direction is aimed opposite of the lateral side of the body that is connected to the hand of the user, select the second target position as the final target position.

8 . The vehicle of claim 6 , wherein the machine-readable instructions further cause the one or more processors to:
determine a relative direction of the pointing direction with respect to the body of the user; compare the relative direction of the pointing direction with the lateral side of the body that is connected to the hand of the user; and in response to determining that the relative direction of the pointing direction is aimed towards the lateral side of the body that is connected to the hand of the user, select the first target position as the final target position.

9 . The vehicle of claim 1 , wherein the machine-readable instructions further cause the one or more processors to:
determine that the user is pointing towards the second target position by the hand, wherein the hand includes a plurality of digits that each define a respective tip; and set the first point located on the hand of the user to the respective tip of a digit.

10 . The vehicle of claim 1 , wherein the machine-readable instructions further cause the one or more processors to:
determine that the user is pointing towards the second target position by an article defining a proximate end and a distal end, wherein the user grasps the proximate end of the article with the hand and the distal end of the article is directed towards the second target position; and set the first point located on the hand of the user as the distal end of the article.

11 . A vehicle, comprising:
a user detection system configured to output a gesture signal in response to a hand of a user performing at least one gesture to indicate a final target position; a user gaze monitoring system configured to output an eye location signal that indicates an actual eye position of the user; one or more processors; and one or more non-transitory memory modules communicatively coupled to the one or more processors and storing machine-readable instructions that, when executed, cause the one or more processors to perform at least the following:
determine a first point and a second point located on the hand of the user based at least in part on the gesture signal from the user detection system, wherein the first point and the second point define a pointing direction of the hand of the user;
calculate a virtual eye position based at least in part on the first point located on the hand of the user and the actual eye position;
calculate a first target position based on the virtual eye position, wherein the first target position includes a first accuracy;
calculate a second target position based on the pointing direction of the hand of the user, wherein the second target position includes a second accuracy;
compare the first accuracy associated with the first target position with the second accuracy associated with the second target position;
in response to determining the first accuracy is greater than the second accuracy, select the first target position to be the final target position;
in response to determining the second accuracy is greater than the first accuracy, select the second target position as the final target position; and
control at least one vehicle system based at least in part on the final target position.

12 . The vehicle of claim 11 , wherein the machine-readable instructions further cause the one or more

Assignees

• Toyota Res Inst Inc

Inventors

• Yamaoka Masaaki
• Horiuchi Yuki

Classifications

• G06F3/013 Primary

  Eye tracking input arrangements (G06F3/015 takes precedence) · CPC title

• G06F2203/0381

  Multimodal input, i.e. interface arrangements enabling the user to issue commands by simultaneous use of input devices of different nature, e.g. voice plus gesture on digitizer · CPC title

• B60W2540/00

  Input parameters relating to occupants · CPC title

• B60W50/10

  Interpretation of driver requests or demands · CPC title

• G06F3/017 Primary

  Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883) · CPC title