Vehicle systems and methods for determining a target based on a virtual eye position and a pointing direction

Patent metadata

Field	Value
Publication number	US-10706300-B2
Application number	US-201815877906-A
Country	US
Kind code	B2
Filing date	Jan 23, 2018
Priority date	Jan 23, 2018
Publication date	Jul 7, 2020
Grant date	Jul 7, 2020

Abstract

Vehicle systems and methods for determining a target position are disclosed. A vehicle includes a user detection system configured to output a gesture signal in response to a hand of a user performing at least one gesture to indicate a final target position. The vehicle also includes a user gaze monitoring system configured to output an eye location signal that indicates an actual eye position of the user. The vehicle also includes one or more processors and one or more non-transitory memory modules communicatively coupled to the processors. The processors store machine-readable instructions that, when executed, cause the one or more processors to determine a first point and a second point located on the hand of the user based at least in part on the gesture signal from the user detection system. The first point and the second point define a pointing axis of the hand of the user.

First claim

The invention claimed is:

1. A vehicle, comprising:
a user detection system configured to output a gesture signal in response to a hand of a user performing at least one gesture to indicate a final target position;
a user gaze monitoring system configured to output an eye location signal that indicates an actual eye position of the user;
one or more processors; and
one or more non-transitory memory modules communicatively coupled to the one or more processors and storing machine-readable instructions that, when executed, cause the one or more processors to perform at least the following:
determine a first point and a second point located on the hand of the user based at least in part on the gesture signal from the user detection system, wherein the first point and the second point define a pointing axis of the hand of the user;
calculate a virtual eye position based at least in part on the first point located on the hand of the user and the actual eye position;
determine a midpoint measured between a pair of virtual eyes;
determine a vector that originates at the midpoint and intersects the first point located on the hand of the user, wherein the vector represents a virtual gaze direction of the user;
calculate a first target position based on the virtual eye position;
calculate a second target position based on the pointing axis of the hand of the user;
determine the final target position based on the first target position and the second target position; and
control at least one vehicle system based at least in part on the final target position.

2. The vehicle of claim 1 , wherein the machine-readable instructions further cause the one or more processors to:
determine a presence of an object located in an environment surrounding the vehicle that intersects with the vector representing the virtual gaze direction; and
identify the object that intersects with the vector as the first target position.

3. The vehicle of claim 1 , wherein the machine-readable instructions further cause the one or more processors to:
calculate an angle between two respective vectors that intersect one another, wherein a first vector represents the virtual gaze direction and a second vector represents the pointing axis of the hand of the user; and
calculate an intersecting vector by dividing the angle into a first sub-angle and a second sub-angle, wherein the intersecting vector is directed towards the final target position.

4. A vehicle, comprising:
a user detection system configured to output a gesture signal in response to a hand of a user performing at least one gesture to indicate a final target position;
a user gaze monitoring system configured to output an eye location signal that indicates an actual eye position of the user;
one or more processors; and
one or more non-transitory memory modules communicatively coupled to the one or more processors and storing machine-readable instructions that, when executed, cause the one or more processors to perform at least the following:
determine a first point and a second point located on the hand of the user based at least in part on the gesture signal from the user detection system, wherein the first point and the second point define a pointing axis of the hand of the user;
determine a vector by extending a line segment representing the pointing axis beyond the hand of the user, wherein the vector represents a pointing direction;
determine a presence of an object located in an environment surrounding the vehicle that intersects with the vector representing the pointing direction;
calculate a virtual eye position based at least in part on the first point located on the hand of the user and the actual eye position;
calculate a first target position based on the virtual eye position;
calculate a second target position based on the pointing axis of the hand of the user;
identify the object that intersects with the vector as the second target position;
determine the final target position based on the first target position and the second target position; and
control at least one vehicle system based at least in part on the final target position.

5. The vehicle of claim 4 , wherein the machine-readable instructions further cause the one or more processors to:
determine that the user is pointing towards the second target position by the hand, wherein the hand includes a plurality of digits that each define a respective tip;
set the first point located on the hand of the user to the respective tip of a digit; and
determine the vector that represents the pointing direction by extending the line segment beyond the first point of the hand of the user.

6. The vehicle of claim 4 , wherein the machine-readable instructions further cause the one or more processors to:
determine that the user is pointing towards the second target position by an article defining a proximate end and a distal end, wherein the user grasps the proximate end of the article with the hand and the distal end of the article is directed towards the second target position;
set the first point located on the hand of the user as the distal end of the article; and
determine the vector that represents the pointing direction by extending the line segment beyond the first point.

7. A method of determining a final target position that a user of a vehicle is gesturing towards, the method comprising:
determining, by a computer, a first point and a second point located on a hand of the user based at least in part on a gesture signal generated by a user detection system, wherein the first point and the second point define a pointing axis of the hand of the user;
calculating a virtual eye position based at least in part on the first point located on the hand of the user and an actual eye position of the user, wherein an eye location signal generated by a user gaze monitoring system generates the actual eye position;
determining a midpoint measured between a pair of virtual eyes;
determining a vector that originates at the midpoint and intersects the first point located on the hand of the user, wherein the vector represents a virtual gaze direction of the user;
calculating, by the computer, a first target position based on the virtual eye position;
calculating, by the computer, a second target position based on the pointing axis of the hand of the user;
determining the final target position based on the first target position and the second target position by the computer; and
controlling at least one vehicle system based at least in part on the final target position.

8. The method of claim 7 , further comprising:
determining a presence of an object located in an environment surrounding the vehicle that intersects with the vector representing the virtual gaze direction; and
identifying the object that intersects with the vector as the first target position.

9. The method of claim 7 , further comprising:
calculating an angle between two respective vectors that intersect one another, wherein a first vector represents the virtual gaze direction and a second vector represents the pointing axis of the hand of the user; and
calculating an intersecting vector by dividing the angle into a first sub-angle and a second sub-angle, wherein the intersecting vector is directed towards the final target position.

Assignees

• Toyota Res Inst Inc

Inventors

• Yamaoka Masaaki
• Horiuchi Yuki

Classifications

• B60K35/20

  Output arrangements, i.e. from vehicle to user, associated with vehicle functions or specially adapted therefor · CPC title

• B60K2360/149

  Instrument input by detecting viewing direction not otherwise provided for · CPC title

• B60K35/10 Primary

  Input arrangements, i.e. from user to vehicle, associated with vehicle functions or specially adapted therefor · CPC title

• G06V40/113

  Recognition of static hand signs · CPC title

• G06V20/597 Primary

  Recognising the driver's state or behaviour, e.g. attention or drowsiness · CPC title