Systems, methods, and apparatus for controlling devices based on a detected gaze

Patent metadata

Field	Value
Publication number	US-9517776-B2
Application number	US-201113977516-A
Country	US
Kind code	B2
Filing date	Dec 29, 2011
Priority date	Dec 29, 2011
Publication date	Dec 13, 2016
Grant date	Dec 13, 2016

Abstract

Certain embodiments of the invention may include systems, methods, and apparatus for controlling devices based on a detected gaze. According to an example embodiment of the invention, a method is provided for executing computer executable instructions by one or more processors for controlling one or more devices attached to a vehicle based at least in part on a detected direction of observation of an occupant of the vehicle, the method further includes receiving one or more images from at least one camera attached to the vehicle; locating, from the one or more images, one or more body features associated with the occupant; determining a direction of gaze from the one or more located body features; and controlling the one or more devices based at least in part on the direction of gaze.

First claim

The claimed invention is:

1. A method comprising executing computer executable instructions by one or more processors for controlling at least a first device and a second device attached to a vehicle based at least in part on a detected direction of observation of an occupant of the vehicle, the method further comprising:
receiving one or more images from at least one camera attached to the vehicle, the vehicle including an object comprising a first portion associated with the first device and a second portion associated with the second device;
locating, from the one or more images, one or more body features associated with the occupant;
determining a direction of gaze towards the object using the one or more located body features; and
controlling a first output of the first device associated with the first portion and controlling a second output of the second device associated with the second portion, based on the determined direction of gaze being towards the first portion, wherein controlling the second output of the second device comprises altering the second output.

2. The method of claim 1 , wherein the locating of the one or more body features comprises locating at least a head associated with the occupant.

3. The method of claim 1 , wherein the locating of the one or more body features comprises locating one or more eyes associated with the occupant.

4. The method of claim 1 , further comprising determining a gesture of the occupant from the one or more body features and controlling at least one of the first device and the second device based at least in part on the gesture.

5. The method of claim 4 , wherein the gesture comprises one or more of a hand movement or a voice command.

6. The method of claim 1 , wherein altering the second output comprises one or more of adjusting a brightness or contrast of at least a portion of a display, actuating a motor, adjusting a control setting, or prompting the occupant with audible or visual information.

7. The method of claim 1 , wherein determining the direction of gaze comprises tracking the occupant's head attitude duration and stability.

8. The method of claim 1 , wherein the first device and the second device comprise an integrated device.

9. A vehicle comprising:
a first device attached to the vehicle;
a second device attached to the vehicle;
at least one camera attached to the vehicle, the vehicle including an object comprising a first portion associated with the first device and a second portion associated with the second device;
at least one memory for storing data and computer-executable instructions; and
one or more processors configured to access the at least one memory and further configured to execute computer executable instructions for:
receiving one or more images from at least one camera attached to the vehicle;
locating, from the one or more images, one or more body features associated with an occupant of the vehicle;
determining a direction of gaze towards the object using the one or more located body features; and
controlling a first output of the first device associated with the first portion and controlling a second output of the second device associated with the second portion, based on the determined direction of gaze being towards the first portion, wherein controlling the second output of the second device comprises altering the second output.

10. The vehicle of claim 9 , wherein the locating of the one or more body features comprises locating at least a head associated with the occupant.

11. The vehicle of claim 9 , wherein the locating of the one or more body features comprises locating one or more eyes associated with the occupant.

12. The vehicle of claim 9 , wherein the one or more processors are further configured for determining a gesture of the occupant from the one or more body features and controlling at least one of the first device and the second device based at least in part on the gesture.

13. The vehicle of claim 12 , wherein the gesture comprises one or more of a hand movement or a voice command.

14. The vehicle of claim 9 , wherein altering the second output comprises one or more of adjusting a brightness or contrast of at least a portion of a display, actuating a motor, adjusting a control setting, or prompting the occupant with audible or visual information.

15. The vehicle of claim 9 , wherein determining the direction of gaze comprises tracking the occupant's head attitude duration and stability.

16. The vehicle of claim 9 , wherein the at least one of the first device and the second device comprises one or more of a video screen, a mirror with an integrated display, a stereo system, a sun roof, a car window, a lock, a car seat, a climate control system, or a dashboard console.

17. The vehicle of claim 9 , wherein the first device and the second device comprise an integrated device.

18. An apparatus comprising:
at least one memory for storing data and computer-executable instructions; and
one or more processors configured to access the at least one memory and further configured to execute computer executable instructions for:
receiving one or more images from at least one camera attached to the vehicle, the vehicle including an object comprising a first portion associated with a first device and a second portion associated with a second device;
locating, from the one or more images, one or more body features associated with an occupant of the vehicle;
determining a direction of gaze towards the object using the one or more located body features; and
controlling a first output of the first device associated with the first portion and controlling a second output of the second device associated with the second portion, based on the determined direction of gaze being towards the first portion, wherein controlling the second output of the second device comprises altering the second output.

19. The apparatus of claim 18 , wherein the locating of the one or more body features comprises locating at least a head associated with the occupant.

20. The apparatus of claim 18 , wherein the locating of the one or more body features comprises locating one or more eyes associated with the occupant.

21. The apparatus of claim 18 , wherein the one or more processors are further configured for determining a gesture of the occupant from the one or more body features and controlling at least one of the first device and the second device based at least in part on the gesture.

22. The apparatus of claim 21 , wherein the gesture comprises one or more of a hand movement or a voice command.

23. The apparatus of claim 18 , wherein altering the second output comprises one or more of adjusting a brightness or contrast of at least a portion of a display, actuating a motor, adjusting a control setting, or prompting the occupant with audible or visual information.

24. The apparatus of claim 18 , wherein determining the direction of gaze comprises tracking the occupant's head attitude duration and stability.

25. The apparatus of claim 18 , wherein at least one of the first device and the second device comprises one or more of a video screen, a mirror with an integrated display, a stereo system, a sun roof, a car window, a lock, a car seat, a climate control system, or a dashboard console.

26. The apparatus of claim 18 , wherein the first device and the second device comprise an integrated device.

2

Assignees

• Graumann David L
• Intel Corp

Inventors

• Graumann David L

Classifications

• B60W50/08

  Interaction between the driver and the control system · CPC title

• B60W2540/22

  Psychological state; Stress level or workload · CPC title

• B60W50/10 Primary

  Interpretation of driver requests or demands · CPC title

• G06F3/013

  Eye tracking input arrangements (G06F3/015 takes precedence) · CPC title

• G06F3/012

  Head tracking input arrangements · CPC title