User attention-based user experience

Patent metadata

Field	Value
Publication number	US-12055994-B2
Application number	US-202318141019-A
Country	US
Kind code	B2
Filing date	Apr 28, 2023
Priority date	Dec 27, 2019
Publication date	Aug 6, 2024
Grant date	Aug 6, 2024

Abstract

Various systems and methods for adapting a computer based on user attentiveness are described herein. A system for attention-based gesture recognition includes processing circuitry to: access an image of a user, the user proximate to a computing device; determine, based on the image, whether user is attentive to the computing device; and selectively enable or disable a function of the computing device depending on whether the user is attentive.

First claim

What is claimed is:

1. At least one non-transitory machine-readable medium including instructions, which when executed by processing circuitry of a computing device, cause the processing circuitry to perform operations that:
obtain image data of a user;
determine, based on the image data, whether the user is looking at a display screen of the computing device; and
selectively activate or disable use of a voice command to control the computing device depending on whether the user is looking at the display screen, wherein to selectively activate or disable the use of the voice command, includes to:
activate the use of the voice command in response to a determination that the user is looking at the display screen; and
disable the use of the voice command in response to a determination that the user is looking away from the display screen;
wherein the use of the voice command causes an action to be performed in a user interface of the computing device.

2. The non-transitory machine-readable medium of claim 1 , wherein to determine whether the user is looking at the display screen of the computing device, the processing circuitry is to:
determine a direction of an eye gaze of the user, with respect to the computing device; and
determine that the user is looking at the display screen based on the direction of the eye gaze of the user with respect to the computing device.

3. The non-transitory machine-readable medium of claim 1 , wherein to activate the use of the voice command, in response to the determination that the user is looking at the display screen, causes the processing circuitry to identify the voice command from audio data captured with a microphone of the computing device.

4. The non-transitory machine-readable medium of claim 1 , wherein to disable the use of the voice command, in response to the determination that the user is looking away from the display screen, causes the processing circuitry to refrain from identifying the voice command from audio data captured with a microphone of the computing device.

5. The non-transitory machine-readable medium of claim 1 , wherein the image data of the user is provided from a video captured by a user-facing camera of the computing device.

6. The non-transitory machine-readable medium of claim 1 , wherein the action to be performed in the user interface corresponds to a gesture usable in the user interface.

7. The non-transitory machine-readable medium of claim 1 , wherein the action to be performed in the user interface comprises a command to select a user interface object displayed in the user interface.

8. The non-transitory machine-readable medium of claim 1 , wherein the image data of the user is provided from a picture captured by a user-facing camera of the computing device.

9. A computing device, comprising:
a display screen to display a user interface;
a front-facing camera to obtain image data of a user;
a microphone to capture audio data from the user; and
processing circuitry configured to:
determine, based on the image data, whether the user is looking at the display screen of the computing device; and
selectively activate or disable use of a voice command to control the computing device depending on whether the user is looking at the display screen, wherein to selectively activate or disable the use of the voice command, includes to:
activate the use of the voice command in response to a determination that the user is looking at the display screen; and
disable the use of the voice command in response to a determination that the user is looking away from the display screen;
wherein the use of voice command causes an action to be performed in the user interface of the computing device.

10. The computing device of claim 9 , wherein to determine whether the user is looking at the display screen of the computing device, the processing circuitry is to:
determine a direction of an eye gaze of the user, with respect to the computing device; and
determine that the user is looking at the display screen based on the direction of the eye gaze of the user with respect to the computing device.

11. The computing device of claim 9 , wherein to activate the use of the voice command, in response to the determination that the user is looking at the display screen, causes the processing circuitry to identify the voice command from the audio data captured with the microphone.

12. The computing device of claim 9 , wherein to disable the use of the voice command, in response to the determination that the user is looking away from the display screen, causes the processing circuitry to refrain from identifying the voice command from the audio data captured with the microphone.

13. The computing device of claim 9 , wherein the image data of the user is provided from a video captured by the front-facing camera of the computing device.

14. The computing device of claim 9 , wherein the image data of the user is provided from a picture captured by the front-facing camera of the computing device.

15. An apparatus, comprising:
a display screen for displaying a user interface;
means for capturing image data of a user;
means for capturing audio data from the user; and
means for:
determining, based on the image data, whether the user is looking at the display screen; and
selectively activating or disabling use of a voice command to control the apparatus depending on whether the user is looking at the display screen, wherein to selectively activate or disable the use of the voice command, includes:
activating the use of the voice command in response to a determination that the user is looking at the display screen; and
disabling the use of the voice command in response to a determination that the user is looking away from the display scree;
wherein the use of the voice command causes an action to be performed in the user interface of the apparatus.

16. The apparatus of claim 15 , wherein determining whether the user is looking at the display screen, includes:
determining a direction of an eye gaze of the user, with respect to the display screen; and
determining that the user is looking at the display screen based on the direction of the eye gaze of the user with respect to the display screen.

17. The apparatus of claim 15 , wherein activating the use of the voice command, in response to the determination that the user is looking at the display screen, causes the apparatus to identify the voice command from the audio data.

18. The apparatus of claim 15 , wherein disabling the use of the voice command, in response to the determination that the user is looking away from the display screen, causes the apparatus to refrain from identifying the voice command from the audio data.

19. The apparatus of claim 15 , wherein the image data of the user comprises a video or a picture.

Assignees

• Intel Corp

Inventors

• Magi Aleksander
• Bui Kathy
• Diefenbaugh Paul
• Bartscherer Marko

Classifications

• G06F3/013

  Eye tracking input arrangements (G06F3/015 takes precedence) · CPC title

• G06F3/017

  Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883) · CPC title

• G06F1/3231

  Monitoring the presence, absence or movement of users · CPC title

• Y02D10/00

  Energy efficient computing, e.g. low power processors, power management or thermal management · CPC title

• G06F3/012

  Head tracking input arrangements · CPC title